From janssen at parc.com  Sun Mar  1 00:33:17 2009
From: janssen at parc.com (Bill Janssen)
Date: Sat, 28 Feb 2009 15:33:17 PST
Subject: [Python-Dev] More on Py3K urllib -- urlencode()
In-Reply-To: <34291.1235858913@parc.com>
References: <49A99E7B.7030602@digidescorp.com> <34291.1235858913@parc.com>
Message-ID: <35441.1235863997@parc.com>

Bill Janssen <janssen at parc.com> wrote:

> Dan Mahn <dan.mahn at digidescorp.com> wrote:
> 
> > 3) Regarding the following code fragment in urlencode():
> > 
> >            k = quote_plus(str(k))
> >           if isinstance(v, str):
> >                v = quote_plus(v)
> >                l.append(k + '=' + v)
> >            elif isinstance(v, str):
> >                # is there a reasonable way to convert to ASCII?
> >                # encode generates a string, but "replace" or "ignore"
> >                # lose information and "strict" can raise UnicodeError
> >                v = quote_plus(v.encode("ASCII","replace"))
> >                l.append(k + '=' + v)
> > 
> > I don't understand how the "elif" section is invoked, as it uses the
> > same condition as the "if" section.
> 
> This looks like a 2->3 bug; clearly only the second branch should be
> used in Py3K.  And that "replace" is also a bug; it should signal an
> error on encoding failures.  It should probably catch UnicodeError and
> explain the problem, which is that only Latin-1 values can be passed in
> the query string.  So the encode() to "ASCII" is also a mistake; it
> should be "ISO-8859-1", and the "replace" should be a "strict", I think.

Sorry!  In 3.0.1, this whole thing boils down to

   l.append(quote_plus(k) + '=' + quote_plus(v))

Bill

From dan.mahn at digidescorp.com  Sun Mar  1 01:24:15 2009
From: dan.mahn at digidescorp.com (Dan Mahn)
Date: Sat, 28 Feb 2009 18:24:15 -0600
Subject: [Python-Dev] More on Py3K urllib -- urlencode()
In-Reply-To: <35441.1235863997@parc.com>
References: <49A99E7B.7030602@digidescorp.com> <34291.1235858913@parc.com>
	<35441.1235863997@parc.com>
Message-ID: <49A9D5AF.4050801@digidescorp.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090228/ae7938b1/attachment.htm>

From steve at integrityintegrators.net  Sun Mar  1 02:56:36 2009
From: steve at integrityintegrators.net (Heracles)
Date: Sat, 28 Feb 2009 17:56:36 -0800 (PST)
Subject: [Python-Dev] Patch Ready for review
In-Reply-To: <49A85269.4000601@v.loewis.de>
References: <22254109.post@talk.nabble.com> <49A85269.4000601@v.loewis.de>
Message-ID: <22269124.post@talk.nabble.com>


Well, what got me interested in Python was the fact that it had support for
reading mail files and for the curses library.  When I found out there was a
need for folks to work on this module I thought I would start contributing,
I would love to become a primary maintainer of this particular module,
because I think writing console apps is cool (yes I know my inner nerd is
showing).

Maybe someday I will also create a computational geometry module that can be
plugged in to Python as well, but for the time being I think I could handle
working on this module.



&quot;Martin v. L?wis&quot; wrote:
> 
>> I am wondering what to expect next.  How long will it be before it is
>> applied etc.  This is my first attempt to submit a patch to Python.
> 
> Unfortunately, it may take any time between a day and five years, see
> below.
> 
>> Also, does anyone know who the main person is for running changes to the
>> curses library I wanted to get guidance for implementing support for
>> family
>> of functions newterm, set_term, reset_term.
> 
> The curses module is largely unmaintained; nobody is really responsible
> for it (my feeling is that it also has fairly few users). Use "svn log"
> to find out what people have been making changes over time; if the
> commit messages indicate that they had applied patches from the bug
> tracker, also try to recognize any regular non-commit contributors.
> 
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/lists%40nabble.com
> 
> 

-- 
View this message in context: http://www.nabble.com/Patch-Ready-for-review-tp22254109p22269124.html
Sent from the Python - python-dev mailing list archive at Nabble.com.


From ocean-city at m2.ccsnet.ne.jp  Sun Mar  1 19:45:44 2009
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Mon, 02 Mar 2009 03:45:44 +0900
Subject: [Python-Dev] What type of object mmap.read_byte should return
 on py3k?
In-Reply-To: <49A956F0.9050700@m2.ccsnet.ne.jp>
References: <49A91DA4.5080903@m2.ccsnet.ne.jp>	<200902281521.48100.victor.stinner@haypocalc.com>
	<49A956F0.9050700@m2.ccsnet.ne.jp>
Message-ID: <49AAD7D8.9080301@m2.ccsnet.ne.jp>

I uploaded the patch with choice (a)
http://bugs.python.org/file13215/py3k_mmap_and_bytes.patch
If (b) is suitable, I'll rewrite the patch.

From armin.ronacher at active-4.com  Sun Mar  1 20:13:27 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Sun, 1 Mar 2009 19:13:27 +0000 (UTC)
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to collections
	ready for pronouncement
Message-ID: <loom.20090301T182201-999@post.gmane.org>

Hi everybody,

PEP 372 was modified so that it provides a simpler API (only the dict API
to be exact) and it was decided to start with a Python-only implementation
and replace it with a C version later if necessary.

Annotated changes from earlier versions of the PEP:

-   the extra API for ordered dict was dropped to keep the interface
    simple and clean.  Future versions can still be expanded but it's
    impossible to drop features later on.

-   To keep the implementation simple 3.1 / 2.7 will ship with a
    Python-only version of the class.  It can still be rewritten in
    C if it turns out to be too slow or thread safety is required.

The corresponding issue in the tracker: http://bugs.python.org/issue5397
Link to the PEP: http://www.python.org/dev/peps/pep-0372/

Anything else that should be done?


Regards,
Armin


From p.f.moore at gmail.com  Sun Mar  1 21:58:22 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 1 Mar 2009 20:58:22 +0000
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <1afaf6160902271541y6e6d5937ieab4c3b0f111a5cc@mail.gmail.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<loom.20090227T211114-386@post.gmane.org>
	<24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1>
	<1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com>
	<49A8781A.60309@gmail.com>
	<1afaf6160902271541y6e6d5937ieab4c3b0f111a5cc@mail.gmail.com>
Message-ID: <79990c6b0903011258i4e64c990s9d29107646ac5372@mail.gmail.com>

2009/2/27 Benjamin Peterson <benjamin at python.org>:
> 2009/2/27 Nick Coghlan <ncoghlan at gmail.com> schrieb:
>>
>> I should have a PEP (and implementation) ready for alpha 2 to address
>> the current discrepancy between contextlib.nested and actual nested with
>> statements:
>> http://bugs.python.org/issue5251
>>
>> If you do add a reference to that bug report to the release PEP, mark
>> fixing it as a maybe though - with the associated PEP not even written
>> yet, I obviously still have some work to do to get the semantic change
>> approved by Guido and the rest of python-dev.
>
> Ok. I've added it.

Is it worth getting simplegeneric exposed in 3.1
(http://bugs.python.org/issue5135)? If it's going to be in 2.7, I'd
like to see it hit 3.1. The patch is against trunk (for 2.7) at the
moment, I'm not sure what the process would be for forward-porting it
(do I generate a new patch against the py3k branch, or should it be
applied to trunk and merged in?)

Paul.

From ncoghlan at gmail.com  Sun Mar  1 22:42:20 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 02 Mar 2009 07:42:20 +1000
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <79990c6b0903011258i4e64c990s9d29107646ac5372@mail.gmail.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>	
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>	
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>	
	<loom.20090227T211114-386@post.gmane.org>	
	<24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1>	
	<1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com>	
	<49A8781A.60309@gmail.com>	
	<1afaf6160902271541y6e6d5937ieab4c3b0f111a5cc@mail.gmail.com>
	<79990c6b0903011258i4e64c990s9d29107646ac5372@mail.gmail.com>
Message-ID: <49AB013C.4040409@gmail.com>

Paul Moore wrote:
> 2009/2/27 Benjamin Peterson <benjamin at python.org>:
>> 2009/2/27 Nick Coghlan <ncoghlan at gmail.com> schrieb:
>>> I should have a PEP (and implementation) ready for alpha 2 to address
>>> the current discrepancy between contextlib.nested and actual nested with
>>> statements:
>>> http://bugs.python.org/issue5251
>>>
>>> If you do add a reference to that bug report to the release PEP, mark
>>> fixing it as a maybe though - with the associated PEP not even written
>>> yet, I obviously still have some work to do to get the semantic change
>>> approved by Guido and the rest of python-dev.
>> Ok. I've added it.
> 
> Is it worth getting simplegeneric exposed in 3.1
> (http://bugs.python.org/issue5135)? If it's going to be in 2.7, I'd
> like to see it hit 3.1. The patch is against trunk (for 2.7) at the
> moment, I'm not sure what the process would be for forward-porting it
> (do I generate a new patch against the py3k branch, or should it be
> applied to trunk and merged in?)

As much as I'd like to get a simple generic implementation into
functools, the lack of support for ABCs still bothers me (despite my
last post about that on the tracker item). I'd be a -0 on it going in as
is, but if someone can figure out a clever way of supporting ABCs
without completing killing the invocation speed for generics, that would
go up to a +1.

(The current difficulty of this may actually reflect a more significant
limitation on the available metadata for ABCs in PEP 3119: it is easy to
ask "is this specific type an example of this ABC?", but difficult to
ask "which ABCs is this type as example of?". For actual inheritance,
the __mro__ attribute means that both questions are easy to answer, but
I'm not aware of any corresponding way of answering the latter question
for ABCs)

Cheers,
Nick.

P.S. I just unassigned myself from that tracker item - I'm going to have
my hands full working on the proposed change to the with statement.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From cesare.dimauro at a-tono.com  Sun Mar  1 23:04:08 2009
From: cesare.dimauro at a-tono.com (Cesare Di Mauro)
Date: Sun, 1 Mar 2009 23:04:08 +0100 (CET)
Subject: [Python-Dev] test_io fails on test_1686475
In-Reply-To: <49AB013C.4040409@gmail.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>	
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>	
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>	
	<loom.20090227T211114-386@post.gmane.org>	
	<24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1>	
	<1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com>	
	<49A8781A.60309@gmail.com>	
	<1afaf6160902271541y6e6d5937ieab4c3b0f111a5cc@mail.gmail.com>
	<79990c6b0903011258i4e64c990s9d29107646ac5372@mail.gmail.com>
	<49AB013C.4040409@gmail.com>
Message-ID: <55961.151.53.158.182.1235945048.squirrel@webmail8.pair.com>

Running the test suite with Python 2.6.1 32 bit (compiled in DEBUG mode
with Visual Studio Express Edition 2008) on Vista x64, I've got an assert
error:

test_1686475 (__main__.StatAttributeTests) ... Assertion failed:
(__int64)(int)((in / 10000000) - secs_between_epochs) == ((in / 10000000)
- secs_between_epochs), file ..\Modules\posixmodule.c, line 790

I have no idea about this failure. Any hint?

Cheers,
Cesare

From robertc at robertcollins.net  Sun Mar  1 23:43:47 2009
From: robertc at robertcollins.net (Robert Collins)
Date: Mon, 02 Mar 2009 09:43:47 +1100
Subject: [Python-Dev] test_io fails on test_1686475
In-Reply-To: <55961.151.53.158.182.1235945048.squirrel@webmail8.pair.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<loom.20090227T211114-386@post.gmane.org>
	<24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1>
	<1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com>
	<49A8781A.60309@gmail.com>
	<1afaf6160902271541y6e6d5937ieab4c3b0f111a5cc@mail.gmail.com>
	<79990c6b0903011258i4e64c990s9d29107646ac5372@mail.gmail.com>
	<49AB013C.4040409@gmail.com>
	<55961.151.53.158.182.1235945048.squirrel@webmail8.pair.com>
Message-ID: <1235947427.27820.47.camel@lifeless-64>

On Sun, 2009-03-01 at 23:04 +0100, Cesare Di Mauro wrote:
> Running the test suite with Python 2.6.1 32 bit (compiled in DEBUG mode
> with Visual Studio Express Edition 2008) on Vista x64, I've got an assert
> error:
> 
> test_1686475 (__main__.StatAttributeTests) ... Assertion failed:
> (__int64)(int)((in / 10000000) - secs_between_epochs) == ((in / 10000000)
> - secs_between_epochs), file ..\Modules\posixmodule.c, line 790
> 
> I have no idea about this failure. Any hint?

[I haven't checked the test yet, let me know if my guess is wrong :)].
It looks to me that its asserting something to do with stat times; note
that windows has up to a 2 second granularity for stat times on files -
you may be hitting some code that assumes a sane file system :)

-Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090302/940ff778/attachment.pgp>

From amauryfa at gmail.com  Mon Mar  2 00:13:46 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 2 Mar 2009 00:13:46 +0100
Subject: [Python-Dev] test_io fails on test_1686475
In-Reply-To: <55961.151.53.158.182.1235945048.squirrel@webmail8.pair.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<loom.20090227T211114-386@post.gmane.org>
	<24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1>
	<1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com>
	<49A8781A.60309@gmail.com>
	<1afaf6160902271541y6e6d5937ieab4c3b0f111a5cc@mail.gmail.com>
	<79990c6b0903011258i4e64c990s9d29107646ac5372@mail.gmail.com>
	<49AB013C.4040409@gmail.com>
	<55961.151.53.158.182.1235945048.squirrel@webmail8.pair.com>
Message-ID: <e27efe130903011513s4d383190rbf1ab9dec159f69a@mail.gmail.com>

Hello,

On Sun, Mar 1, 2009 at 23:04, Cesare Di Mauro <cesare.dimauro at a-tono.com> wrote:
> Running the test suite with Python 2.6.1 32 bit (compiled in DEBUG mode
> with Visual Studio Express Edition 2008) on Vista x64, I've got an assert
> error:
>
> test_1686475 (__main__.StatAttributeTests) ... Assertion failed:
> (__int64)(int)((in / 10000000) - secs_between_epochs) == ((in / 10000000)
> - secs_between_epochs), file ..\Modules\posixmodule.c, line 790
>
> I have no idea about this failure. Any hint?

The failing assertion comes from this code in posixmodule.c:

	/* XXX Win32 supports time stamps past 2038; we currently don't */
	*time_out = Py_SAFE_DOWNCAST((in / 10000000) - secs_between_epochs,
__int64, int);

the test (btw, it's in test_os.py) is trying
    os.stat(r"c:\pagefile.sys")

Can you please check the three time stamps of this file (creation,
update, access)?

-- 
Amaury Forgeot d'Arc

From benjamin at python.org  Mon Mar  2 02:23:20 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 1 Mar 2009 19:23:20 -0600
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <79990c6b0903011258i4e64c990s9d29107646ac5372@mail.gmail.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<loom.20090227T211114-386@post.gmane.org>
	<24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1>
	<1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com>
	<49A8781A.60309@gmail.com>
	<1afaf6160902271541y6e6d5937ieab4c3b0f111a5cc@mail.gmail.com>
	<79990c6b0903011258i4e64c990s9d29107646ac5372@mail.gmail.com>
Message-ID: <1afaf6160903011723u68bb0f50l74f9ea6e6b122a3@mail.gmail.com>

2009/3/1 Paul Moore <p.f.moore at gmail.com>:
>
> Is it worth getting simplegeneric exposed in 3.1
> (http://bugs.python.org/issue5135)? If it's going to be in 2.7, I'd
> like to see it hit 3.1. The patch is against trunk (for 2.7) at the
> moment, I'm not sure what the process would be for forward-porting it
> (do I generate a new patch against the py3k branch, or should it be
> applied to trunk and merged in?)

Patches to the trunk are fine.

As for the actual feature, I don't think it should hold up releases.


-- 
Regards,
Benjamin

From exarkun at divmod.com  Mon Mar  2 02:33:10 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Sun, 1 Mar 2009 20:33:10 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <loom.20090301T182201-999@post.gmane.org>
Message-ID: <20090302013310.12853.2036000890.divmod.quotient.16164@henry.divmod.com>

On Sun, 1 Mar 2009 19:13:27 +0000 (UTC), Armin Ronacher <armin.ronacher at active-4.com> wrote:
>Hi everybody,
>
>PEP 372 was modified so that it provides a simpler API (only the dict API
>to be exact) and it was decided to start with a Python-only implementation
>and replace it with a C version later if necessary.
>
>Annotated changes from earlier versions of the PEP:
>
>-   the extra API for ordered dict was dropped to keep the interface
>    simple and clean.  Future versions can still be expanded but it's
>    impossible to drop features later on.

Keeping the API simple and clean sounds great.  I'm all in favor of this.
However, it does no one a service to continue to propagate the idea that
all the code written for Python always has to be perfect.  It's feasible
and even simple to drop features later on, should it turn out to be that
they are not desirable.

Jean-Paul

From python at rcn.com  Mon Mar  2 07:47:26 2009
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 1 Mar 2009 22:47:26 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collectionsready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org>
Message-ID: <EB2271A284C84B41ADA09A41F424ECB1@RaymondLaptop1>


[Armin Ronacher]
> PEP 372 was modified so that it provides a simpler API (only the dict API
> to be exact) and it was decided to start with a Python-only implementation
> and replace it with a C version later if necessary.
> 
> Annotated changes from earlier versions of the PEP:
> 
> -   the extra API for ordered dict was dropped to keep the interface
>    simple and clean.  Future versions can still be expanded but it's
>    impossible to drop features later on.
> 
> -   To keep the implementation simple 3.1 / 2.7 will ship with a
>    Python-only version of the class.  It can still be rewritten in
>    C if it turns out to be too slow or thread safety is required.
> 
> The corresponding issue in the tracker: http://bugs.python.org/issue5397
> Link to the PEP: http://www.python.org/dev/peps/pep-0372/
> 
> Anything else that should be done?


Guido, I'm recommending this PEP for acceptance.


Raymond

From p.f.moore at gmail.com  Mon Mar  2 11:41:52 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 2 Mar 2009 10:41:52 +0000
Subject: [Python-Dev] ABCs and MRO
Message-ID: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>

2009/3/2 Benjamin Peterson <benjamin at python.org>:
> 2009/3/1 Paul Moore <p.f.moore at gmail.com>:
>>
>> Is it worth getting simplegeneric exposed in 3.1
>> (http://bugs.python.org/issue5135)? If it's going to be in 2.7, I'd
>> like to see it hit 3.1. The patch is against trunk (for 2.7) at the
>> moment, I'm not sure what the process would be for forward-porting it
>> (do I generate a new patch against the py3k branch, or should it be
>> applied to trunk and merged in?)
>
> Patches to the trunk are fine.

OK, thanks.

> As for the actual feature, I don't think it should hold up releases.

Fair enough.

The key problem with the patch is that ABCs do not play well with the
type of introspection required to implement a generic function -
namely enumeration of the superclasses of a class. The MRO of the
class is fine for normal inheritance, but for ABCs it is possible to
register classes which don't inherit from the ABC, so that you have a
situation where issubclass (C, MyABC) can be true without MyABC being
in C.__mro__:

>>> import abc
>>> class MyABC(object):
...     __metaclass__ = abc.ABCMeta
...
>>> class C(object):
...     pass
...
>>> MyABC.register(C)
>>> issubclass(C, MyABC)
True
>>> C.__mro__
(<class '__main__.C'>, <type 'object'>)
>>>

More generally, there is NO WAY to determine the list of classes for
which issubclass(C, x) is true.

This could be considered a limitation of, or a bug in, ABCs, I don't
have a particular opinion on that, but it does mean that no code which
relies on being able to traverse the class inheritance graph will see
ABCs. One particular case of this is (any implementation I can think
of, of) generic functions.

In my view, this implies one of the following:

1) It should be a documented limitation of such code that it doesn't
work with ABCs (and conversely, this limitation of ABCs should be
documented in the ABC documentation)
2) Generic functions, and any other code requiring this type of
introspection, is essentially useless unless it can support ABCs, and
should not be used in the light of this limitation.
3) This is a bug in ABCs and should be fixed.
4) Something else I didn't think of :-)

In my view, (2) is an unreasonable position to take, given the fact
that (as I understand it) ABCs are supposed to be largely optional and
shouldn't affect code that doesn't care about them...

It's not clear to me how (3) should be addressed. Adding a slot to all
classes to hold a list of ABCs they are registered against seems to be
a large overhead for a relatively rarely used feature. I guess having
a global registry of ABC registrations could work, but it seems
clumsy. Any other suggestions?

The problem with (1) is that it simply acknowledges the issue, but
doesn't actually fix it. Nick Coghlan has commented (in the tracker)
that he feels that the limitation is significant (enough to push him
from +1 to -0 on the feature).

I have created an issue in the tracker -
http://bugs.python.org/issue5405. In my view, it's fairly critical,
but I've refrained from setting a high priority as that may just be my
vested interest talking :-)

I'd appreciate it if someone could comment (either here or on the
tracker item) on what the correct way forward should be. If the answer
is "fix the ABC bug", then I'll have a look but I'm not sure if I can
do it without help - any assistance would be appreciated.

Paul.

From cesare.dimauro at a-tono.com  Mon Mar  2 12:59:58 2009
From: cesare.dimauro at a-tono.com (Cesare Di Mauro)
Date: Mon, 02 Mar 2009 12:59:58 +0100
Subject: [Python-Dev] test_io fails on test_1686475
In-Reply-To: <e27efe130903011513s4d383190rbf1ab9dec159f69a@mail.gmail.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<loom.20090227T211114-386@post.gmane.org>
	<24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1>
	<1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com>
	<49A8781A.60309@gmail.com>
	<1afaf6160902271541y6e6d5937ieab4c3b0f111a5cc@mail.gmail.com>
	<79990c6b0903011258i4e64c990s9d29107646ac5372@mail.gmail.com>
	<49AB013C.4040409@gmail.com>
	<55961.151.53.158.182.1235945048.squirrel@webmail8.pair.com>
	<e27efe130903011513s4d383190rbf1ab9dec159f69a@mail.gmail.com>
Message-ID: <op.up5x98f603jqhe@cesareprova.org>

On Mar, 02 2009 at 00:13AM, Amaury Forgeot d'Arc <amauryfa at gmail.com> wrote:

> Hello,
>
> On Sun, Mar 1, 2009 at 23:04, Cesare Di Mauro <cesare.dimauro at a-tono.com> wrote:
>> Running the test suite with Python 2.6.1 32 bit (compiled in DEBUG mode
>> with Visual Studio Express Edition 2008) on Vista x64, I've got an assert
>> error:
>>
>> test_1686475 (__main__.StatAttributeTests) ... Assertion failed:
>> (__int64)(int)((in / 10000000) - secs_between_epochs) == ((in / 10000000)
>> - secs_between_epochs), file ..\Modules\posixmodule.c, line 790
>>
>> I have no idea about this failure. Any hint?
>
> The failing assertion comes from this code in posixmodule.c:
>
> 	/* XXX Win32 supports time stamps past 2038; we currently don't */
> 	*time_out = Py_SAFE_DOWNCAST((in / 10000000) - secs_between_epochs,
> __int64, int);
>
> the test (btw, it's in test_os.py) is trying
>     os.stat(r"c:\pagefile.sys")
>
> Can you please check the three time stamps of this file (creation,
> update, access)?
>

You are right. The last modified timestamp had 2099 as year value (the maximum
that I can set on Windows), because of some tests with dates which I made at the
time.

However, they are correct timestamps for Windows files, so I think that at least
the API on posixmodule.c should not fail when working with them. I don't know if
there's a way to handle them correctly.

Also may be that test_os.py need to be changed in some way, because the
pagefile can be put in any partition, and there are Windows installations which
lack it, because the virtual memory can be disabled too.

Cheers
Cesare

From chris at simplistix.co.uk  Mon Mar  2 13:23:38 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 02 Mar 2009 12:23:38 +0000
Subject: [Python-Dev] How do I get commit access?
In-Reply-To: <18856.9294.373185.493616@montanaro.dyndns.org>
References: <49A7A906.8000008@simplistix.co.uk>
	<0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>
	<49A7AD8C.5030209@simplistix.co.uk>
	<A8EBDAFE66D843DBBAFFD2B942917A5D@RaymondLaptop1>
	<49A7B501.4070508@simplistix.co.uk> <go8ed0$vnq$1@ger.gmane.org>
	<18856.9294.373185.493616@montanaro.dyndns.org>
Message-ID: <49ABCFCA.5020402@simplistix.co.uk>

skip at pobox.com wrote:
>     Christian> CPython has a stricter policy than most other Python related
>     Christian> projects.
> 
> Indeed.  I'd be willing to grant you checkin privileges for SpamBayes simply
> because because Christian recognized you and you seem willing to roll up
> your sleeves. Do you do Windows?  <wink>

The irony that Thunderbird put this in my spam folder based on its 
heuristics is not lost on me ;-)

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From ncoghlan at gmail.com  Mon Mar  2 13:25:41 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 02 Mar 2009 22:25:41 +1000
Subject: [Python-Dev] test_io fails on test_1686475
In-Reply-To: <op.up5x98f603jqhe@cesareprova.org>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>	<loom.20090227T211114-386@post.gmane.org>	<24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1>	<1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com>	<49A8781A.60309@gmail.com>	<1afaf6160902271541y6e6d5937ieab4c3b0f111a5cc@mail.gmail.com>	<79990c6b0903011258i4e64c990s9d29107646ac5372@mail.gmail.com>	<49AB013C.4040409@gmail.com>	<55961.151.53.158.182.1235945048.squirrel@webmail8.pair.com>	<e27efe130903011513s4d383190rbf1ab9dec159f69a@mail.gmail.com>
	<op.up5x98f603jqhe@cesareprova.org>
Message-ID: <49ABD045.2010303@gmail.com>

Cesare Di Mauro wrote:
> However, they are correct timestamps for Windows files, so I think that at least
> the API on posixmodule.c should not fail when working with them. I don't know if
> there's a way to handle them correctly.

Use 64-bit time values (which is highly unlikely to ever be the case for
CPython on a 32-bit OS).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From p.f.moore at gmail.com  Mon Mar  2 14:25:10 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 2 Mar 2009 13:25:10 +0000
Subject: [Python-Dev] ABCs and simplegeneric use case
Message-ID: <79990c6b0903020525x7ce01507reff6481bb9d204c@mail.gmail.com>

2009/3/1 Nick Coghlan <ncoghlan at gmail.com>:
> As much as I'd like to get a simple generic implementation into
> functools, the lack of support for ABCs still bothers me (despite my
> last post about that on the tracker item). I'd be a -0 on it going in as
> is, but if someone can figure out a clever way of supporting ABCs
> without completing killing the invocation speed for generics, that would
> go up to a +1.

Nick,
Armin Ronacher pointed out that it's not likely to be possible to
enumerate ABCs even in theory, as ABCs can do semantic checks (e.g.
checking for the presence of special methods like __iter__) without
needing either inheritance or explicit registration.

As you had a genuine use case for simplegeneric, how badly does this
limitation damage it?

Paul.

From solipsis at pitrou.net  Mon Mar  2 14:56:55 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 2 Mar 2009 13:56:55 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?PEP_372_--_Adding_an_ordered_directory_to?=
	=?utf-8?q?=09collectionsready_for_pronouncement?=
References: <loom.20090301T182201-999@post.gmane.org>
	<EB2271A284C84B41ADA09A41F424ECB1@RaymondLaptop1>
Message-ID: <loom.20090302T135344-234@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:
> 
> Guido, I'm recommending this PEP for acceptance.

Given you were bitten by it in your own unit tests (the "eval(repr()) does not
maintain ordering" problem pointed by Georg), I think it would be better to
reconsider the current __eq__ behaviour, and make it ordering-aware.

Regards

Antoine.



From rameshiitk at rediffmail.com  Mon Mar  2 12:17:05 2009
From: rameshiitk at rediffmail.com (ramesh nayak)
Date: 2 Mar 2009 11:17:05 -0000
Subject: [Python-Dev] help-tkinter
Message-ID: <20090302111705.13090.qmail@f5mail-236-235.rediffmail.com>

Dear Sir,
I have a fortran-95 code for my own mathematical application .

and also I have the *.exe fiel for the same. when I double click it , it will run.

I want to make a GUI for it by using TKinter and run the program through the button on GUI.

Could you please send the syntax for the same?

Thanks in advance for your help.

Thanks.
Regards
Ramesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090302/82f72250/attachment.htm>

From vaibhavmallya at gmail.com  Mon Mar  2 13:23:37 2009
From: vaibhavmallya at gmail.com (Vaibhav Mallya)
Date: Mon, 02 Mar 2009 07:23:37 -0500
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
Message-ID: <49ABCFC9.1070508@gmail.com>

I haven't seen a lot of discussion on this - maybe I didn't search hard 
enough - but  what are people's thoughts on including BeautifulSoup in 
stdlib? It's small, fast, and pretty widely-liked by the people who know 
about it. Someone mentioned that web scraping needs are infrequent. My 
argument is that people ask questions about them less because they feel 
they can just reinvent the wheel really easily using urllib and regexes. 
It seems like this is similar to the CSV problem from a while back 
actually, with everyone implementing their own parsers.

We do have HTMLParser, but that doesn't handle malformed pages well, and 
just isn't as nice as BeautifulSoup.

In a not-entirely-unrelated vein, has there been any discussion on just 
throwing all of Mechanize into stdlib?

BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/
mechanize: http://wwwsearch.sourceforge.net/mechanize/

Regards,
Vaibhav Mallya

From teewey2000 at yahoo.com  Mon Mar  2 11:02:00 2009
From: teewey2000 at yahoo.com (toks teewey)
Date: Mon, 2 Mar 2009 02:02:00 -0800 (PST)
Subject: [Python-Dev] executable
Message-ID: <990226.65536.qm@web59202.mail.re1.yahoo.com>

Hello,

I want to ask if it is possible to link a program written in python to a database and also on how
to make a program written in python executable.

Regards




      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090302/87ec92f9/attachment.htm>

From benjamin at python.org  Mon Mar  2 15:19:21 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 2 Mar 2009 08:19:21 -0600
Subject: [Python-Dev] executable
In-Reply-To: <990226.65536.qm@web59202.mail.re1.yahoo.com>
References: <990226.65536.qm@web59202.mail.re1.yahoo.com>
Message-ID: <1afaf6160903020619p1a42dac9gf3b26ca6fea481f0@mail.gmail.com>

2009/3/2 toks teewey <teewey2000 at yahoo.com>:
> Hello,
>
> I want to ask if it is possible to link a program written in python to a
> database and also on how
> to make a program written in python executable.

This list is for the development of Python, not language questions.
Please mail python-list at python.org



-- 
Regards,
Benjamin

From brett at python.org  Mon Mar  2 15:21:57 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 2 Mar 2009 06:21:57 -0800
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49ABCFC9.1070508@gmail.com>
References: <49ABCFC9.1070508@gmail.com>
Message-ID: <bbaeab100903020621m759d4694y9468bb8e5169fce8@mail.gmail.com>

On Mon, Mar 2, 2009 at 04:23, Vaibhav Mallya <vaibhavmallya at gmail.com>wrote:

> I haven't seen a lot of discussion on this - maybe I didn't search hard
> enough - but  what are people's thoughts on including BeautifulSoup in
> stdlib? It's small, fast, and pretty widely-liked by the people who know
> about it. Someone mentioned that web scraping needs are infrequent. My
> argument is that people ask questions about them less because they feel they
> can just reinvent the wheel really easily using urllib and regexes. It seems
> like this is similar to the CSV problem from a while back actually, with
> everyone implementing their own parsers.
>
> We do have HTMLParser, but that doesn't handle malformed pages well, and
> just isn't as nice as BeautifulSoup.
>
> In a not-entirely-unrelated vein, has there been any discussion on just
> throwing all of Mechanize into stdlib?


Discussions of including modules in the standard library only occurs when
the module creators step forward to offer to support the modules. To my
knowledge neither the creators of BeautifulSoup or Mechanize have come
forward to offer to manage the code in Python's standard library.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090302/a38b8adf/attachment.htm>

From steve at holdenweb.com  Mon Mar  2 15:52:27 2009
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 02 Mar 2009 09:52:27 -0500
Subject: [Python-Dev] help-tkinter
In-Reply-To: <20090302111705.13090.qmail@f5mail-236-235.rediffmail.com>
References: <20090302111705.13090.qmail@f5mail-236-235.rediffmail.com>
Message-ID: <gogrrv$vis$1@ger.gmane.org>

ramesh nayak wrote:
> Dear Sir,
> I have a fortran-95 code for my own mathematical application .
> 
> and also I have the *.exe fiel for the same. when I double click it , it
> will run.
> 
> I want to make a GUI for it by using TKinter and run the program through
> the button on GUI.
> 
> Could you please send the syntax for the same?
> 
This list is for the development *of* Python. Your question would be
better addressed to comp.lang.python (python-list at python dot org).
There are many people on that list who will be able to help you.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From skip at pobox.com  Mon Mar  2 16:27:04 2009
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 2 Mar 2009 09:27:04 -0600
Subject: [Python-Dev] How do I get commit access?
In-Reply-To: <49ABCFCA.5020402@simplistix.co.uk>
References: <49A7A906.8000008@simplistix.co.uk>
	<0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>
	<49A7AD8C.5030209@simplistix.co.uk>
	<A8EBDAFE66D843DBBAFFD2B942917A5D@RaymondLaptop1>
	<49A7B501.4070508@simplistix.co.uk> <go8ed0$vnq$1@ger.gmane.org>
	<18856.9294.373185.493616@montanaro.dyndns.org>
	<49ABCFCA.5020402@simplistix.co.uk>
Message-ID: <18859.64200.859197.70559@montanaro.dyndns.org>


    >> Indeed.  I'd be willing to grant you checkin privileges for SpamBayes
    >> simply because because Christian recognized you and you seem willing
    >> to roll up your sleeves. Do you do Windows?  <wink>

    Chris> The irony that Thunderbird put this in my spam folder based on
    Chris> its heuristics is not lost on me ;-)

So, when can you start? :-)

S

From chris at simplistix.co.uk  Mon Mar  2 16:26:12 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 02 Mar 2009 15:26:12 +0000
Subject: [Python-Dev] How do I get commit access?
In-Reply-To: <18859.64200.859197.70559@montanaro.dyndns.org>
References: <49A7A906.8000008@simplistix.co.uk>
	<0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>
	<49A7AD8C.5030209@simplistix.co.uk>
	<A8EBDAFE66D843DBBAFFD2B942917A5D@RaymondLaptop1>
	<49A7B501.4070508@simplistix.co.uk> <go8ed0$vnq$1@ger.gmane.org>
	<18856.9294.373185.493616@montanaro.dyndns.org>
	<49ABCFCA.5020402@simplistix.co.uk>
	<18859.64200.859197.70559@montanaro.dyndns.org>
Message-ID: <49ABFA94.3090807@simplistix.co.uk>

skip at pobox.com wrote:
>     >> Indeed.  I'd be willing to grant you checkin privileges for SpamBayes
>     >> simply because because Christian recognized you and you seem willing
>     >> to roll up your sleeves. Do you do Windows?  <wink>
> 
>     Chris> The irony that Thunderbird put this in my spam folder based on
>     Chris> its heuristics is not lost on me ;-)
> 
> So, when can you start? :-)

When the test and patch attached to this issue get committed:

http://bugs.python.org/issue4308

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From pje at telecommunity.com  Mon Mar  2 16:52:57 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 02 Mar 2009 10:52:57 -0500
Subject: [Python-Dev] ABCs and MRO
In-Reply-To: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com
 >
References: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>
Message-ID: <20090302155044.B81F83A4109@sparrow.telecommunity.com>

At 10:41 AM 3/2/2009 +0000, Paul Moore wrote:
>2009/3/2 Benjamin Peterson <benjamin at python.org>:
> > 2009/3/1 Paul Moore <p.f.moore at gmail.com>:
> >>
> >> Is it worth getting simplegeneric exposed in 3.1
> >> (http://bugs.python.org/issue5135)? If it's going to be in 2.7, I'd
> >> like to see it hit 3.1. The patch is against trunk (for 2.7) at the
> >> moment, I'm not sure what the process would be for forward-porting it
> >> (do I generate a new patch against the py3k branch, or should it be
> >> applied to trunk and merged in?)

By the way guys, are you aware of:

   http://pypi.python.org/pypi/simplegeneric

There might be a bit of name confusion by exposing pkgutils' internal 
simplegeneric there.  Perhaps it should be called "trivialgeneric", 
as it's even tinier than simplegeneric.  ;-)


>The key problem with the patch is that ABCs do not play well with the
>type of introspection required to implement a generic function -
>namely enumeration of the superclasses of a class. The MRO of the
>class is fine for normal inheritance, but for ABCs it is possible to
>register classes which don't inherit from the ABC, so that you have a
>situation where issubclass (C, MyABC) can be true without MyABC being
>in C.__mro__:
>
> >>> import abc
> >>> class MyABC(object):
>...     __metaclass__ = abc.ABCMeta
>...
> >>> class C(object):
>...     pass
>...
> >>> MyABC.register(C)
> >>> issubclass(C, MyABC)
>True
> >>> C.__mro__
>(<class '__main__.C'>, <type 'object'>)
> >>>
>
>More generally, there is NO WAY to determine the list of classes for
>which issubclass(C, x) is true.
>
>This could be considered a limitation of, or a bug in, ABCs, I don't
>have a particular opinion on that, but it does mean that no code which
>relies on being able to traverse the class inheritance graph will see
>ABCs. One particular case of this is (any implementation I can think
>of, of) generic functions.
>
>In my view, this implies one of the following:
>
>1) It should be a documented limitation of such code that it doesn't
>work with ABCs (and conversely, this limitation of ABCs should be
>documented in the ABC documentation)
>2) Generic functions, and any other code requiring this type of
>introspection, is essentially useless unless it can support ABCs, and
>should not be used in the light of this limitation.
>3) This is a bug in ABCs and should be fixed.
>4) Something else I didn't think of :-)
>
>In my view, (2) is an unreasonable position to take, given the fact
>that (as I understand it) ABCs are supposed to be largely optional and
>shouldn't affect code that doesn't care about them...
>
>It's not clear to me how (3) should be addressed. Adding a slot to all
>classes to hold a list of ABCs they are registered against seems to be
>a large overhead for a relatively rarely used feature. I guess having
>a global registry of ABC registrations could work, but it seems
>clumsy. Any other suggestions?

This isn't really a new problem; if you base your generic function 
methods off of interfaces implemented by a type or instance, you have 
the same basic issues.

For systems that use a cache based on object type (like Guido's 
tuple-dispatch prototype, and my enhanced version in PEAK-Rules), the 
actual lookup is not a big deal.  You have a type-based test and you 
cache the result for the type.

PEAK-Rules' predicate dispatching is a bit more complex, because you 
need a rather more complex type test; the tree generator has to look 
at whether a type test is an ABC, and effectively translate it to 
"oldstyleisinstance(arg, ABC) or not oldstyleisinstance(arg, ABC) and 
ABC.__instancecheck__(arg)".  (Where oldstyleisinstance represents an 
__instancecheck__-free version of isinstance.)

This isn't a major problem either, just a bit of a bore/pain to implement.

The hairier issue for these types of systems is method precedence, 
though.  Since __mro__'s have to be consistently ordered, you can 
straightforwardly determine whether one class is "more specific" than 
another in a static way.  But with dynamic registration, the question 
could be more complex.

Personally, I'd like to see some way to subscribe to changes in ABC 
registration, so that generic functions or other tools can update 
their caches.  With that feature, you might even be able to implement 
full ABC support for simplegeneric, by treating ABC registrations as 
equivalent to mass registration of the ABC's registrants.

That is, if "AnABC.register(X)" and "afunc.register(AnABC, meth)" 
then "afunc.register(X, meth)".  So each time AnABC gets a new 
registrant, you automatically register the ABC method for the new 
registrant, as long as there's not already a method registered for 
that specific type.  That would probably be sufficient for what 
simplegeneric is doing.


From tjreedy at udel.edu  Mon Mar  2 17:28:41 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 02 Mar 2009 11:28:41 -0500
Subject: [Python-Dev] ABCs and MRO
In-Reply-To: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>
References: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>
Message-ID: <goh1fp$m43$1@ger.gmane.org>

Paul Moore wrote:
> 2009/3/2 Benjamin Peterson <benjamin at python.org>:
>> 2009/3/1 Paul Moore <p.f.moore at gmail.com>:
>>> Is it worth getting simplegeneric exposed in 3.1
>>> (http://bugs.python.org/issue5135)? If it's going to be in 2.7, I'd
>>> like to see it hit 3.1. The patch is against trunk (for 2.7) at the
>>> moment, I'm not sure what the process would be for forward-porting it
>>> (do I generate a new patch against the py3k branch, or should it be
>>> applied to trunk and merged in?)
>> Patches to the trunk are fine.
> 
> OK, thanks.
> 
>> As for the actual feature, I don't think it should hold up releases.
> 
> Fair enough.

Given that the purpose of 2.7 is
a) maintenance of existing code (which can include minor new features 
for existing facilities), and
b) easing conversion of code to 3.1
I am puzzled at the idea of adding a new facility to 2.7 that would not 
be in 3.1+.  It would lock code into 2.7+ and counter purpose b).

> The key problem with the patch is that ABCs do not play well with the
> type of introspection required to implement a generic function -
> namely enumeration of the superclasses of a class. The MRO of the
> class is fine for normal inheritance, but for ABCs it is possible to
> register classes which don't inherit from the ABC, so that you have a
> situation where issubclass (C, MyABC) can be true without MyABC being
> in C.__mro__:

I have no idea of what the solution should be other than your proposed 
'document the limitations', but someone who has moved on to 3.x and is 
only interested in generics would find it frustrating to be denied them 
because of the existence of something else they are not using.

Terry Jan Reedy




From amauryfa at gmail.com  Mon Mar  2 17:30:01 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 2 Mar 2009 17:30:01 +0100
Subject: [Python-Dev] test_io fails on test_1686475
In-Reply-To: <49ABD045.2010303@gmail.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com>
	<49A8781A.60309@gmail.com>
	<1afaf6160902271541y6e6d5937ieab4c3b0f111a5cc@mail.gmail.com>
	<79990c6b0903011258i4e64c990s9d29107646ac5372@mail.gmail.com>
	<49AB013C.4040409@gmail.com>
	<55961.151.53.158.182.1235945048.squirrel@webmail8.pair.com>
	<e27efe130903011513s4d383190rbf1ab9dec159f69a@mail.gmail.com>
	<op.up5x98f603jqhe@cesareprova.org> <49ABD045.2010303@gmail.com>
Message-ID: <e27efe130903020830p32e3a87fs615bc624801e4b66@mail.gmail.com>

On Mon, Mar 2, 2009 at 13:25, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Cesare Di Mauro wrote:
>> However, they are correct timestamps for Windows files, so I think that at least
>> the API on posixmodule.c should not fail when working with them. I don't know if
>> there's a way to handle them correctly.
>
> Use 64-bit time values (which is highly unlikely to ever be the case for
> CPython on a 32-bit OS).

64bit time_t is the default since VS2005.
See the patch at http://bugs.python.org/issue4379

-- 
Amaury Forgeot d'Arc

From python at rcn.com  Mon Mar  2 17:43:05 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Mar 2009 08:43:05 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
	to	collectionsready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org><EB2271A284C84B41ADA09A41F424ECB1@RaymondLaptop1>
	<loom.20090302T135344-234@post.gmane.org>
Message-ID: <A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1>


[Antoine Pitrou] 
> Given you were bitten by it in your own unit tests (the "eval(repr()) does not
> maintain ordering" problem pointed by Georg),

Completely unrelated.  The original test passed because the arbitrarily
ordered data in the regular dict happened to match the order added in
a regular dict because I didn't shuffle the keys.  There was no direct
dict-to-ordered dictcomparison.  Had the __eq__ method been defined
differently, the test still would have passed (had a false positive).


> I think it would be better to
> reconsider the current __eq__ behaviour, and make it ordering-aware.

If someone wants to explicitly ask for an order-sensitive comparison,
the docs give a clear, simple example of how to do that.  Otherwise,
it's best to leave regular dict equality in-place because OrderedDicts
need to be substitutable anywhere dicts are used and some of those
places may make the assumption that order insensitive compares are
being used.  

Also, the PEP outlines another issue with trying to make 
comparisons order sensitive.  It leads to weirdness with
ordereddict-to-dict comparisons making a behavior shift
based on the type of one of the two inputs.  It's just asking
for problems and it introduces an unnecessary difference
from regular dicts.   

Given that either choice will be surprising to someone, we opted
for the simplest API with the fewest special cases and made sure
the choice was clearly noted in the docs.


Raymond




From p.f.moore at gmail.com  Mon Mar  2 18:01:10 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 2 Mar 2009 17:01:10 +0000
Subject: [Python-Dev] ABCs and MRO
In-Reply-To: <20090302155044.B81F83A4109@sparrow.telecommunity.com>
References: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>
	<20090302155044.B81F83A4109@sparrow.telecommunity.com>
Message-ID: <79990c6b0903020901tc781d64h6e3462b609d2ba2e@mail.gmail.com>

2009/3/2 P.J. Eby <pje at telecommunity.com>:
> By the way guys, are you aware of:
>
> ?http://pypi.python.org/pypi/simplegeneric

Yes. It has been mentioned, and I am certainly aware of both it and
RuleDispatch.

> There might be a bit of name confusion by exposing pkgutils' internal
> simplegeneric there. ?Perhaps it should be called "trivialgeneric", as it's
> even tinier than simplegeneric. ?;-)

:-)

The problem is that any discussion of more than the bare minimum
functionality regularly results in people's heads exploding. Most
specifically, Guido usually ends up hating the whole idea when it gets
too complex. I'd rather stick with the most basic (and frankly,
useful) aspects and avoid the headaches and messy explosions :-)

> This isn't really a new problem; if you base your generic function methods
> off of interfaces implemented by a type or instance, you have the same basic
> issues.
[...]

See? You just made my head hurt :-)

> Personally, I'd like to see some way to subscribe to changes in ABC
> registration, so that generic functions or other tools can update their
> caches. ?With that feature, you might even be able to implement full ABC
> support for simplegeneric, by treating ABC registrations as equivalent to
> mass registration of the ABC's registrants.

But the issue is that the __subclasshook__ method allows things to be
virtual subclasses of an ABC without either being a real subclass,
*or* being explicitly registered. There simply is no way to answer the
question "which ABCs does this class implement" and get such ABCs
reported in the answer.

Consider:

class MyABC:
    __metaclass__ = ABCMeta
    @classmethod
    def __subclasshook__(cls, C):
        if cls is MyABC:
            if any("foo" in B.__dict__ for B in C.__mro__):
                return True
        return NotImplemented

class C(object):
    def foo(self):
        pass

c = C()

Here, isinstance(c, MyABC) is True. But if f is a generic function
with a registered implementation for MyABC, the only way I can see of
ensuring that f(c) calls that implementation, would be to test
isinstance(c, ImlCls) for all ImplCls for which there is a registered
implementation. There'd need to be a cache of results of some form, to
avoid a serious performance penalty, and it's not entirely obvious
what the definition of "best fit" would be.

That may be what you just said. :-)

But regardless, this is getting beyond what I'd describe as "simple",
and certainly is beyond any remit of tweaking pkgutil.simplegeneric to
make it public.

So either pkgutil.simplegeneric is useful with current (ignoring ABC)
semantics, or it isn't suitable for exposure. If the latter, someone
will have to propose a more complete implementation for the stdlib,
which is something that traditionally is doomed to failure :-(

So I don't intend to extend the functionality of
pkgutil.simplegeneric, and I await a decision on whether or not the
patch as it stands can go in. If the decision is that the current
implementation isn't powerful enough to be of any use, I'd suggest
that you propose your simplegeneric implementation for the stdlib
(enhancing it to handle the ABC issue, if it doesn't already). Or we
continue managing without generic functions in the stdlib, and the
pprint rewrite which started all this off, goes ahead with an adhoc
approach to extensibility.

Paul.

From barry at python.org  Mon Mar  2 18:02:53 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 2 Mar 2009 12:02:53 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
	to	collectionsready for pronouncement
In-Reply-To: <A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1>
References: <loom.20090301T182201-999@post.gmane.org><EB2271A284C84B41ADA09A41F424ECB1@RaymondLaptop1>
	<loom.20090302T135344-234@post.gmane.org>
	<A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1>
Message-ID: <38B32B8A-CE05-4094-82C1-43B70ADA9930@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Raymond,

+1 on adding this to the stdlib.  I especially like the idea of being  
able to use an ordered dict in a class's namespace.  I might be able  
use something like that to make my enum package simpler (which  
currently requires assignment of the name to the integer sort order).

I wanted to point out that the email package uses an ordered  
dictionary in its Message implementation.  Messages present a  
dictionary-like API for its headers.  I don't think I'd be able to use  
odicts though for this because of other semantic differences, e.g.  
multiple keys are allowed (though only visible through the non-mapping  
interface) and KeyError is never raised (like a defaultdict with a  
__missing__() returning None).  Note though that Message's odict-like  
implementation is about as horribly gross as it can be: it's just a  
list with linear searching for key lookup.  Messages should not have a  
billion headers. ;)

Have you or Armin considered the possibility of wanting both the  
defaultdict and odict behavior in, say a mixin subclass?

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSawRPXEjvBPtnXfVAQIunwQAlty1Gk3EByWK1fwOaN7+X/eC4QN4YpJL
MxWy5l/So3zUM/ofu32kLEjnBLmZOZFp28ExP5QTgse6c0VzNIGP9s6JrZeeAZ7s
uYk+EPChLw2GWuFgLQERpHnX9MA3XpCMbv+SheuqBROs31I7L/TCbDISk3+nOjtA
LngNgWVlKW4=
=glyD
-----END PGP SIGNATURE-----

From solipsis at pitrou.net  Mon Mar  2 18:06:37 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 2 Mar 2009 17:06:37 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?PEP_372_--_Adding_an_ordered_directory=09t?=
	=?utf-8?q?o=09collectionsready_for_pronouncement?=
References: <loom.20090301T182201-999@post.gmane.org><EB2271A284C84B41ADA09A41F424ECB1@RaymondLaptop1>
	<loom.20090302T135344-234@post.gmane.org>
	<A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1>
Message-ID: <loom.20090302T164935-18@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:
> Completely unrelated.  The original test passed because the arbitrarily
> ordered data in the regular dict happened to match the order added in
> a regular dict because I didn't shuffle the keys.

Well, I may be mistaken, but it seems your test_copying (in od5.diff) still
fails to check that ordering is preserved after a copy. Unless it's not part of
the contract, but then the datatype would really be ill-designed IMO.

> If someone wants to explicitly ask for an order-sensitive comparison,
> the docs give a clear, simple example of how to do that.

That's not the point. The point is that it's not enabled by default, which is
rather awkward since the whole point of the OrderedDict is that it is
ordering-sensitive.
Right now, equality of ordering-sensitive datatypes (e.g. list) is
ordering-sensitive by default.
And equality of ordering-insensitive datatypes (e.g. set) is
ordering-insensitive by default.

> Otherwise,
> it's best to leave regular dict equality in-place because OrderedDicts
> need to be substitutable anywhere dicts are used and some of those
> places may make the assumption that order insensitive compares are
> being used.  

You seem to imply that it is more important for __eq__ to work intuitively
between a non-OrderedDict and an OrderedDict, than it is to work intuitively
between two OrderedDicts.

It doesn't look like a common principle in Python. Witness:

>>> list(range(3)) == set(range(3))
False
>>> list(range(3)) == tuple(range(3))
False
>>> 1 == Decimal(1)
True
>>> 1 == 1.0
True
>>> 1.0 == Decimal(1)
False

IMO, comparison between different types should be "best effort", and not at the
price of making comparison between values of the same type less intuitive.

> It's just asking
> for problems and it introduces an unnecessary difference
> from regular dicts.

What you call an "unnecessary difference" seems to be the whole motivation for
introducing OrderedDicts. I see no point in trying to mitigate that difference
if the new type is to be genuinely useful.

> Given that either choice will be surprising to someone, we opted
> for the simplest API with the fewest special cases

But the current __eq__ does look like a special case, given the intended
semantics of the new datatype. Not having a special case would imply having an
ordering-sensitive comparison.

Regards

Antoine.



From python at rcn.com  Mon Mar  2 18:39:49 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Mar 2009 09:39:49 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered
	directory	to	collectionsready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org><EB2271A284C84B41ADA09A41F424ECB1@RaymondLaptop1><loom.20090302T135344-234@post.gmane.org><A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1>
	<loom.20090302T164935-18@post.gmane.org>
Message-ID: <16D8363EE402436C8059E1842AB56331@RaymondLaptop1>


[Antoine Pitrou]
> You seem to imply that it is more important for __eq__ to work intuitively
> between a non-OrderedDict and an OrderedDict, than it is to work intuitively
> between two OrderedDicts.

Yes.  When Armin and I worked through this, it became clear that
he had multiple use cases where ordered dicts needed to be used
in places that had been originally designed to expect regular dicts.
That was also the reason for subclassing dict.  Otherwise, we would
have just made a standalone class that defined all the mapping methods.

I don't think we going to convince you and that's okay.  We don't
have to agree on every design decision.  There were some reasons
for either approach and we picked the one that best fit Armin's use
cases, that was simplest, that introduced the fewest special rules,
and did not create a Liskov violation.  The choice was clearly
documented and an alternative was provided for people that
needed it.

Outside of your differing judgment on the __eq__ method, are you
basically happy with the ordered dict PEP?


Raymond





From guido at python.org  Mon Mar  2 19:27:16 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Mar 2009 10:27:16 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collectionsready for pronouncement
In-Reply-To: <16D8363EE402436C8059E1842AB56331@RaymondLaptop1>
References: <loom.20090301T182201-999@post.gmane.org>
	<EB2271A284C84B41ADA09A41F424ECB1@RaymondLaptop1>
	<loom.20090302T135344-234@post.gmane.org>
	<A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1>
	<loom.20090302T164935-18@post.gmane.org>
	<16D8363EE402436C8059E1842AB56331@RaymondLaptop1>
Message-ID: <ca471dc20903021027i10ba0401w502f124a3560e9f3@mail.gmail.com>

On Mon, Mar 2, 2009 at 9:39 AM, Raymond Hettinger <python at rcn.com> wrote:
>
> [Antoine Pitrou]
>>
>> You seem to imply that it is more important for __eq__ to work intuitively
>> between a non-OrderedDict and an OrderedDict, than it is to work
>> intuitively
>> between two OrderedDicts.
>
> Yes. ?When Armin and I worked through this, it became clear that
> he had multiple use cases where ordered dicts needed to be used
> in places that had been originally designed to expect regular dicts.
> That was also the reason for subclassing dict. ?Otherwise, we would
> have just made a standalone class that defined all the mapping methods.
>
> I don't think we going to convince you and that's okay. ?We don't
> have to agree on every design decision. ?There were some reasons
> for either approach and we picked the one that best fit Armin's use
> cases, that was simplest, that introduced the fewest special rules,
> and did not create a Liskov violation. ?The choice was clearly
> documented and an alternative was provided for people that
> needed it.

But you'll have to convince me, and so far I agree with Antoine that
doing the comparison without taking the order into account feels
really weird. I also think that comparing an odict to a dict with the
same items and expecting them to be the same feels wrong. It is not
needed to invoke Liskov: Liskov cares about the signature, not about
the computed value. There is no rule that says you are not allowed to
override odict.__eq__ so that it returns False in cases where
Mapping.__eq__ returns True.

I would propose the following formal specification for odict.__eq__:

  def __eq__(self, other):
    if not isinstance(other, odict):
      return NotImplemented  # Give other a chance; defaults to False
    return list(self.items()) == list(other.items())

Obviously an actual implementation can do something more complex
instead of the last line, like:

  for a, b in zip(self.items(), other.items()):
    if a != b:
      return False
  return True

> Outside of your differing judgment on the __eq__ method, are you
> basically happy with the ordered dict PEP?

I am.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jyasskin at gmail.com  Mon Mar  2 19:37:53 2009
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Mon, 2 Mar 2009 10:37:53 -0800
Subject: [Python-Dev] ABCs and MRO
In-Reply-To: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>
References: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>
Message-ID: <5d44f72f0903021037i14b68d97s3d86612024c97893@mail.gmail.com>

On Mon, Mar 2, 2009 at 2:41 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> ...
> More generally, there is NO WAY to determine the list of classes for
> which issubclass(C, x) is true.
>
> This could be considered a limitation of, or a bug in, ABCs, I don't
> have a particular opinion on that, but it does mean that no code which
> relies on being able to traverse the class inheritance graph will see
> ABCs. One particular case of this is (any implementation I can think
> of, of) generic functions.
>
> In my view, this implies one of the following:
>
> 1) It should be a documented limitation of such code that it doesn't
> work with ABCs (and conversely, this limitation of ABCs should be
> documented in the ABC documentation)
> 2) Generic functions, and any other code requiring this type of
> introspection, is essentially useless unless it can support ABCs, and
> should not be used in the light of this limitation.
> 3) This is a bug in ABCs and should be fixed.
> 4) Something else I didn't think of :-)

I tend to think it's a bug in ABCs. You seem to have thought of
several possible ways to fix it, and I don't have strong preferences
between them.

One way to deal with __subclasscheck__ definitions other than
ABCMeta.__subclasscheck__ would be to make the "someone registered a
subclass" interface public in addition to the "tell me when someone
registers a subclass" interface. Then other __subclasscheck__
definitions could call the "someone registered a subclass" method when
the class is defined if they want to be usable with generic functions.

You can get something similar to the mro by checking whether ABCs are
subclasses of each other.

Jeffrey

From guido at python.org  Mon Mar  2 19:50:57 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Mar 2009 10:50:57 -0800
Subject: [Python-Dev] How do I get commit access?
In-Reply-To: <49ABFA94.3090807@simplistix.co.uk>
References: <49A7A906.8000008@simplistix.co.uk>
	<0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>
	<49A7AD8C.5030209@simplistix.co.uk>
	<A8EBDAFE66D843DBBAFFD2B942917A5D@RaymondLaptop1>
	<49A7B501.4070508@simplistix.co.uk> <go8ed0$vnq$1@ger.gmane.org>
	<18856.9294.373185.493616@montanaro.dyndns.org>
	<49ABCFCA.5020402@simplistix.co.uk>
	<18859.64200.859197.70559@montanaro.dyndns.org>
	<49ABFA94.3090807@simplistix.co.uk>
Message-ID: <ca471dc20903021050g6b356795ndd1c8341d87360c@mail.gmail.com>

On Mon, Mar 2, 2009 at 7:26 AM, Chris Withers <chris at simplistix.co.uk> wrote:
> skip at pobox.com wrote:
>>
>> ? ?>> Indeed. ?I'd be willing to grant you checkin privileges for
>> SpamBayes
>> ? ?>> simply because because Christian recognized you and you seem willing
>> ? ?>> to roll up your sleeves. Do you do Windows? ?<wink>
>>
>> ? ?Chris> The irony that Thunderbird put this in my spam folder based on
>> ? ?Chris> its heuristics is not lost on me ;-)
>>
>> So, when can you start? :-)
>
> When the test and patch attached to this issue get committed:
>
> http://bugs.python.org/issue4308

I'd like to jump in and vouch for Chris. I've known him for many years
and while we haven't worked closely I expect he'd be a valuable
contributor. So +1 from me for giving Chris commit privileges for core
Python. (Chris, I assume you'll go through an apprentice phase where
you'll be letting another committer review your changes before you
commit them yourself. Rietveld at codereview.appspot.com should be
helpful for getting your code reviewed. (Just use upload.py instead of
the Create Issue form. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Mon Mar  2 20:20:10 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Mar 2009 11:20:10 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collectionsready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org>
	<EB2271A284C84B41ADA09A41F424ECB1@RaymondLaptop1>
	<loom.20090302T135344-234@post.gmane.org>
	<A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1>
	<loom.20090302T164935-18@post.gmane.org>
	<16D8363EE402436C8059E1842AB56331@RaymondLaptop1>
	<ca471dc20903021027i10ba0401w502f124a3560e9f3@mail.gmail.com>
Message-ID: <41CB54FAE58F43B5BFDD3492CF60086F@RaymondLaptop1>

> But you'll have to convince me,

Okay, here's one stab at it.  If it doesn't take, I give in.
ISTM, either way is right depending on your point of view and what
you're trying do at the time.  My judgment tips in favor of not
specializing the __eq__ method.  But it is not lost on me why
one might think that something that iterates in a specified order
would also make an order sensitive comparison.



> Liskov cares about the signature, not about the computed value.

That wasn't my understanding.  I thought it was entirely about computed values, "Let q(x) be a property provable about objects x of 
type T. Then q(y) should be true for objects y of type S where S is a subtype of T."  Or phrased differently, "In class hierarchies, 
it should be possible to treat a specialized object as if it were a base class object."

In this case, Armin wants to be able to pass in an ordered dictionary to functions that weren't designed with ordered dicts in mind 
(config parser, json/yaml parsers, nose, unittest, etc.).  Those functions should be able to assume that all the usual dictionary 
properties are still true.  In particular, those functions may make internal comparisons to a regular dict (perhaps as a cached 
value) and would expect those comparisons to succeed.


 > I would propose the following formal specification for odict.__eq__:
>   def __eq__(self, other):
>       if not isinstance(other, odict):
>           return NotImplemented  # Give other a chance; defaults to False
>       return list(self.items()) == list(other.items())


If I haven't convinced you, then I would be happy to put this in.



>> Outside of your differing judgment on the __eq__ method, are you
>> basically happy with the ordered dict PEP?

> I am.

Once you've decided on __eq__, can I mark the PEP as approved?



Raymond 


From guido at python.org  Mon Mar  2 20:46:26 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Mar 2009 11:46:26 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collectionsready for pronouncement
In-Reply-To: <41CB54FAE58F43B5BFDD3492CF60086F@RaymondLaptop1>
References: <loom.20090301T182201-999@post.gmane.org>
	<EB2271A284C84B41ADA09A41F424ECB1@RaymondLaptop1>
	<loom.20090302T135344-234@post.gmane.org>
	<A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1>
	<loom.20090302T164935-18@post.gmane.org>
	<16D8363EE402436C8059E1842AB56331@RaymondLaptop1>
	<ca471dc20903021027i10ba0401w502f124a3560e9f3@mail.gmail.com>
	<41CB54FAE58F43B5BFDD3492CF60086F@RaymondLaptop1>
Message-ID: <ca471dc20903021146v75d577f9if13353a3db5cb08b@mail.gmail.com>

On Mon, Mar 2, 2009 at 11:20 AM, Raymond Hettinger <python at rcn.com> wrote:
>> But you'll have to convince me,
>
> Okay, here's one stab at it. ?If it doesn't take, I give in.
> ISTM, either way is right depending on your point of view and what
> you're trying do at the time. ?My judgment tips in favor of not
> specializing the __eq__ method.

Comparing dicts is relatively uncommon. So we'd have to find and look
at use cases to decide.

> But it is not lost on me why
> one might think that something that iterates in a specified order
> would also make an order sensitive comparison.

My hunch is that not taking the ordering into account is going to
confuse people who consciously use odicts and bother to compare them.
I expect this is going to be a FAQ, no matter how much you try to
document it -- especially since the concept of an odict is so simple
and the API so clean that most people will not bother reading *any*
docs. I expect this to be the most common use case.

Use cases where people compare mappings knowing they may have
different concrete types are probably exceedingly rare, and if I was
doing that I wouldn't rely on __eq__ not being overridden -- IOW I'd
either explicitly invoke Mapping.__eq__(a, b) or write the equivalent
code myself.

The third class of use case is people comparing dicts not thinking
much about the possibility of there being a subclass that overrides
__eq__, and being surprised by the substitution of an odict. This
seems to be the use case you favor; to me it seems pretty rare too. To
convince me otherwise you'd have to find a common use case that does
this *and* is likely to encounter an odict in real life.

>> Liskov cares about the signature, not about the computed value.
>
> That wasn't my understanding. ?I thought it was entirely about computed
> values, "Let q(x) be a property provable about objects x of type T. Then
> q(y) should be true for objects y of type S where S is a subtype of T." ?Or
> phrased differently, "In class hierarchies, it should be possible to treat a
> specialized object as if it were a base class object."

This strict interpretation is violated all the time in OO programming;
consider e.g. the common overriding of object.__repr__. (In fact, even
the definition of dict.__eq__ overriding object.__eq__ would validate
it.) AFAIK a more common use of the term in OO languages is about
signatures only: if a method of class C accepts an argument of type T,
then you shouldn't override that method in a class D derived from C to
require an argument type S which is a subtype of T. (This is also
known as Contravariance. Read the section on Design by contract in
http://en.wikipedia.org/wiki/Liskov_substitution_principle.)

> In this case, Armin wants to be able to pass in an ordered dictionary to
> functions that weren't designed with ordered dicts in mind (config parser,
> json/yaml parsers, nose, unittest, etc.). ?Those functions should be able to
> assume that all the usual dictionary properties are still true. ?In
> particular, those functions may make internal comparisons to a regular dict
> (perhaps as a cached value) and would expect those comparisons to succeed.

That's a hypothetical use case. Have you found any real code that uses
__eq__ on dicts in this matter?

>> I would propose the following formal specification for odict.__eq__:
>>
>> ?def __eq__(self, other):
>> ? ? ?if not isinstance(other, odict):
>> ? ? ? ? ?return NotImplemented ?# Give other a chance; defaults to False
>> ? ? ?return list(self.items()) == list(other.items())
>
>
> If I haven't convinced you, then I would be happy to put this in.

Great.

>>> Outside of your differing judgment on the __eq__ method, are you
>>> basically happy with the ordered dict PEP?
>
>> I am.
>
> Once you've decided on __eq__, can I mark the PEP as approved?

Yes. (With the caveat that I haven't read it very closely, but the
basic spec seems sound apart from the __eq__ issue.)

Hm, I wonder if you should spec odict.__repr__ (and hence
odict.__str__). It would be confusing if an odict's repr() were the
same as a plain dict's.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Mon Mar  2 20:47:45 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Mar 2009 11:47:45 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
	tocollectionsready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org><EB2271A284C84B41ADA09A41F424ECB1@RaymondLaptop1><loom.20090302T135344-234@post.gmane.org><A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1><loom.20090302T164935-18@post.gmane.org><16D8363EE402436C8059E1842AB56331@RaymondLaptop1><ca471dc20903021027i10ba0401w502f124a3560e9f3@mail.gmail.com>
	<41CB54FAE58F43B5BFDD3492CF60086F@RaymondLaptop1>
Message-ID: <2C51A8EDF4384C72A5D9E2673B54177A@RaymondLaptop1>

[Me]
> In this case, Armin wants to be able to pass in an ordered dictionary to functions that weren't designed with ordered dicts in 
> mind (config parser, json/yaml parsers, nose, unittest, etc.).  Those functions should be able to assume that all the usual 
> dictionary properties are still true.  In particular, those functions may make internal comparisons to a regular dict (perhaps as 
> a cached value) and would expect those comparisons to succeed.

One other thought:  I was intending to modify namedtuple's _asdict() method to return an OrderedDict but don't want to break any 
existing code that relies on the returned object having an order insensitive comparison.

A object that currently returns a dict should be able to return an OrderedDict without breaking anything.  The proposed change 
precludes this possibility as well as the ones mentioned above.


Raymond 


From guido at python.org  Mon Mar  2 21:02:08 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Mar 2009 12:02:08 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
	tocollectionsready for pronouncement
In-Reply-To: <2C51A8EDF4384C72A5D9E2673B54177A@RaymondLaptop1>
References: <loom.20090301T182201-999@post.gmane.org>
	<EB2271A284C84B41ADA09A41F424ECB1@RaymondLaptop1>
	<loom.20090302T135344-234@post.gmane.org>
	<A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1>
	<loom.20090302T164935-18@post.gmane.org>
	<16D8363EE402436C8059E1842AB56331@RaymondLaptop1>
	<ca471dc20903021027i10ba0401w502f124a3560e9f3@mail.gmail.com>
	<41CB54FAE58F43B5BFDD3492CF60086F@RaymondLaptop1>
	<2C51A8EDF4384C72A5D9E2673B54177A@RaymondLaptop1>
Message-ID: <ca471dc20903021202q4e937771kcb6d4fb90a56420@mail.gmail.com>

On Mon, Mar 2, 2009 at 11:47 AM, Raymond Hettinger <python at rcn.com> wrote:
> [Me]
>>
>> In this case, Armin wants to be able to pass in an ordered dictionary to
>> functions that weren't designed with ordered dicts in mind (config parser,
>> json/yaml parsers, nose, unittest, etc.). ?Those functions should be able to
>> assume that all the usual dictionary properties are still true. ?In
>> particular, those functions may make internal comparisons to a regular dict
>> (perhaps as a cached value) and would expect those comparisons to succeed.
>
> One other thought: ?I was intending to modify namedtuple's _asdict() method
> to return an OrderedDict but don't want to break any existing code that
> relies on the returned object having an order insensitive comparison.

But do you know if any code that relies on that? Have you ever written
any yourself? If you really worry about this so much you could not
make this change, or you'd have to return a custom subclass of odict
that overrides __eq__ again.

> A object that currently returns a dict should be able to return an
> OrderedDict without breaking anything. ?The proposed change precludes this
> possibility as well as the ones mentioned above.

Well, such are the joys of strict backwards compatibility. If
something works fine with a dict today, there's no strong need to
return an odict tomorrow. You could always add a new API that returns
an odict.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Mon Mar  2 21:08:44 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Mar 2009 21:08:44 +0100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
 to	collectionsready for pronouncement
In-Reply-To: <ca471dc20903021146v75d577f9if13353a3db5cb08b@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>	<EB2271A284C84B41ADA09A41F424ECB1@RaymondLaptop1>	<loom.20090302T135344-234@post.gmane.org>	<A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1>	<loom.20090302T164935-18@post.gmane.org>	<16D8363EE402436C8059E1842AB56331@RaymondLaptop1>	<ca471dc20903021027i10ba0401w502f124a3560e9f3@mail.gmail.com>	<41CB54FAE58F43B5BFDD3492CF60086F@RaymondLaptop1>
	<ca471dc20903021146v75d577f9if13353a3db5cb08b@mail.gmail.com>
Message-ID: <49AC3CCC.1060005@v.loewis.de>

> This strict interpretation is violated all the time in OO programming;
> consider e.g. the common overriding of object.__repr__. (In fact, even
> the definition of dict.__eq__ overriding object.__eq__ would validate
> it.) AFAIK a more common use of the term in OO languages is about
> signatures only:

Slightly off-topic: I think what matters in reality is a
difficult-to-formulate *specification* of the behavior of the operation
also. I.e. not /all/ provable properties of the base implementation
need to be maintained, but only those occurring in the specification
of the base operation. Applications using the base then can only rely
on the *specified* properties of the operations they use, and there you
get substitutability.

Of course, what properties are part of the specification is an ongoing
discussion for many class hierarchies, in many languages (see e.g. the
relationship between __eq__ and __hash__).

Beyond transitivity and consistency with __hash__ (which is irrelevant
here), I don't think odict.__eq__ should be restricted to behave the
same as dict.__eq__.

Regards,
Martin

From python at rcn.com  Mon Mar  2 21:07:59 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Mar 2009 12:07:59 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
	tocollectionsready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org>
	<EB2271A284C84B41ADA09A41F424ECB1@RaymondLaptop1>
	<loom.20090302T135344-234@post.gmane.org>
	<A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1>
	<loom.20090302T164935-18@post.gmane.org>
	<16D8363EE402436C8059E1842AB56331@RaymondLaptop1>
	<ca471dc20903021027i10ba0401w502f124a3560e9f3@mail.gmail.com>
	<41CB54FAE58F43B5BFDD3492CF60086F@RaymondLaptop1>
	<2C51A8EDF4384C72A5D9E2673B54177A@RaymondLaptop1>
	<ca471dc20903021202q4e937771kcb6d4fb90a56420@mail.gmail.com>
Message-ID: <5165D52DA0D744568871D1CD46E283CF@RaymondLaptop1>

Compromise?

     def __eq__(self, other):
        if isinstance(other, OrderedDict):
            return all(map(operator.eq, self.items(), other.items()))
        if isinstance(other, Mapping):
            return dict.__eq__(self, other)
        return NotImplemented # Give other a chance; defaults to False

OrderedDict-to-OrderedDict comparisons are order sensitive -- matching your intuition
OrderedDict-to-OtherMappings -- allow me and Armin to have our substitutability for dicts.


Raymond

From ncoghlan at gmail.com  Mon Mar  2 21:20:54 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 03 Mar 2009 06:20:54 +1000
Subject: [Python-Dev] ABCs and simplegeneric use case
In-Reply-To: <79990c6b0903020525x7ce01507reff6481bb9d204c@mail.gmail.com>
References: <79990c6b0903020525x7ce01507reff6481bb9d204c@mail.gmail.com>
Message-ID: <49AC3FA6.3040902@gmail.com>

Paul Moore wrote:
> 2009/3/1 Nick Coghlan <ncoghlan at gmail.com>:
>> As much as I'd like to get a simple generic implementation into
>> functools, the lack of support for ABCs still bothers me (despite my
>> last post about that on the tracker item). I'd be a -0 on it going in as
>> is, but if someone can figure out a clever way of supporting ABCs
>> without completing killing the invocation speed for generics, that would
>> go up to a +1.
> 
> Nick,
> Armin Ronacher pointed out that it's not likely to be possible to
> enumerate ABCs even in theory, as ABCs can do semantic checks (e.g.
> checking for the presence of special methods like __iter__) without
> needing either inheritance or explicit registration.
> 
> As you had a genuine use case for simplegeneric, how badly does this
> limitation damage it?

That's a very good point that Armin raises (i.e. that enumerating the
ABCs a given class implements is impossible even in theory, let alone in
practice). It is actually enough to put me back to the perspective that
I posted on the tracker item: the failure to inherit generic function
implementations when a class doesn't actually have the ABC in its MRO is
similar in nature to the failure to inherit the ABCs own method
implementations.

It still bugs me a little, but I'll get over it :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From guido at python.org  Mon Mar  2 21:21:17 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Mar 2009 12:21:17 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
	tocollectionsready for pronouncement
In-Reply-To: <5165D52DA0D744568871D1CD46E283CF@RaymondLaptop1>
References: <loom.20090301T182201-999@post.gmane.org>
	<loom.20090302T135344-234@post.gmane.org>
	<A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1>
	<loom.20090302T164935-18@post.gmane.org>
	<16D8363EE402436C8059E1842AB56331@RaymondLaptop1>
	<ca471dc20903021027i10ba0401w502f124a3560e9f3@mail.gmail.com>
	<41CB54FAE58F43B5BFDD3492CF60086F@RaymondLaptop1>
	<2C51A8EDF4384C72A5D9E2673B54177A@RaymondLaptop1>
	<ca471dc20903021202q4e937771kcb6d4fb90a56420@mail.gmail.com>
	<5165D52DA0D744568871D1CD46E283CF@RaymondLaptop1>
Message-ID: <ca471dc20903021221j1c1cc3eahb48c73c7cdfdf46d@mail.gmail.com>

On Mon, Mar 2, 2009 at 12:07 PM, Raymond Hettinger <python at rcn.com> wrote:
> Compromise?
>
> ? ?def __eq__(self, other):
> ? ? ? if isinstance(other, OrderedDict):
> ? ? ? ? ? return all(map(operator.eq, self.items(), other.items()))
> ? ? ? if isinstance(other, Mapping):
> ? ? ? ? ? return dict.__eq__(self, other)
> ? ? ? return NotImplemented # Give other a chance; defaults to False
>
> OrderedDict-to-OrderedDict comparisons are order sensitive -- matching your
> intuition
> OrderedDict-to-OtherMappings -- allow me and Armin to have our
> substitutability for dicts.

This sounds fair. Note that dict.__eq__ actually returns
NotImplemented if not isinstance(other, dict) so you could tighten the
test to isinstance(other, dict) if you wanted to.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Mon Mar  2 21:23:51 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Mar 2009 12:23:51 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
	tocollectionsready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org>
	<loom.20090302T135344-234@post.gmane.org>
	<A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1>
	<loom.20090302T164935-18@post.gmane.org>
	<16D8363EE402436C8059E1842AB56331@RaymondLaptop1>
	<ca471dc20903021027i10ba0401w502f124a3560e9f3@mail.gmail.com>
	<41CB54FAE58F43B5BFDD3492CF60086F@RaymondLaptop1>
	<2C51A8EDF4384C72A5D9E2673B54177A@RaymondLaptop1>
	<ca471dc20903021202q4e937771kcb6d4fb90a56420@mail.gmail.com>
	<5165D52DA0D744568871D1CD46E283CF@RaymondLaptop1>
	<ca471dc20903021221j1c1cc3eahb48c73c7cdfdf46d@mail.gmail.com>
Message-ID: <4767FECD700B4062924FEBE5EC59D492@RaymondLaptop1>

[GvR]
> This sounds fair. Note that dict.__eq__ actually returns
> NotImplemented if not isinstance(other, dict) so you could tighten the
> test to isinstance(other, dict) if you wanted to.

Okay.  Done deal.


Raymond

From benjamin at python.org  Mon Mar  2 21:28:42 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 2 Mar 2009 15:28:42 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <loom.20090301T182201-999@post.gmane.org>
References: <loom.20090301T182201-999@post.gmane.org>
Message-ID: <1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>

2009/3/1 Armin Ronacher <armin.ronacher at active-4.com>:
> Hi everybody,
>
> PEP 372 was modified so that it provides a simpler API (only the dict API
> to be exact) and it was decided to start with a Python-only implementation
> and replace it with a C version later if necessary.
>
> Annotated changes from earlier versions of the PEP:
>
> - ? the extra API for ordered dict was dropped to keep the interface
> ? ?simple and clean. ?Future versions can still be expanded but it's
> ? ?impossible to drop features later on.
>
> - ? To keep the implementation simple 3.1 / 2.7 will ship with a
> ? ?Python-only version of the class. ?It can still be rewritten in
> ? ?C if it turns out to be too slow or thread safety is required.
>
> The corresponding issue in the tracker: http://bugs.python.org/issue5397
> Link to the PEP: http://www.python.org/dev/peps/pep-0372/
>
> Anything else that should be done?

Have you considered naming? I would think that "odict" or
"ordereddict" would be more consistent with other collections names
especially "defaultdict".



-- 
Regards,
Benjamin

From ncoghlan at gmail.com  Mon Mar  2 21:48:28 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 03 Mar 2009 06:48:28 +1000
Subject: [Python-Dev] ABCs and MRO
In-Reply-To: <goh1fp$m43$1@ger.gmane.org>
References: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>
	<goh1fp$m43$1@ger.gmane.org>
Message-ID: <49AC461C.5030503@gmail.com>

Terry Reedy wrote:
>>> As for the actual feature, I don't think it should hold up releases.
>>
>> Fair enough.
> 
> Given that the purpose of 2.7 is
> a) maintenance of existing code (which can include minor new features
> for existing facilities), and
> b) easing conversion of code to 3.1
> I am puzzled at the idea of adding a new facility to 2.7 that would not
> be in 3.1+.  It would lock code into 2.7+ and counter purpose b).

It's possible we will end up in a situation where 3.0 and 3.1 are both
aligned with 2.6, while 2.7 aligns with 3.2. That's particularly so with
only 6 months or so between 3.0 and 3.1, while I currently expect the
gap between 2.6 and 2.7 to be closer to the traditional 18 months.

Of course, it will depend on how the relative timing of the releases
plays out in practice :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From p.f.moore at gmail.com  Mon Mar  2 22:14:52 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 2 Mar 2009 21:14:52 +0000
Subject: [Python-Dev] ABCs and MRO
In-Reply-To: <5d44f72f0903021037i14b68d97s3d86612024c97893@mail.gmail.com>
References: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>
	<5d44f72f0903021037i14b68d97s3d86612024c97893@mail.gmail.com>
Message-ID: <79990c6b0903021314g30ca852fjb39f0bc4d89ada63@mail.gmail.com>

2009/3/2 Jeffrey Yasskin <jyasskin at gmail.com>:
> I tend to think it's a bug in ABCs. You seem to have thought of
> several possible ways to fix it, and I don't have strong preferences
> between them.

I've discussed ways of fixing simplegeneric, but not of fixing the
issue with ABCs. I'm not sure the ABC "issue" is fixable - after all,
it's by design that ABCs can implement __issubclass__ and "magically"
become superclasses of arbitrary classes as a result.

I'm not happy about fixing simplegeneric, though, as the whole point
was just to expose an existing implementation, because it might be
generally useful. If we start expanding and enhancing it, there are
better implementations already available on PyPI (better in the sense
of having seen real-world use). And if they don't handle ABCs, then
that might indicate that needing to handle ABCs isn't as vital as this
discussion would seem to imply (but I have no real-world data myself
to make such a claim).

> One way to deal with __subclasscheck__ definitions other than
> ABCMeta.__subclasscheck__ would be to make the "someone registered a
> subclass" interface public in addition to the "tell me when someone
> registers a subclass" interface. Then other __subclasscheck__
> definitions could call the "someone registered a subclass" method when
> the class is defined if they want to be usable with generic functions.

I'm not sure I follow this. Can you clarify? My intuition is that *no*
notification hook can deal with ABCs that code arbitrary checks in
their __subclasscheck__ method (so there's no registration involved,
particular classes just "magically" become subclasses).

> You can get something similar to the mro by checking whether ABCs are
> subclasses of each other.

The key point about the MRO is that you get it just from the initial
class, via __mro__. You're not checking if another class is *in* the
MRO, but rather you're *generating* the list of classes in the MRO.

Paul.

From MLMLists at Comcast.net  Mon Mar  2 21:58:33 2009
From: MLMLists at Comcast.net (Mitchell L Model)
Date: Mon, 2 Mar 2009 15:58:33 -0500
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <mailman.29.1235991618.1178.python-dev@python.org>
References: <mailman.29.1235991618.1178.python-dev@python.org>
Message-ID: <p06240803c5d1f82dbc4b@[10.0.1.221]>

Would whoever is responsible for IDLE please take a look at the 
patches I submitted for Python 2 & 3 [tracker IDs 5233 and 5234 
respectively]. These change the behavior of IDLE so that IDLESTARTUP 
or PYTHONSTARTUP files are executed with each restart. This allows 
loading frequently used packages, personal utilities, etc. 
automatically at each restart. I consider this a very important 
problem in IDLE, especially when using it to teach. I would really 
like to see them in 3.1. The patch is already there; someone just has 
to do whatever gets done with patches to validate it and check it in. 
It's not a lot of code changes.

From g.brandl at gmx.net  Mon Mar  2 22:28:53 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 02 Mar 2009 22:28:53 +0100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
Message-ID: <gohj2l$63r$1@ger.gmane.org>

Benjamin Peterson schrieb:
> 2009/3/1 Armin Ronacher <armin.ronacher at active-4.com>:
>> Hi everybody,
>>
>> PEP 372 was modified so that it provides a simpler API (only the dict API
>> to be exact) and it was decided to start with a Python-only implementation
>> and replace it with a C version later if necessary.
>>
>> Annotated changes from earlier versions of the PEP:
>>
>> -   the extra API for ordered dict was dropped to keep the interface
>>    simple and clean.  Future versions can still be expanded but it's
>>    impossible to drop features later on.
>>
>> -   To keep the implementation simple 3.1 / 2.7 will ship with a
>>    Python-only version of the class.  It can still be rewritten in
>>    C if it turns out to be too slow or thread safety is required.
>>
>> The corresponding issue in the tracker: http://bugs.python.org/issue5397
>> Link to the PEP: http://www.python.org/dev/peps/pep-0372/
>>
>> Anything else that should be done?
> 
> Have you considered naming? I would think that "odict" or
> "ordereddict" would be more consistent with other collections names
> especially "defaultdict".

We're already quite inconsistent with type name casing in the collections
module, so it wouldn't matter so much.  (Though I'd find symmetry with
defaultdict pleasing as well.)

Georg


From armin.ronacher at active-4.com  Mon Mar  2 22:45:35 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Mon, 2 Mar 2009 21:45:35 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?PEP_372_--_Adding_an_ordered_directory=09t?=
	=?utf-8?q?ocollectionsready_for_pronouncement?=
References: <loom.20090301T182201-999@post.gmane.org>
	<loom.20090302T135344-234@post.gmane.org>
	<A715D86D7A7D4DACBE15D7890D415BD9@RaymondLaptop1>
	<loom.20090302T164935-18@post.gmane.org>
	<16D8363EE402436C8059E1842AB56331@RaymondLaptop1>
	<ca471dc20903021027i10ba0401w502f124a3560e9f3@mail.gmail.com>
	<41CB54FAE58F43B5BFDD3492CF60086F@RaymondLaptop1>
	<2C51A8EDF4384C72A5D9E2673B54177A@RaymondLaptop1>
	<ca471dc20903021202q4e937771kcb6d4fb90a56420@mail.gmail.com>
	<5165D52DA0D744568871D1CD46E283CF@RaymondLaptop1>
	<ca471dc20903021221j1c1cc3eahb48c73c7cdfdf46d@mail.gmail.com>
Message-ID: <loom.20090302T214438-652@post.gmane.org>

Hi,

Guido van Rossum <guido <at> python.org> writes:
> This sounds fair. Note that dict.__eq__ actually returns
> NotImplemented if not isinstance(other, dict) so you could tighten the
> test to isinstance(other, dict) if you wanted to.
I'm actually very happy with that decision.  The original PEP was doing
exactly that and I still think it makes more sense.

[sorry Raymond :)]

Regards,
Armin




From guido at python.org  Mon Mar  2 22:46:10 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Mar 2009 13:46:10 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <gohj2l$63r$1@ger.gmane.org>
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
Message-ID: <ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>

On Mon, Mar 2, 2009 at 1:28 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Benjamin Peterson schrieb:
>> 2009/3/1 Armin Ronacher <armin.ronacher at active-4.com>:
>>> Hi everybody,
>>>
>>> PEP 372 was modified so that it provides a simpler API (only the dict API
>>> to be exact) and it was decided to start with a Python-only implementation
>>> and replace it with a C version later if necessary.
>>>
>>> Annotated changes from earlier versions of the PEP:
>>>
>>> - ? the extra API for ordered dict was dropped to keep the interface
>>> ? ?simple and clean. ?Future versions can still be expanded but it's
>>> ? ?impossible to drop features later on.
>>>
>>> - ? To keep the implementation simple 3.1 / 2.7 will ship with a
>>> ? ?Python-only version of the class. ?It can still be rewritten in
>>> ? ?C if it turns out to be too slow or thread safety is required.
>>>
>>> The corresponding issue in the tracker: http://bugs.python.org/issue5397
>>> Link to the PEP: http://www.python.org/dev/peps/pep-0372/
>>>
>>> Anything else that should be done?
>>
>> Have you considered naming? I would think that "odict" or
>> "ordereddict" would be more consistent with other collections names
>> especially "defaultdict".
>
> We're already quite inconsistent with type name casing in the collections
> module, so it wouldn't matter so much. ?(Though I'd find symmetry with
> defaultdict pleasing as well.)

+1 for odict. Somehow I thought that was the name proposed by the PEP. :-(

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From armin.ronacher at active-4.com  Mon Mar  2 22:48:30 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Mon, 2 Mar 2009 21:48:30 +0000 (UTC)
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
Message-ID: <loom.20090302T214604-321@post.gmane.org>

Hi,

Georg Brandl <g.brandl <at> gmx.net> writes:

> We're already quite inconsistent with type name casing in the collections
> module, so it wouldn't matter so much.  (Though I'd find symmetry with
> defaultdict pleasing as well.)
We either have the way to be consistent with defaultdict and dict or with
Counter, MutableMapping etc.

I think it's a bit too chaotic already to make a fair decision here.  If we
seriously consider a C implementation it would probably be a good idea to call
it `odict`.  C-Classes are usually lower cased as far as I can see.

Regards,
Armin


From pje at telecommunity.com  Mon Mar  2 22:52:27 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Mon, 02 Mar 2009 16:52:27 -0500
Subject: [Python-Dev] ABCs and MRO
In-Reply-To: <79990c6b0903021314g30ca852fjb39f0bc4d89ada63@mail.gmail.co
 m>
References: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>
	<5d44f72f0903021037i14b68d97s3d86612024c97893@mail.gmail.com>
	<79990c6b0903021314g30ca852fjb39f0bc4d89ada63@mail.gmail.com>
Message-ID: <20090302215016.023C23A409E@sparrow.telecommunity.com>

At 09:14 PM 3/2/2009 +0000, Paul Moore wrote:
>2009/3/2 Jeffrey Yasskin <jyasskin at gmail.com>:
> > I tend to think it's a bug in ABCs. You seem to have thought of
> > several possible ways to fix it, and I don't have strong preferences
> > between them.
>
>I've discussed ways of fixing simplegeneric, but not of fixing the
>issue with ABCs. I'm not sure the ABC "issue" is fixable - after all,
>it's by design that ABCs can implement __issubclass__ and "magically"
>become superclasses of arbitrary classes as a result.
>
>I'm not happy about fixing simplegeneric, though, as the whole point
>was just to expose an existing implementation, because it might be
>generally useful. If we start expanding and enhancing it, there are
>better implementations already available on PyPI (better in the sense
>of having seen real-world use). And if they don't handle ABCs, then
>that might indicate that needing to handle ABCs isn't as vital as this
>discussion would seem to imply (but I have no real-world data myself
>to make such a claim).

I would say it's mainly an annoyance, as long as you have a way to 
access the method registered for a given ABC, because then you can 
explicitly register it for types that are being dynamically detected.

Now, how widespread an annoyance/FAQ issue it'll be, will depend on:

1. whether the stdlib itself is using GF's and
2. whether those GF's are meaningfully used against ABCs, and
3. how common it is for classes to be "implicit subclasses" of one of 
those ABCs *and* used with one of those GFs.

This sounds like a lot of conditions that have to line up, for it to 
become an (easily worked-around) annoyance.

My inclination is simply to document that it works on explicit type 
relationships, and note that ABC's and other exotic users of 
__*check__ methods may require explicit method registrations to work correctly.


From steve at holdenweb.com  Mon Mar  2 22:50:09 2009
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 02 Mar 2009 16:50:09 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
Message-ID: <gohkar$an6$1@ger.gmane.org>

Benjamin Peterson wrote:
> 2009/3/1 Armin Ronacher <armin.ronacher at active-4.com>:
[...]
>> The corresponding issue in the tracker: http://bugs.python.org/issue5397
>> Link to the PEP: http://www.python.org/dev/peps/pep-0372/
>>
>> Anything else that should be done?
> 
> Have you considered naming? I would think that "odict" or
> "ordereddict" would be more consistent with other collections names
> especially "defaultdict".
> 
Surely that's just a thinko in the subject line? The PEP specifies
"ordered dictionary" and nobody has been talking about "directories".

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From benjamin at python.org  Mon Mar  2 22:54:34 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 2 Mar 2009 15:54:34 -0600
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <loom.20090302T214604-321@post.gmane.org>
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org> <loom.20090302T214604-321@post.gmane.org>
Message-ID: <1afaf6160903021354r248f8a86i4d0f6e3df60523cd@mail.gmail.com>

2009/3/2 Armin Ronacher <armin.ronacher at active-4.com>:
> Hi,
>
> Georg Brandl <g.brandl <at> gmx.net> writes:
>
>> We're already quite inconsistent with type name casing in the collections
>> module, so it wouldn't matter so much. ?(Though I'd find symmetry with
>> defaultdict pleasing as well.)
> We either have the way to be consistent with defaultdict and dict or with
> Counter, MutableMapping etc.

I think "normal" class names are fine for ABCs, but I brought it up
because the other dictionary class in collections had a all lowername.

>
> I think it's a bit too chaotic already to make a fair decision here. ?If we
> seriously consider a C implementation it would probably be a good idea to call
> it `odict`. ?C-Classes are usually lower cased as far as I can see.

I don't implementation language should determine naming.



-- 
Regards,
Benjamin

From armin.ronacher at active-4.com  Mon Mar  2 22:50:36 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Mon, 2 Mar 2009 21:50:36 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?PEP_372_--_Adding_an_ordered_directory_to?=
	=?utf-8?q?=09collections_ready_for_pronouncement?=
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
Message-ID: <loom.20090302T214855-85@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:

> +1 for odict. Somehow I thought that was the name proposed by the PEP. 
It originally was, Raymond wanted to change it.  I would still vote for odict if
that's still possible :)


Regards,
Armin


From armin.ronacher at active-4.com  Mon Mar  2 22:55:15 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Mon, 2 Mar 2009 21:55:15 +0000 (UTC)
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohkar$an6$1@ger.gmane.org>
Message-ID: <loom.20090302T215428-152@post.gmane.org>

Steve Holden <steve <at> holdenweb.com> writes:

> Surely that's just a thinko in the subject line? The PEP specifies
> "ordered dictionary" and nobody has been talking about "directories".
Actually, the initial version of the PEP had that typo in the topic.  Guess I
copy pasted wrong again:
  http://www.google.com/search?q=%22adding+an+ordered+directory%22

Regards,
Armin


From python at rcn.com  Mon Mar  2 22:58:28 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Mar 2009 13:58:28 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
	tocollections ready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org><1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com><gohj2l$63r$1@ger.gmane.org>
	<loom.20090302T214604-321@post.gmane.org>
Message-ID: <69FF52B54AF644AD86D4F966569ED34B@RaymondLaptop1>

My preference is OrderedDict.
That says that it is a pure python class like Counter, UserDict, UserList, MutableMapping and other collections classes.
It is clear and explicit in its intention and doesn't make you try to remember what the o in odict stands for.

Raymond


From dirkjan at ochtman.nl  Mon Mar  2 22:41:30 2009
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Mon, 02 Mar 2009 22:41:30 +0100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <gohj2l$63r$1@ger.gmane.org>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
Message-ID: <49AC528A.9090505@ochtman.nl>

On 02/03/2009 22:28, Georg Brandl wrote:
> We're already quite inconsistent with type name casing in the collections
> module, so it wouldn't matter so much.  (Though I'd find symmetry with
> defaultdict pleasing as well.)

Since the odict naming is already so prevalent in the wild, it seems to 
me like that would be the best candidate. (Plus, it's shorter!)

/bikeshedding

Cheers,

Dirkjan

From benjamin at python.org  Mon Mar  2 23:03:56 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 2 Mar 2009 16:03:56 -0600
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
	tocollections ready for pronouncement
In-Reply-To: <69FF52B54AF644AD86D4F966569ED34B@RaymondLaptop1>
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org> <loom.20090302T214604-321@post.gmane.org>
	<69FF52B54AF644AD86D4F966569ED34B@RaymondLaptop1>
Message-ID: <1afaf6160903021403g58091109ib1d64463ccb5740b@mail.gmail.com>

2009/3/2 Raymond Hettinger <python at rcn.com>:
> My preference is OrderedDict.
> That says that it is a pure python class like Counter, UserDict, UserList,
> MutableMapping and other collections classes.

I don't understand why implementation language should have any
significance in naming. Classes should be able to be implemented in
any language transparently.

> It is clear and explicit in its intention and doesn't make you try to
> remember what the o in odict stands for.

I agree and that's why I propose "ordereddict"


-- 
Regards,
Benjamin

From python at rcn.com  Mon Mar  2 23:05:25 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Mar 2009 14:05:25 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com><gohj2l$63r$1@ger.gmane.org>
	<49AC528A.9090505@ochtman.nl>
Message-ID: <53A87C7F41B04307B6CBD18F73614422@RaymondLaptop1>

> /bikeshedding

Yes.  Also we need to paint it green with pink polka dots :-)

From ncoghlan at gmail.com  Mon Mar  2 23:15:56 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 03 Mar 2009 08:15:56 +1000
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
 to	collections ready for pronouncement
In-Reply-To: <ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
Message-ID: <49AC5A9C.3030507@gmail.com>

Guido van Rossum wrote:
> +1 for odict. Somehow I thought that was the name proposed by the PEP. :-(

The examples in the PEP used 'odict' (until recently), but the patch was
for OrderedDict.

I don't personally mind either way.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Mar  2 23:16:43 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 03 Mar 2009 08:16:43 +1000
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
 to	collections ready for pronouncement
In-Reply-To: <53A87C7F41B04307B6CBD18F73614422@RaymondLaptop1>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com><gohj2l$63r$1@ger.gmane.org>	<49AC528A.9090505@ochtman.nl>
	<53A87C7F41B04307B6CBD18F73614422@RaymondLaptop1>
Message-ID: <49AC5ACB.10002@gmail.com>

Raymond Hettinger wrote:
>> /bikeshedding
> 
> Yes.  Also we need to paint it green with pink polka dots :-)

Or should that be pink with green polka dots? ;)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From martin at v.loewis.de  Mon Mar  2 23:22:03 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Mar 2009 23:22:03 +0100
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <p06240803c5d1f82dbc4b@[10.0.1.221]>
References: <mailman.29.1235991618.1178.python-dev@python.org>
	<p06240803c5d1f82dbc4b@[10.0.1.221]>
Message-ID: <49AC5C0B.4070806@v.loewis.de>

> Would whoever is responsible for IDLE please take a look at the patches
> I submitted for Python 2 & 3 [tracker IDs 5233 and 5234 respectively].
> These change the behavior of IDLE so that IDLESTARTUP or PYTHONSTARTUP
> files are executed with each restart. This allows loading frequently
> used packages, personal utilities, etc. automatically at each restart. I
> consider this a very important problem in IDLE, especially when using it
> to teach. 

Just to put this into perspective: I personally don't see that as a very
important problem. I didn't know IDLESTARTUP existed, and I use
PYTHONSTARTUP only for the command line (to setup readline and history).
I think there are many more open issues that are *way* more important.

This is not to say that the patch should not applied - I haven't even
looked at it. It's just a warning that, if no other committer feels this
is as important as you fell it is, it may not be committed reviewed and
committed before 3.1.

Regards,
Martin

From python at rcn.com  Mon Mar  2 23:36:32 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Mar 2009 14:36:32 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
	to	collections ready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org><ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
Message-ID: <A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>


[Nick Coghlan]
> The examples in the PEP used 'odict' (until recently), but the patch was
> for OrderedDict.

As an experiment, try walking down the hall asking a few programmers who aren't in this conversion what they think 
collections.odict() is?
Is it a class or function?  What does it do?  Can the English as second language folks guess what the o stands for?  Is it a builtin 
or pure python?  My guess is that the experiment will be informative.

When we use the class, we typically only spell-out the constructor once while actually using the returned object many times.  So, 
have we really saved any typing?

In the context of other applications, which is clearer?

    json.loads(jtext, object_pairs_hook=odict)
    config = ConfigParser(dict_type=odict)

or

    json.loads(jtext, object_pairs_hook=OrderedDict)
    config = ConfigParser(dict_type=OrderedDict)

I find the former to be non-communicative.


Raymond 


From solipsis at pitrou.net  Mon Mar  2 23:46:39 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 2 Mar 2009 22:46:39 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?PEP_372_--_Adding_an_ordered_directory=09t?=
	=?utf-8?q?o=09collections_ready_for_pronouncement?=
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org><ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
Message-ID: <loom.20090302T224539-504@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:
> 
> [Nick Coghlan]
> > The examples in the PEP used 'odict' (until recently), but the patch was
> > for OrderedDict.
> 
> As an experiment, try walking down the hall asking a few programmers who
aren't in this conversion what they
> think 
> collections.odict() is?

I second that odict is too terse, and it's also a recipe for subtle typos.
I think that both ordereddict and OrderedDict can't go wrong.

Regards

Antoine.



From martin at v.loewis.de  Mon Mar  2 23:48:21 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Mar 2009 23:48:21 +0100
Subject: [Python-Dev] PEP 372 -- Adding an ordered
 directory	to	collections ready for pronouncement
In-Reply-To: <A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org><ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
Message-ID: <49AC6235.6000702@v.loewis.de>

> Is it a class or function?  What does it do?  Can the English as second
> language folks guess what the o stands for?  Is it a builtin or pure
> python?  My guess is that the experiment will be informative.

I'll do that tomorrow (if I manage to remember). My guess is that
"ordered dictionary" is as descriptive to them as "odict" or "blonzo"
(well, perhaps they do recognize the "dictionary" part of it, and
manage not to confuse it with "directory"). As for the "ordered"
part, my guess is that most people will suggest that it means "sorted"
(native speakers or not).

Regards,
Martin

From armin.ronacher at active-4.com  Tue Mar  3 00:09:50 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Mon, 2 Mar 2009 23:09:50 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?PEP_372_--_Adding_an_ordered_directory=09t?=
	=?utf-8?q?o=09collections_ready_for_pronouncement?=
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org><ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
Message-ID: <loom.20090302T230553-35@post.gmane.org>

Hi,

Raymond Hettinger <python <at> rcn.com> writes:
> When we use the class, we typically only spell-out the constructor
> once while actually using the returned
> object many times.  So, 
> have we really saved any typing?
I'm fine with the typed out name as well, but I still would prefer lowercase to
stay consistent with defaultdict/dict.

Unfortunately PEP 8 never really took off naming-wise, so we're mostly following
the "reuse the naming scheme from existing code in the same module" rule, and I
think there lowercase wins, thanks to defaultdict.

Regards,
Armin


From benjamin at python.org  Tue Mar  3 00:10:37 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 2 Mar 2009 17:10:37 -0600
Subject: [Python-Dev] first 3.1 alpha this Saturday
Message-ID: <1afaf6160903021510td535759p7b33aa0db802de97@mail.gmail.com>

Hi everyone!

The first alpha of 3.1 will be coming out this Saturday, so I would
appreciate you not breaking anything too egregiously. :) If you think
some thing should hold up the release, mark the issue a release
blocker or contact me by email or "gutworth" on #python-dev.

I have been talking with Antoine about io in C implementation. We
would like to merge the branch before Saturday, so it can get some
real world testing. io-c, however, hasn't received much through
review. Would somebody have time to go through a diff on Rietveld?

On Saturday, I will try to start doing the release around 16:00 UTC.


-- 
Regards,
Benjamin

From benjamin at python.org  Tue Mar  3 00:11:29 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 2 Mar 2009 17:11:29 -0600
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <loom.20090302T230553-35@post.gmane.org>
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<loom.20090302T230553-35@post.gmane.org>
Message-ID: <1afaf6160903021511i18a4cc4ct8c5d43e569bb7492@mail.gmail.com>

2009/3/2 Armin Ronacher <armin.ronacher at active-4.com>:
> Hi,
>
> Raymond Hettinger <python <at> rcn.com> writes:
>> When we use the class, we typically only spell-out the constructor
>> once while actually using the returned
>> object many times. ?So,
>> have we really saved any typing?
> I'm fine with the typed out name as well, but I still would prefer lowercase to
> stay consistent with defaultdict/dict.

+1



-- 
Regards,
Benjamin

From python at rcn.com  Tue Mar  3 00:13:34 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Mar 2009 15:13:34 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered
	directory	to	collections ready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org><ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com><49AC5A9C.3030507@gmail.com><A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<loom.20090302T230553-35@post.gmane.org>
Message-ID: <E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1>


> Unfortunately PEP 8 never really took off naming-wise, so we're mostly following
> the "reuse the naming scheme from existing code in the same module" rule, and I
> think there lowercase wins, thanks to defaultdict.

Traditionally, the all lowercase name referred to a C type.  The other classes in
collections are named Counter, UserDict, UserList, UserString, MutableMapping, etc.
Besides, the lowercase/uppercase distinction helps us distinguish functions from classes.
This is the way I've see every Python book do it since the dawn of time.


Raymond



From solipsis at pitrou.net  Tue Mar  3 00:19:02 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 2 Mar 2009 23:19:02 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?Interpreter_crash_in_test=5Fstrftime=3F?=
Message-ID: <loom.20090302T231559-720@post.gmane.org>

Hello everybody,

I'm trying the current py3k under a Windows virtual machine (with VS Express
2008), and the interpreter crashes in test_strftime in test_datetime.py.
Has anyone been getting something like that?

(not being a Windows user, it's a bit hard for me to investigate what's wrong)

Regards

Antoine.



From barry at python.org  Tue Mar  3 00:32:29 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 2 Mar 2009 18:32:29 -0500
Subject: [Python-Dev] Instructions on using git mirror
In-Reply-To: <bbaeab100902271424p7cb74a2dr69df34d4c12b19c9@mail.gmail.com>
References: <20090227052116.GA18994@arctrix.com>
	<e45086040902270636p57e269ffs3d6d59de6f01f941@mail.gmail.com>
	<20090227154817.GB24580@arctrix.com>
	<2778C919-3351-4B89-B450-3D59F87A3485@python.org>
	<bbaeab100902271231t31f4dcbbn5621a221529b0445@mail.gmail.com>
	<61339809-37B9-42FD-AFA5-4FBB82BB422C@python.org>
	<bbaeab100902271424p7cb74a2dr69df34d4c12b19c9@mail.gmail.com>
Message-ID: <129960B0-9FCA-428F-9C00-22831D7B4E27@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Feb 27, 2009, at 5:24 PM, Brett Cannon wrote:

> Where's a good place for this?  I'm thinking three bullets under  
> "Core Development" on this page: http://wiki.python.org/moin/

I've now moved the Bazaar instructions to the wiki.

See

* http://wiki.python.org/moin/FrontPage
* http://wiki.python.org/moin/Bazaar

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSaxsjXEjvBPtnXfVAQIqpQP9HHXHzbXG5D51QeKr3B9F4AppINrznndC
Jam/IN+zuy+nm0T8Tq41rCqywuQm5Cj5+oBI6U5pQj58nQ6YAJzBYuJ1/mmRZOYv
0fTzIt0OEVvzodHdqDiHYMxNDB/JyKFdx/BInbRWSdXJFc+4C+2pd618ywahP7YU
y0m9Xo3fXA0=
=GD1M
-----END PGP SIGNATURE-----

From guido at python.org  Tue Mar  3 00:38:41 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Mar 2009 15:38:41 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1>
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<loom.20090302T230553-35@post.gmane.org>
	<E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1>
Message-ID: <ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>

On Mon, Mar 2, 2009 at 3:13 PM, Raymond Hettinger <python at rcn.com> wrote:
>
>> Unfortunately PEP 8 never really took off naming-wise, so we're mostly
>> following
>> the "reuse the naming scheme from existing code in the same module" rule,
>> and I
>> think there lowercase wins, thanks to defaultdict.
>
> Traditionally, the all lowercase name referred to a C type. ?The other
> classes in
> collections are named Counter, UserDict, UserList, UserString,
> MutableMapping, etc.
> Besides, the lowercase/uppercase distinction helps us distinguish functions
> from classes.
> This is the way I've see every Python book do it since the dawn of time.

Then they're all wrong. In 3.0 we're moving away from this, e.g.
cPickle is gone, so is cStringIO. The implementation language should
not shine through. *Maybe* the "built-in status" should guide the
capitalization, so only built-in types are lowercase (str, int, dict
etc.).

Anyway, it seems the collections module in particular is already
internally inconsistent -- NamedTuple vs. defaultdict. In a sense
defaultdict is the odd one out here, since these are things you import
from some module, they're not built-in. Maybe it should be renamed to
NamedDict?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From amauryfa at gmail.com  Tue Mar  3 00:39:23 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 3 Mar 2009 00:39:23 +0100
Subject: [Python-Dev] Interpreter crash in test_strftime?
In-Reply-To: <loom.20090302T231559-720@post.gmane.org>
References: <loom.20090302T231559-720@post.gmane.org>
Message-ID: <e27efe130903021539m1990f24ci1bfb2746d06d8aaf@mail.gmail.com>

On Tue, Mar 3, 2009 at 00:19, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Hello everybody,
>
> I'm trying the current py3k under a Windows virtual machine (with VS Express
> 2008), and the interpreter crashes in test_strftime in test_datetime.py.
> Has anyone been getting something like that?
>
> (not being a Windows user, it's a bit hard for me to investigate what's wrong)

Let me guess: it a RuntimeError, "Invalid format directive", when
calling strftime.
CRT assertions where enabled a few weeks ago (issue4804)
It may be a bad merge from trunk to the py3k branch.

-- 
Amaury Forgeot d'Arc

From python at rcn.com  Tue Mar  3 00:43:05 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Mar 2009 15:43:05 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<loom.20090302T230553-35@post.gmane.org>
	<E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1>
	<ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>
Message-ID: <8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1>

[GvR]
> *Maybe* the "built-in status" should guide the
> capitalization, so only built-in types are lowercase (str, int, dict
> etc.).

That makes sense.


> Anyway, it seems the collections module in particular is already
> internally inconsistent -- NamedTuple vs. defaultdict.

FWIW, namedtuple() is a factory function that creates a class, it isn't
a class itself.  There are no instances of namedtuple().  Most functions
are all lowercase.  Don't know if that applies to factory functions too.


Raymond



From armin.ronacher at active-4.com  Tue Mar  3 00:44:11 2009
From: armin.ronacher at active-4.com (Armin Ronacher)
Date: Mon, 2 Mar 2009 23:44:11 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?PEP_372_--_Adding_an_ordered_directory_to?=
	=?utf-8?q?=09collections_ready_for_pronouncement?=
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<loom.20090302T230553-35@post.gmane.org>
	<E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1>
	<ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>
Message-ID: <loom.20090302T234213-91@post.gmane.org>

Hi,

Guido van Rossum <guido <at> python.org> writes:
> Anyway, it seems the collections module in particular is already
> internally inconsistent -- NamedTuple vs. defaultdict. In a sense
> defaultdict is the odd one out here, since these are things you import
> from some module, they're not built-in. Maybe it should be renamed to
> NamedDict?
I suppose you mean "DefaultDict".  That would actually be the best solution. 
Then the module would be consistent and the new ordered dict version would go by
the name "OrderedDict".

Regards,
Armin

PS.: so is datetime.datetime a builtin then? :)


From solipsis at pitrou.net  Tue Mar  3 00:47:01 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 03 Mar 2009 00:47:01 +0100
Subject: [Python-Dev] Interpreter crash in test_strftime?
In-Reply-To: <e27efe130903021539m1990f24ci1bfb2746d06d8aaf@mail.gmail.com>
References: <loom.20090302T231559-720@post.gmane.org>
	<e27efe130903021539m1990f24ci1bfb2746d06d8aaf@mail.gmail.com>
Message-ID: <1236037621.12524.5.camel@fsol>


Hi Amaury,

Le mardi 03 mars 2009 ? 00:39 +0100, Amaury Forgeot d'Arc a ?crit :
> >
> > I'm trying the current py3k under a Windows virtual machine (with VS Express
> > 2008), and the interpreter crashes in test_strftime in test_datetime.py.
> > Has anyone been getting something like that?
> >
> > (not being a Windows user, it's a bit hard for me to investigate what's wrong)
> 
> Let me guess: it a RuntimeError, "Invalid format directive", when
> calling strftime.

I don't know. I've built in release mode, and I only get a window
proposing me to send the crash information to Microsoft...
(I must mention that I recently upgraded that VM to XP SP3)

cheers

Antoine.



From guido at python.org  Tue Mar  3 00:47:22 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Mar 2009 15:47:22 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1>
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<loom.20090302T230553-35@post.gmane.org>
	<E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1>
	<ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>
	<8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1>
Message-ID: <ca471dc20903021547w69100e2ev1cf17e18480540c3@mail.gmail.com>

On Mon, Mar 2, 2009 at 3:43 PM, Raymond Hettinger <python at rcn.com> wrote:
> [GvR]
>>
>> *Maybe* the "built-in status" should guide the
>> capitalization, so only built-in types are lowercase (str, int, dict
>> etc.).
>
> That makes sense.
>
>
>> Anyway, it seems the collections module in particular is already
>> internally inconsistent -- NamedTuple vs. defaultdict.
>
> FWIW, namedtuple() is a factory function that creates a class, it isn't
> a class itself. ?There are no instances of namedtuple(). ?Most functions
> are all lowercase. ?Don't know if that applies to factory functions too.

This is unfortunately ambiguous; e.g. threading.Lock() is a factory
function too. Anyways, I was mistaken about this example; I should
have pointed to Counter and the UserXxx classes in collections.py.

On Mon, Mar 2, 2009 at 3:44 PM, Armin Ronacher <armin.ronacher at active-
> I suppose you mean "DefaultDict".

Yes, I've been distracted. :-(

> That would actually be the best solution.
> Then the module would be consistent and the new ordered dict version would go by
> the name "OrderedDict".

OK.

> PS.: so is datetime.datetime a builtin then? :)

Another historic accident. Like socket.socket. :-(

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From amauryfa at gmail.com  Tue Mar  3 00:53:44 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 3 Mar 2009 00:53:44 +0100
Subject: [Python-Dev] Interpreter crash in test_strftime?
In-Reply-To: <1236037621.12524.5.camel@fsol>
References: <loom.20090302T231559-720@post.gmane.org>
	<e27efe130903021539m1990f24ci1bfb2746d06d8aaf@mail.gmail.com>
	<1236037621.12524.5.camel@fsol>
Message-ID: <e27efe130903021553kf44b42fsc1f7792ff440cbd0@mail.gmail.com>

On Tue, Mar 3, 2009 at 00:47, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Hi Amaury,
>
> Le mardi 03 mars 2009 ? 00:39 +0100, Amaury Forgeot d'Arc a ?crit :
>> >
>> > I'm trying the current py3k under a Windows virtual machine (with VS Express
>> > 2008), and the interpreter crashes in test_strftime in test_datetime.py.
>> > Has anyone been getting something like that?
>> >
>> > (not being a Windows user, it's a bit hard for me to investigate what's wrong)
>>
>> Let me guess: it a RuntimeError, "Invalid format directive", when
>> calling strftime.
>
> I don't know. I've built in release mode, and I only get a window
> proposing me to send the crash information to Microsoft...
> (I must mention that I recently upgraded that VM to XP SP3)

This should be fixed with r70114.

-- 
Amaury Forgeot d'Arc

From python at rcn.com  Tue Mar  3 00:52:57 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 2 Mar 2009 15:52:57 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<loom.20090302T230553-35@post.gmane.org>
	<E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1>
	<ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>
Message-ID: <45D696B127FC48058B2FB0E3212F8DF1@RaymondLaptop1>

Quick question?  Is PEP 8 still current for what is being done in Py3.x?
I just took a quick look and it says:

    Class Names

      Almost without exception, class names use the CapWords convention.
      Classes for internal use have a leading underscore in addition.


----- Original Message ----- 
From: "Guido van Rossum" <guido at python.org>
To: "Raymond Hettinger" <python at rcn.com>
Cc: <python-dev at python.org>; "Armin Ronacher" <armin.ronacher at active-4.com>
Sent: Monday, March 02, 2009 3:38 PM
Subject: Re: [Python-Dev] PEP 372 -- Adding an ordered directory to collections ready for pronouncement


On Mon, Mar 2, 2009 at 3:13 PM, Raymond Hettinger <python at rcn.com> wrote:
>
>> Unfortunately PEP 8 never really took off naming-wise, so we're mostly
>> following
>> the "reuse the naming scheme from existing code in the same module" rule,
>> and I
>> think there lowercase wins, thanks to defaultdict.
>
> Traditionally, the all lowercase name referred to a C type. The other
> classes in
> collections are named Counter, UserDict, UserList, UserString,
> MutableMapping, etc.
> Besides, the lowercase/uppercase distinction helps us distinguish functions
> from classes.
> This is the way I've see every Python book do it since the dawn of time.

Then they're all wrong. In 3.0 we're moving away from this, e.g.
cPickle is gone, so is cStringIO. The implementation language should
not shine through. *Maybe* the "built-in status" should guide the
capitalization, so only built-in types are lowercase (str, int, dict
etc.).

Anyway, it seems the collections module in particular is already
internally inconsistent -- NamedTuple vs. defaultdict. In a sense
defaultdict is the odd one out here, since these are things you import
from some module, they're not built-in. Maybe it should be renamed to
NamedDict?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Mar  3 00:58:44 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 2 Mar 2009 15:58:44 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <45D696B127FC48058B2FB0E3212F8DF1@RaymondLaptop1>
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<loom.20090302T230553-35@post.gmane.org>
	<E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1>
	<ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>
	<45D696B127FC48058B2FB0E3212F8DF1@RaymondLaptop1>
Message-ID: <ca471dc20903021558x33dd8290y17daecc8394aaeb9@mail.gmail.com>

On Mon, Mar 2, 2009 at 3:52 PM, Raymond Hettinger <python at rcn.com> wrote:
> Quick question? ?Is PEP 8 still current for what is being done in Py3.x?
> I just took a quick look and it says:
>
> ? Class Names
>
> ? ? Almost without exception, class names use the CapWords convention.
> ? ? Classes for internal use have a leading underscore in addition.

Yes, this is still the rule for new classes.

I am *not* (have never been) in favor of a hasty overhaul of
established APIs. Some of these were fixed for 3.0 (e.g. cPickle). The
rest will just be deviant forever. Not a big deal as long as the
number is fixed and limit.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steve at holdenweb.com  Tue Mar  3 01:08:25 2009
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 02 Mar 2009 19:08:25 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <ca471dc20903021547w69100e2ev1cf17e18480540c3@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<loom.20090302T230553-35@post.gmane.org>	<E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1>	<ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>	<8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1>
	<ca471dc20903021547w69100e2ev1cf17e18480540c3@mail.gmail.com>
Message-ID: <gohse2$797$1@ger.gmane.org>

Guido van Rossum wrote:
> On Mon, Mar 2, 2009 at 3:43 PM, Raymond Hettinger <python at rcn.com> wrote:
>> [GvR]
>>> *Maybe* the "built-in status" should guide the
>>> capitalization, so only built-in types are lowercase (str, int, dict
>>> etc.).
>> That makes sense.
>>
>>
>>> Anyway, it seems the collections module in particular is already
>>> internally inconsistent -- NamedTuple vs. defaultdict.
>> FWIW, namedtuple() is a factory function that creates a class, it isn't
>> a class itself.  There are no instances of namedtuple().  Most functions
>> are all lowercase.  Don't know if that applies to factory functions too.
> 
> This is unfortunately ambiguous; e.g. threading.Lock() is a factory
> function too. Anyways, I was mistaken about this example; I should
> have pointed to Counter and the UserXxx classes in collections.py.
> 
> On Mon, Mar 2, 2009 at 3:44 PM, Armin Ronacher <armin.ronacher at active-
>> I suppose you mean "DefaultDict".
> 
> Yes, I've been distracted. :-(
> 
>> That would actually be the best solution.
>> Then the module would be consistent and the new ordered dict version would go by
>> the name "OrderedDict".
> 
> OK.
> 
>> PS.: so is datetime.datetime a builtin then? :)
> 
> Another historic accident. Like socket.socket. :-(
> 
A pity this stuff wasn't addressed for 3.0. Way too late now, though.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From solipsis at pitrou.net  Tue Mar  3 02:00:43 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Mar 2009 01:00:43 +0000 (UTC)
Subject: [Python-Dev] first 3.1 alpha this Saturday
References: <1afaf6160903021510td535759p7b33aa0db802de97@mail.gmail.com>
Message-ID: <loom.20090303T010012-958@post.gmane.org>

Benjamin Peterson <benjamin <at> python.org> writes:
> 
> I have been talking with Antoine about io in C implementation. We
> would like to merge the branch before Saturday, so it can get some
> real world testing. io-c, however, hasn't received much through
> review. Would somebody have time to go through a diff on Rietveld?

For people who want to take a look, the review is at:
http://codereview.appspot.com/22061

Regards

Antoine.



From solipsis at pitrou.net  Tue Mar  3 02:03:27 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 3 Mar 2009 01:03:27 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?Interpreter_crash_in_test=5Fstrftime=3F?=
References: <loom.20090302T231559-720@post.gmane.org>
	<e27efe130903021539m1990f24ci1bfb2746d06d8aaf@mail.gmail.com>
	<1236037621.12524.5.camel@fsol>
	<e27efe130903021553kf44b42fsc1f7792ff440cbd0@mail.gmail.com>
Message-ID: <loom.20090303T010306-881@post.gmane.org>

Amaury Forgeot d'Arc <amauryfa <at> gmail.com> writes:
> 
> This should be fixed with r70114.

Indeed, thanks!

Regards

Antoine.



From rasky at develer.com  Tue Mar  3 02:53:09 2009
From: rasky at develer.com (Giovanni Bajo)
Date: Tue, 3 Mar 2009 01:53:09 +0000 (UTC)
Subject: [Python-Dev] PEP 372 -- Adding an ordered
	directory	to	collections ready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
Message-ID: <goi2i5$n0n$1@ger.gmane.org>

On Mon, 02 Mar 2009 14:36:32 -0800, Raymond Hettinger wrote:

> [Nick Coghlan]
>> The examples in the PEP used 'odict' (until recently), but the patch
>> was for OrderedDict.
> 
> As an experiment, try walking down the hall asking a few programmers who
> aren't in this conversion what they think collections.odict() is?
> Is it a class or function?  What does it do?  Can the English as second
> language folks guess what the o stands for?  Is it a builtin or pure
> python?  My guess is that the experiment will be informative.

Just today, I was talking with a colleague (which is learning Python 
right now) about "ordered dict". His first thought was a dictionary that, 
when iterated, would return keys in sorted order.

I beleive he was partly misguided by his knowledge of C++. C++ has always 
had std::map which returns sorted data upon iteration (it's a binary 
tree); they're now adding std::unordered_map (and std::unordered_set), to 
be implemented with a hash table. So, if you come from C++, it's easy to 
mistake the meaning of an ordered dict.

This said, I don't have a specific suggestion, but I would stay with 
lowercase-only for simmetry with defaultdict.
-- 
Giovanni Bajo
Develer S.r.l.
http://www.develer.com


From tjreedy at udel.edu  Tue Mar  3 04:20:54 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 02 Mar 2009 22:20:54 -0500
Subject: [Python-Dev] ABCs and MRO
In-Reply-To: <49AC461C.5030503@gmail.com>
References: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>	<goh1fp$m43$1@ger.gmane.org>
	<49AC461C.5030503@gmail.com>
Message-ID: <goi7mj$u5$1@ger.gmane.org>

Nick Coghlan wrote:
> Terry Reedy wrote:
>>>> As for the actual feature, I don't think it should hold up releases.
>>> Fair enough.
>> Given that the purpose of 2.7 is
>> a) maintenance of existing code (which can include minor new features
>> for existing facilities), and
>> b) easing conversion of code to 3.1
>> I am puzzled at the idea of adding a new facility to 2.7 that would not
>> be in 3.1+.  It would lock code into 2.7+ and counter purpose b).
> 
> It's possible we will end up in a situation where 3.0 and 3.1 are both
> aligned with 2.6, while 2.7 aligns with 3.2. That's particularly so with
> only 6 months or so between 3.0 and 3.1, while I currently expect the
> gap between 2.6 and 2.7 to be closer to the traditional 18 months.

OK, that suggests that the new feature should only be committed, if 
ever, to 2.7 after 3.1, when it can also be committed to 3.2 at the same 
time.


From martin.hellwig at dcuktec.org  Tue Mar  3 03:14:57 2009
From: martin.hellwig at dcuktec.org (Martin P. Hellwig)
Date: Tue, 03 Mar 2009 02:14:57 +0000
Subject: [Python-Dev] =?windows-1252?q?=5BANN=5D_EuroPython_2009_=96_Open_?=
 =?windows-1252?q?for_Registration_and_Reminder_of_Participation!?=
Message-ID: <49AC92A1.1000806@dcuktec.org>

On behalf of the EuroPython 2009 organisation it is my privilege and 
honour to announce that EuroPython 2009 is open for registration!  
EuroPython is the conference for the communities around Python, 
including the Django, Zope and Plone communities.
This year's conference will be held in Birmingham, UK from Monday 30th 
June to Monday 2nd July 2009.
Preceding the conference, on Saturday 28th June and Sunday 29th June, 
are the tutorial days, which can be attended separately.

We have a special extra early bird rate, just 95 GBP for the conference 
and 70 GBP for the tutorials. Be quick, this offer only last untill the 
14th of March!
For your convenience we negotiated special hotel prices, you can book 
your conference and hotels all at ones. Register at 
http://www.europython.eu/registration/ .

Talks & Themes
We already received a number of very interesting talks, why not add yours?
Go to http://www.europython.eu/talks/cfp/ for this year's themes and 
submissions criteria, the deadline is on 5th April 2009.

Other Talks, Activities and Events
Have you got something which does not fit the above? Visit 
http://www.europython.eu/talks/ .

Help Us Out
We could use a hand, any contribution is welcome, please take a look at 
http://www.europython.eu/contact/ .

Sponsors
An unique opportunity to affiliate with the prestigious EuroPython 
conference!
http://www.europython.eu/sponsors/

Spread the Word
Improve our publicity by distributing this announcement in your corner 
of the community, coordinating this with the organizers is highly 
appreciated.     http://www.europython.eu/contact/

General Information
For more information about the conference, please visit 
http://www.europython.eu/ .

Looking forward to see you!

The EuroPython Team


From ncoghlan at gmail.com  Tue Mar  3 10:29:26 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 03 Mar 2009 19:29:26 +1000
Subject: [Python-Dev] ABCs and MRO
In-Reply-To: <goi7mj$u5$1@ger.gmane.org>
References: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>	<goh1fp$m43$1@ger.gmane.org>	<49AC461C.5030503@gmail.com>
	<goi7mj$u5$1@ger.gmane.org>
Message-ID: <49ACF876.8000904@gmail.com>

Terry Reedy wrote:
> Nick Coghlan wrote:
>> Terry Reedy wrote:
>>>>> As for the actual feature, I don't think it should hold up releases.
>>>> Fair enough.
>>> Given that the purpose of 2.7 is
>>> a) maintenance of existing code (which can include minor new features
>>> for existing facilities), and
>>> b) easing conversion of code to 3.1
>>> I am puzzled at the idea of adding a new facility to 2.7 that would not
>>> be in 3.1+.  It would lock code into 2.7+ and counter purpose b).
>>
>> It's possible we will end up in a situation where 3.0 and 3.1 are both
>> aligned with 2.6, while 2.7 aligns with 3.2. That's particularly so with
>> only 6 months or so between 3.0 and 3.1, while I currently expect the
>> gap between 2.6 and 2.7 to be closer to the traditional 18 months.
> 
> OK, that suggests that the new feature should only be committed, if
> ever, to 2.7 after 3.1, when it can also be committed to 3.2 at the same
> time.

Not really - there's already stuff in 3.0 that wasn't backported the
first time around. I suspect the discrepancy here relates to different
views on the purpose of future 2.x releases. I would demote your two
points above to be goals b) and c) and give its primary purpose as:

a) Provide an updated 2.x compatible release with new features for the
benefit of those not yet able to make the transition to the 3.x series.

Some of those new features will be additional 3.x series backports, some
of them will be new features that were added to both 2.x and 3.x at the
same time. The only thing that I believe *shouldn't* happen is for 2.7
to be released with new features that aren't yet available in a 3.x release.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From krstic at solarsail.hcs.harvard.edu  Tue Mar  3 12:01:07 2009
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?Q?Ivan_Krsti=C4=87?=)
Date: Tue, 3 Mar 2009 06:01:07 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <gohse2$797$1@ger.gmane.org>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<loom.20090302T230553-35@post.gmane.org>	<E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1>	<ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>	<8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1>
	<ca471dc20903021547w69100e2ev1cf17e18480540c3@mail.gmail.com>
	<gohse2$797$1@ger.gmane.org>
Message-ID: <D1F247C9-1D96-495C-841E-1DC6F8BBC2EB@solarsail.hcs.harvard.edu>

On Mar 2, 2009, at 7:08 PM, Steve Holden wrote:
>>> PS.: so is datetime.datetime a builtin then? :)
>>
>> Another historic accident. Like socket.socket. :-(
>>
> A pity this stuff wasn't addressed for 3.0. Way too late now, though.


It may be too late to rename the existing accidents, but why not add  
consistently-named aliases (socket.Socket, datetime.DateTime, etc) and  
strongly encourage their use in new code?

--
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | http://radian.org


From rdmurray at bitdance.com  Tue Mar  3 14:13:12 2009
From: rdmurray at bitdance.com (rdmurray at bitdance.com)
Date: Tue, 3 Mar 2009 08:13:12 -0500 (EST)
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
 currently conform
In-Reply-To: <D1F247C9-1D96-495C-841E-1DC6F8BBC2EB@solarsail.hcs.harvard.edu>
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<loom.20090302T230553-35@post.gmane.org>
	<E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1>
	<ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>
	<8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1>
	<ca471dc20903021547w69100e2ev1cf17e18480540c3@mail.gmail.com>
	<gohse2$797$1@ger.gmane.org>
	<D1F247C9-1D96-495C-841E-1DC6F8BBC2EB@solarsail.hcs.harvard.edu>
Message-ID: <Pine.LNX.4.64.0903030753060.12396@kimball.webabinitio.net>

On Tue, 3 Mar 2009 at 06:01, Ivan Krsti?~G wrote:
> On Mar 2, 2009, at 7:08 PM, Steve Holden wrote:
>> > >  PS.: so is datetime.datetime a builtin then? :)
>> > 
>> >  Another historic accident. Like socket.socket. :-(
>> >
>>  A pity this stuff wasn't addressed for 3.0. Way too late now, though.
>
>
> It may be too late to rename the existing accidents, but why not add 
> consistently-named aliases (socket.Socket, datetime.DateTime, etc) and 
> strongly encourage their use in new code?

As a user I'd be +1 on that.  In fact, I might even start using 'as'
in my own code for that purpose right now.  I've always felt vaguely
confused and disturbed whenever I imported 'datetime', but until this
discussion I didn't realize why :)  Thinking about it, I know I've
written 'from datetime import DateTime' a number of times and then had
to go back and fix my code when I tried to run it.  And I'm sure that
sometimes when that happens I've had to (re)read the docs (or do a 'dir')
to find out why my import wasn't working.

Having said all that out loud, I think I might be stronger than a +1 on
this idea.  I'd be willing to help with doc and even code patches once
I finish learning how to contribute properly.

--RDM

From tjreedy at udel.edu  Tue Mar  3 18:30:54 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 03 Mar 2009 12:30:54 -0500
Subject: [Python-Dev] ABCs and MRO
In-Reply-To: <49ACF876.8000904@gmail.com>
References: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>	<goh1fp$m43$1@ger.gmane.org>	<49AC461C.5030503@gmail.com>	<goi7mj$u5$1@ger.gmane.org>
	<49ACF876.8000904@gmail.com>
Message-ID: <gojpgb$rta$1@ger.gmane.org>

Nick Coghlan wrote:
> Terry Reedy wrote:

>> OK, that suggests that the new feature should only be committed, if
>> ever, to 2.7 after 3.1, when it can also be committed to 3.2 at the same
>> time.
> 
> Not really - there's already stuff in 3.0 that wasn't backported the
> first time around. 

Irrelevant.  It is intended that Py3 have features that never appear in 
Py2.  My suggestion is aimed at avoiding the possibility that a new 
feature such as simplegeneric appear in 2.7 and *not* in 3.x.
[snip]

 > The only thing that I believe *shouldn't* happen is for 2.7
> to be released with new features that aren't yet available in a 3.x release.

Which is the point of my suggestion -- correctly read and understood ;-).

Terry


From jyasskin at gmail.com  Tue Mar  3 19:07:52 2009
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Tue, 3 Mar 2009 10:07:52 -0800
Subject: [Python-Dev] ABCs and MRO
In-Reply-To: <79990c6b0903021314g30ca852fjb39f0bc4d89ada63@mail.gmail.com>
References: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>
	<5d44f72f0903021037i14b68d97s3d86612024c97893@mail.gmail.com>
	<79990c6b0903021314g30ca852fjb39f0bc4d89ada63@mail.gmail.com>
Message-ID: <5d44f72f0903031007n376e9cb4w9dbdb1900dd2a36b@mail.gmail.com>

On Mon, Mar 2, 2009 at 1:14 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/3/2 Jeffrey Yasskin <jyasskin at gmail.com>:
>> I tend to think it's a bug in ABCs. You seem to have thought of
>> several possible ways to fix it, and I don't have strong preferences
>> between them.
>
> I've discussed ways of fixing simplegeneric, but not of fixing the
> issue with ABCs. I'm not sure the ABC "issue" is fixable - after all,
> it's by design that ABCs can implement __issubclass__ and "magically"
> become superclasses of arbitrary classes as a result.
>
> I'm not happy about fixing simplegeneric, though, as the whole point
> was just to expose an existing implementation, because it might be
> generally useful. If we start expanding and enhancing it, there are
> better implementations already available on PyPI (better in the sense
> of having seen real-world use). And if they don't handle ABCs, then
> that might indicate that needing to handle ABCs isn't as vital as this
> discussion would seem to imply (but I have no real-world data myself
> to make such a claim).
>
>> One way to deal with __subclasscheck__ definitions other than
>> ABCMeta.__subclasscheck__ would be to make the "someone registered a
>> subclass" interface public in addition to the "tell me when someone
>> registers a subclass" interface. Then other __subclasscheck__
>> definitions could call the "someone registered a subclass" method when
>> the class is defined if they want to be usable with generic functions.
>
> I'm not sure I follow this. Can you clarify? My intuition is that *no*
> notification hook can deal with ABCs that code arbitrary checks in
> their __subclasscheck__ method (so there's no registration involved,
> particular classes just "magically" become subclasses).

Hm, oops, you're right. So you'd have to scan the relevant abcs and
cache the list that turns up true ... probably with cooperation from
the ABCs so you'd know when to invalidate the cache. Ick.
Unfortunately, I think overloading functions on Number or Iterable
would be really useful, and constraining it to only look at base
classes would be unfortunate.

>> You can get something similar to the mro by checking whether ABCs are
>> subclasses of each other.
>
> The key point about the MRO is that you get it just from the initial
> class, via __mro__. You're not checking if another class is *in* the
> MRO, but rather you're *generating* the list of classes in the MRO.

I was trying to reply to PJE's statement that "The hairier issue for
these types of systems is method precedence". I agree that getting the
set of abcs for an instance is more difficult that I'd thought.

From list8a.forest at tibit.com  Tue Mar  3 20:20:37 2009
From: list8a.forest at tibit.com (Forest)
Date: Tue, 3 Mar 2009 11:20:37 -0800 (PST)
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
Message-ID: <04fb076678f894104924b2289dbbbe5a.squirrel@webmail.sonic.net>

I'm looking forward to an ordered dictionary in the standard library,
especially for things like LRU caches.  I was just reading the PEP, and
caught this bit:

"Does OrderedDict.popitem() return a particular key/value pair?
Yes. It pops-off the most recently inserted new key and its corresponding
value."

Okay, but I'd also like a convenient and fast way to find the oldest entry
in an OrderedDict, which I think I'd need for an LRU cache.  Skimming the
current patch (od7.diff), I didn't notice one.  Perhaps I simply missed
something.  Shouldn't popitem() allow the caller to choose which end from
which to pop?



From guido at python.org  Tue Mar  3 20:54:18 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Mar 2009 11:54:18 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <04fb076678f894104924b2289dbbbe5a.squirrel@webmail.sonic.net>
References: <04fb076678f894104924b2289dbbbe5a.squirrel@webmail.sonic.net>
Message-ID: <ca471dc20903031154s3b391b73u37cbcab67f98013a@mail.gmail.com>

On Tue, Mar 3, 2009 at 11:20 AM, Forest <list8a.forest at tibit.com> wrote:
> I'm looking forward to an ordered dictionary in the standard library,
> especially for things like LRU caches. ?I was just reading the PEP, and
> caught this bit:
>
> "Does OrderedDict.popitem() return a particular key/value pair?
> Yes. It pops-off the most recently inserted new key and its corresponding
> value."
>
> Okay, but I'd also like a convenient and fast way to find the oldest entry
> in an OrderedDict, which I think I'd need for an LRU cache. ?Skimming the
> current patch (od7.diff), I didn't notice one. ?Perhaps I simply missed
> something.

What's your use case?

If you really need this you can do it easily by taking the first key
returned by the iterator:

def popfirstitem(o):
  key = next(iter(o))  # raises StopIteration if o is empty
  value = o[key]
  del o[key]
  return key, value

> Shouldn't popitem() allow the caller to choose which end from
> which to pop?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ajaksu at gmail.com  Tue Mar  3 20:57:23 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 3 Mar 2009 16:57:23 -0300
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
Message-ID: <2d75d7660903031157k20b4fa0ar91b56f20428e17d7@mail.gmail.com>

Hi,

Benjamin Peterson wrote:
> 3.1a1 March 7
> 3.1a2 April 4
> 3.1b1 May 2
> 3.1rc1 May 30
> 3.1rc2 June 13
> 3.1 Final June 27

Benjamin, I'd like to nominate a couple (minor) RFEs[1] and bug
fixes[2] for 3.1. By 'nominate' I mean 'group related issues together,
offer tests, docs, patches and/or reviews as needed and
ask-pretty-please-for-inclusion' :)

Would early post-3.1a1 versus pre-3.1a1 make a difference in
likelihood of proposed changes going in? I can try to come up with a
detailed list before March 5, but I'd rather present it next week,
after taking a look at all remaining open issues.

FWIW, further tracker cleanup should happen sometime next week, let me
know if you need any tracker janitorvelopment done :)

Regards,
Daniel

[1] Current list:
http://bugs.python.org/issue1097797
http://bugs.python.org/issue3244
http://bugs.python.org/issue736428
http://bugs.python.org/issue1175686
http://bugs.python.org/issue4733

[2] Examples:
http://bugs.python.org/issue4953
http://bugs.python.org/issue1074333

From ajaksu at gmail.com  Tue Mar  3 20:57:59 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 3 Mar 2009 16:57:59 -0300
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <p06240803c5d1f82dbc4b@10.0.1.221>
References: <mailman.29.1235991618.1178.python-dev@python.org>
	<p06240803c5d1f82dbc4b@10.0.1.221>
Message-ID: <2d75d7660903031157n6da1e241n3fc800ebff0eb4bc@mail.gmail.com>

Mitchell L Model wrote:
> Would whoever is responsible for IDLE please take a look at the patches
> I submitted for Python 2 & 3 [tracker IDs 5233 and 5234 respectively].
[...]
> I would really like to see them in 3.1. The patch is already there;
> someone just has to do whatever gets done with patches to validate it
> and check it in. It's not a lot of code changes.

Mitchell, thanks for the reports and patches you've been contributing.
FWIW, I plan to follow up on these specific issues (and 5276) before
3.1a2, mostly to add tests and a +1 for integration.

Regards,
Daniel

From python at rcn.com  Tue Mar  3 21:02:14 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 3 Mar 2009 12:02:14 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
References: <04fb076678f894104924b2289dbbbe5a.squirrel@webmail.sonic.net>
Message-ID: <4205208755B141AA97A662B87BA76FC5@RaymondLaptop1>


[Forest]
> Okay, but I'd also like a convenient and fast way to find the oldest entry
> in an OrderedDict, which I think I'd need for an LRU cache.  Skimming the
> current patch (od7.diff), I didn't notice one.  Perhaps I simply missed
> something.  Shouldn't popitem() allow the caller to choose which end from
> which to pop?

Forest, I've taken another look at what's involved and am inclined to recommend the idea.  It can be done without mucking-up the 
regular dict API and without precluding any of the other possible underlying algorithms.  I like that it supports an entire new 
categoy of use cases with only a trivial, easily-understood API extension.  Patch is attached at http://bugs.python.org/issue5397


Raymond 


From list8a.forest at tibit.com  Tue Mar  3 21:07:38 2009
From: list8a.forest at tibit.com (Forest)
Date: Tue, 3 Mar 2009 12:07:38 -0800 (PST)
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <ca471dc20903031154s3b391b73u37cbcab67f98013a@mail.gmail.com>
References: <04fb076678f894104924b2289dbbbe5a.squirrel@webmail.sonic.net>
	<ca471dc20903031154s3b391b73u37cbcab67f98013a@mail.gmail.com>
Message-ID: <bcdf198b43edcc7ccde2eeef0478569a.squirrel@webmail.sonic.net>

On Tue, March 3, 2009 11:54 am, Guido van Rossum wrote:
> On Tue, Mar 3, 2009 at 11:20 AM, Forest <list8a.forest at tibit.com> wrote:
>> Okay, but I'd also like a convenient and fast way to find the oldest
>> entry
>> in an OrderedDict, which I think I'd need for an LRU cache. ?Skimming
>> the
>> current patch (od7.diff), I didn't notice one. ?Perhaps I simply missed
>> something.
>
> What's your use case?

An LRU cache which occasionally needs to scan the oldest keys in the
underlying odict to see if they should be purged.

> If you really need this you can do it easily by taking the first key
> returned by the iterator:

Yep.  I simply missed something.  That's what I get for reading patches
while hungry.  :)



From guido at python.org  Tue Mar  3 21:11:17 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Mar 2009 12:11:17 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <4205208755B141AA97A662B87BA76FC5@RaymondLaptop1>
References: <04fb076678f894104924b2289dbbbe5a.squirrel@webmail.sonic.net>
	<4205208755B141AA97A662B87BA76FC5@RaymondLaptop1>
Message-ID: <ca471dc20903031211y30170d40j2910c2c4aac7da2f@mail.gmail.com>

On Tue, Mar 3, 2009 at 12:02 PM, Raymond Hettinger <python at rcn.com> wrote:
>
> [Forest]
>>
>> Okay, but I'd also like a convenient and fast way to find the oldest entry
>> in an OrderedDict, which I think I'd need for an LRU cache. ?Skimming the
>> current patch (od7.diff), I didn't notice one. ?Perhaps I simply missed
>> something. ?Shouldn't popitem() allow the caller to choose which end from
>> which to pop?
>
> Forest, I've taken another look at what's involved and am inclined to
> recommend the idea. ?It can be done without mucking-up the regular dict API
> and without precluding any of the other possible underlying algorithms. ?I
> like that it supports an entire new categoy of use cases with only a
> trivial, easily-understood API extension. ?Patch is attached at
> http://bugs.python.org/issue5397

-1. This seems creeping featurism -- it's easy enough to do this
without adding a custom method. (And Forest admitted as much.)

However, reviewing your patch made me realize that the list of keys is
accessible as o._keys. I recommend that you either make this a
__private variable (signalling strongly that people shouldn't ever
reference it), or alternatively make it a public API. Otherwise, if
you just leave it _undocumented, people *are* going to figure out
clever things they can do with it (e.g. o._keys.sort() will turn the
odict into a sorted dict).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Tue Mar  3 21:20:56 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 3 Mar 2009 12:20:56 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
References: <04fb076678f894104924b2289dbbbe5a.squirrel@webmail.sonic.net>
	<4205208755B141AA97A662B87BA76FC5@RaymondLaptop1>
	<ca471dc20903031211y30170d40j2910c2c4aac7da2f@mail.gmail.com>
Message-ID: <9166409DE0FE49C09447ED6CFEEF8552@RaymondLaptop1>

> I recommend that you either make this a
> __private variable (signalling strongly that people shouldn't ever
> reference it),

Will do.  We want to make sure we can substitute a C implementation that has a completely different underlying structure (hash table 
plus a doubly linked list).


Raymond 


From tseaver at palladion.com  Tue Mar  3 21:23:16 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 03 Mar 2009 15:23:16 -0500
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <48EE2C77.1050809@palladion.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>	<5003.1223487033@parc.com>	<e6511dbf0810081426q4c576bbak8ab9a7e1d6936254@mail.gmail.com>	<29052.1223516380@parc.com>	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>
	<48EE2C77.1050809@palladion.com>
Message-ID: <49AD91B4.4070005@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tres Seaver wrote:
> Guido van Rossum wrote:
>> On Wed, Oct 8, 2008 at 6:39 PM, Bill Janssen <janssen at parc.com> wrote:
>>> Josiah Carlson <josiah.carlson at gmail.com> wrote:
>>>
>>>> But yes, zope needs to be changed to reflect the updated
>>>> asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
>>>> easier to use now.
>>> Just for completeness, I built a fresh 2.6, installed Medusa (from
>>> http://www.amk.ca/python/code/medusa.html), and it runs just fine (well,
>>> as well as it does on 2.5, anyway).  I think this is a Zope issue.
>> Way back in the day, Zope monkeypatched whole parts of asyncore,
>> replacing them with some of its own code. If that's still the case,
>> this breakage should be no surprise.
> 
> Zope has been hooking the 'asyncore.loop' function (wrapping it in order
> to add a "clean shutdown' flog) since 2001 at least (the 2.5 branch is
> the earliest checkout I have, and it was branched in early January 2002).
> 
> Zope also began patched asyncore's logging functions in 2.7, and later
> updated that patch to make the logger use the stdlib 'logging' module.
> 
> I think the change we need to make (in ZServer/medusa/http_server.py) is
> to emulate the new 'writeable' implementation in asynchat, at least when
> running under 2.6.
> 
> Zope's 'ZServer.medusa.http_server.http_request.writable()'
> implementation is::
> 
>     def writable (self):
>         # this is just the normal async_chat 'writable',
>         # here for comparison
>         return self.ac_out_buffer or len(self.producer_fifo)
> 
> 
> The Python 2.5 asynchat.asynchat.writable does::
> 
>     def writable (self):
>         "predicate for inclusion in the writable for select()"
>         # return len(self.ac_out_buffer) or len(self.producer_fifo) or
>         # (not self.connected)
>         # this is about twice as fast, though not as clear.
>         return not (
>                 (self.ac_out_buffer == '') and
>                 self.producer_fifo.is_empty() and
>                 self.connected
>                 )
> 
> The Python 2.6 asynchat.asynchat.writable now does::
> 
>     def writable (self):
>         "predicate for inclusion in the writable for select()"
>         return self.producer_fifo or (not self.connected)
> 
> 
> medusa 0.5.4's medusa.http_server.http_request doesn't override
> 'writable', but it does have a broken 'writeable_for_proxy':
> 
> 
>     def writable_for_proxy (self):
>         # this version of writable supports the idea of a 'stalled'
>         # producer
>         # [i.e., it's not ready to produce any output yet] This is
>         # needed by
>         # the proxy, which will be waiting for the magic combination of
>         # 1) hostname resolved
>         # 2) connection made
>         # 3) data available.
>         if self.ac_out_buffer:
>             return 1
>         elif len(self.producer_fifo):
>             p = self.producer_fifo.first()
>             if hasattr (p, 'stalled'):
>                 return not p.stalled()
>             else:
>                 return 1

*crickets*.

The 2.6 version of asyncore *also* breaks supervisord, which does *not*
use Zope's medusa:  it uses medusa 0.5.4.  How is anybody supposed to
write a package which sits atop a library like asyncore in a fashion
portable across Python versions?  The changes to the implementation in
2.6 (there is no real API) can't be reconciled, AFAICT.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJrZG0+gerLs4ltQ4RAnsmAJ9v/vPkHgE3AdP5ngVuYaKlxDGhJACgsCi2
3awbUffi2BU41qQgd6eJV18=
=WBt6
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Tue Mar  3 21:23:16 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 03 Mar 2009 15:23:16 -0500
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <48EE2C77.1050809@palladion.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>	<5003.1223487033@parc.com>	<e6511dbf0810081426q4c576bbak8ab9a7e1d6936254@mail.gmail.com>	<29052.1223516380@parc.com>	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>
	<48EE2C77.1050809@palladion.com>
Message-ID: <49AD91B4.4070005@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tres Seaver wrote:
> Guido van Rossum wrote:
>> On Wed, Oct 8, 2008 at 6:39 PM, Bill Janssen <janssen at parc.com> wrote:
>>> Josiah Carlson <josiah.carlson at gmail.com> wrote:
>>>
>>>> But yes, zope needs to be changed to reflect the updated
>>>> asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
>>>> easier to use now.
>>> Just for completeness, I built a fresh 2.6, installed Medusa (from
>>> http://www.amk.ca/python/code/medusa.html), and it runs just fine (well,
>>> as well as it does on 2.5, anyway).  I think this is a Zope issue.
>> Way back in the day, Zope monkeypatched whole parts of asyncore,
>> replacing them with some of its own code. If that's still the case,
>> this breakage should be no surprise.
> 
> Zope has been hooking the 'asyncore.loop' function (wrapping it in order
> to add a "clean shutdown' flog) since 2001 at least (the 2.5 branch is
> the earliest checkout I have, and it was branched in early January 2002).
> 
> Zope also began patched asyncore's logging functions in 2.7, and later
> updated that patch to make the logger use the stdlib 'logging' module.
> 
> I think the change we need to make (in ZServer/medusa/http_server.py) is
> to emulate the new 'writeable' implementation in asynchat, at least when
> running under 2.6.
> 
> Zope's 'ZServer.medusa.http_server.http_request.writable()'
> implementation is::
> 
>     def writable (self):
>         # this is just the normal async_chat 'writable',
>         # here for comparison
>         return self.ac_out_buffer or len(self.producer_fifo)
> 
> 
> The Python 2.5 asynchat.asynchat.writable does::
> 
>     def writable (self):
>         "predicate for inclusion in the writable for select()"
>         # return len(self.ac_out_buffer) or len(self.producer_fifo) or
>         # (not self.connected)
>         # this is about twice as fast, though not as clear.
>         return not (
>                 (self.ac_out_buffer == '') and
>                 self.producer_fifo.is_empty() and
>                 self.connected
>                 )
> 
> The Python 2.6 asynchat.asynchat.writable now does::
> 
>     def writable (self):
>         "predicate for inclusion in the writable for select()"
>         return self.producer_fifo or (not self.connected)
> 
> 
> medusa 0.5.4's medusa.http_server.http_request doesn't override
> 'writable', but it does have a broken 'writeable_for_proxy':
> 
> 
>     def writable_for_proxy (self):
>         # this version of writable supports the idea of a 'stalled'
>         # producer
>         # [i.e., it's not ready to produce any output yet] This is
>         # needed by
>         # the proxy, which will be waiting for the magic combination of
>         # 1) hostname resolved
>         # 2) connection made
>         # 3) data available.
>         if self.ac_out_buffer:
>             return 1
>         elif len(self.producer_fifo):
>             p = self.producer_fifo.first()
>             if hasattr (p, 'stalled'):
>                 return not p.stalled()
>             else:
>                 return 1

*crickets*.

The 2.6 version of asyncore *also* breaks supervisord, which does *not*
use Zope's medusa:  it uses medusa 0.5.4.  How is anybody supposed to
write a package which sits atop a library like asyncore in a fashion
portable across Python versions?  The changes to the implementation in
2.6 (there is no real API) can't be reconciled, AFAICT.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJrZG0+gerLs4ltQ4RAnsmAJ9v/vPkHgE3AdP5ngVuYaKlxDGhJACgsCi2
3awbUffi2BU41qQgd6eJV18=
=WBt6
-----END PGP SIGNATURE-----


From benjamin at python.org  Tue Mar  3 21:27:44 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 3 Mar 2009 15:27:44 -0500
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <2d75d7660903031157k20b4fa0ar91b56f20428e17d7@mail.gmail.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<2d75d7660903031157k20b4fa0ar91b56f20428e17d7@mail.gmail.com>
Message-ID: <1afaf6160903031227i37de3800h2164bf92e0c424a4@mail.gmail.com>

2009/3/3 Daniel (ajax) Diniz <ajaksu at gmail.com>:
> Benjamin, I'd like to nominate a couple (minor) RFEs[1] and bug
> fixes[2] for 3.1. By 'nominate' I mean 'group related issues together,
> offer tests, docs, patches and/or reviews as needed and
> ask-pretty-please-for-inclusion' :)
>
> Would early post-3.1a1 versus pre-3.1a1 make a difference in
> likelihood of proposed changes going in? I can try to come up with a
> detailed list before March 5, but I'd rather present it next week,
> after taking a look at all remaining open issues.

Assuming you find reviews/committers for those patches, it's all good
until the first beta.



-- 
Regards,
Benjamin

From list8a.forest at tibit.com  Tue Mar  3 20:47:42 2009
From: list8a.forest at tibit.com (Forest)
Date: Tue, 3 Mar 2009 11:47:42 -0800 (PST)
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <04fb076678f894104924b2289dbbbe5a.squirrel@webmail.sonic.net>
References: <04fb076678f894104924b2289dbbbe5a.squirrel@webmail.sonic.net>
Message-ID: <54942cb29257e2f35aac571b03185ef5.squirrel@webmail.sonic.net>

On Tue, March 3, 2009 11:20 am, Forest wrote:
> Okay, but I'd also like a convenient and fast way to find the oldest entry
> in an OrderedDict, which I think I'd need for an LRU cache.  Skimming the
> current patch (od7.diff), I didn't notice one.  Perhaps I simply missed
> something.  Shouldn't popitem() allow the caller to choose which end from
> which to pop?

Thinking it through a bit more, and LRU cache would actually need to
access the oldest item before knowing whether to remove it.  Besides,
popitem() should probably retain the signature of dict.popitem().  I
therefore retract my suggestion about popitem().

Still, I think an LRU cache would be a very common use case for an ordered
dict.  The current implementation already uses a list to keep track of
order, which makes accessing the oldest key a simple matter of exposing
it.  Perhaps a new method like getfirst() would be worth while here?


From python at rcn.com  Tue Mar  3 21:41:49 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 3 Mar 2009 12:41:49 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
References: <04fb076678f894104924b2289dbbbe5a.squirrel@webmail.sonic.net>
	<54942cb29257e2f35aac571b03185ef5.squirrel@webmail.sonic.net>
Message-ID: <E698C65DCCE84EA8B3C5CDD7776FC5D5@RaymondLaptop1>

[Forest]
> Perhaps a new method like getfirst() would be worth while here?

Guido already gave you a way to access the first item using the existing API.
Using next(iter(d)) also works.


Raymond


From guido at python.org  Tue Mar  3 21:46:55 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Mar 2009 12:46:55 -0800
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <49AD91B4.4070005@palladion.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>
	<5003.1223487033@parc.com>
	<e6511dbf0810081426q4c576bbak8ab9a7e1d6936254@mail.gmail.com>
	<29052.1223516380@parc.com>
	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>
	<48EE2C77.1050809@palladion.com> <49AD91B4.4070005@palladion.com>
Message-ID: <ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>

On Tue, Mar 3, 2009 at 12:23 PM, Tres Seaver <tseaver at palladion.com> wrote:
> How is anybody supposed to
> write a package which sits atop a library like asyncore in a fashion
> portable across Python versions? ?The changes to the implementation in
> 2.6 (there is no real API) can't be reconciled, AFAICT.

This seems to be the crux of the problem with asyncore, ever since it
was added to the stdlib -- there's no real API, so every change
potentially breaks something. I wish we could start over with a proper
design under a new name. Maybe packages requiring asyncore
functionality should just copy the version of asyncore they like into
their own three and stick with that.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From glyph at divmod.com  Tue Mar  3 22:17:57 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Tue, 03 Mar 2009 21:17:57 -0000
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of	medusa
In-Reply-To: <ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>
	<5003.1223487033@parc.com>
	<e6511dbf0810081426q4c576bbak8ab9a7e1d6936254@mail.gmail.com>
	<29052.1223516380@parc.com>
	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>
	<48EE2C77.1050809@palladion.com> <49AD91B4.4070005@palladion.com>
	<ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
Message-ID: <20090303211757.12555.1049299398.divmod.xquotient.5145@weber.divmod.com>


On 08:46 pm, guido at python.org wrote:
>This seems to be the crux of the problem with asyncore, ever since it
>was added to the stdlib -- there's no real API, so every change
>potentially breaks something. I wish we could start over with a proper
>design under a new name.

Might I suggest "reactor"... or possibly "twisted", as that new name? 
;-)

(Sorry, I was trying to avoid this thread, but that was an opening I 
could drive a truck through).

In all seriousness, I seem to recall that Thomas Wouters was interested 
in doing integrating some portion of Twisted core into the standard 
library as of last PyCon.  I mention him specifically by name in the 
hopes that it will jog his memory.

At the very least, this might serve as a basis for an abstract API for 
asyncore:

http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html

From greg.ewing at canterbury.ac.nz  Tue Mar  3 22:08:44 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 04 Mar 2009 10:08:44 +1300
Subject: [Python-Dev] PEP 372 -- Adding an
 ordered	directory	to	collections ready for pronouncement
In-Reply-To: <goi2i5$n0n$1@ger.gmane.org>
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<goi2i5$n0n$1@ger.gmane.org>
Message-ID: <49AD9C5C.7040009@canterbury.ac.nz>

Giovanni Bajo wrote:

> Just today, I was talking with a colleague (which is learning Python 
> right now) about "ordered dict". His first thought was a dictionary that, 
> when iterated, would return keys in sorted order.

I wonder whether "indexed list" would be a more appropriate
name for what we're talking about here -- basically
a sequence type that holds things in arbitrary order, but
with the additional ability to look things up quickly by
a key.

-- 
Greg

From list8a.forest at tibit.com  Tue Mar  3 22:17:33 2009
From: list8a.forest at tibit.com (Forest)
Date: Tue, 3 Mar 2009 13:17:33 -0800 (PST)
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <mailman.4275.1236114291.11745.python-dev@python.org>
References: <mailman.4275.1236114291.11745.python-dev@python.org>
Message-ID: <29fbcc5ff7b492b0dddec37c0279c5de.squirrel@webmail.sonic.net>

Raymond Hettinger wrote:
> [Forest]
>> Perhaps a new method like getfirst() would be worth while here?
>
> Guido already gave you a way to access the first item using the existing
> API.
> Using next(iter(d)) also works.

Yep.  I think messages are arriving out of order.

-1 on my own suggestion.



From tseaver at palladion.com  Tue Mar  3 23:13:06 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 03 Mar 2009 17:13:06 -0500
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>	<5003.1223487033@parc.com>	<e6511dbf0810081426q4c576bbak8ab9a7e1d6936254@mail.gmail.com>	<29052.1223516380@parc.com>	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>	<48EE2C77.1050809@palladion.com>
	<49AD91B4.4070005@palladion.com>
	<ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
Message-ID: <49ADAB72.9000909@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Guido van Rossum wrote:
> On Tue, Mar 3, 2009 at 12:23 PM, Tres Seaver <tseaver at palladion.com> wrote:
>> How is anybody supposed to
>> write a package which sits atop a library like asyncore in a fashion
>> portable across Python versions?  The changes to the implementation in
>> 2.6 (there is no real API) can't be reconciled, AFAICT.
> 
> This seems to be the crux of the problem with asyncore, ever since it
> was added to the stdlib -- there's no real API, so every change
> potentially breaks something. I wish we could start over with a proper
> design under a new name. Maybe packages requiring asyncore
> functionality should just copy the version of asyncore they like into
> their own three and stick with that.

That was the very solution Chris came up with earlier today:  "stick a
fork in it, its done!"


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJraty+gerLs4ltQ4RAjBSAJ4niecZJusKY4XiioJ18mdhdMixxQCfWvcQ
Dwkh1ZBuxtGRbhUI4qy96Sc=
=ms0I
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Tue Mar  3 23:13:06 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 03 Mar 2009 17:13:06 -0500
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>	<5003.1223487033@parc.com>	<e6511dbf0810081426q4c576bbak8ab9a7e1d6936254@mail.gmail.com>	<29052.1223516380@parc.com>	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>	<48EE2C77.1050809@palladion.com>
	<49AD91B4.4070005@palladion.com>
	<ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
Message-ID: <49ADAB72.9000909@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Guido van Rossum wrote:
> On Tue, Mar 3, 2009 at 12:23 PM, Tres Seaver <tseaver at palladion.com> wrote:
>> How is anybody supposed to
>> write a package which sits atop a library like asyncore in a fashion
>> portable across Python versions?  The changes to the implementation in
>> 2.6 (there is no real API) can't be reconciled, AFAICT.
> 
> This seems to be the crux of the problem with asyncore, ever since it
> was added to the stdlib -- there's no real API, so every change
> potentially breaks something. I wish we could start over with a proper
> design under a new name. Maybe packages requiring asyncore
> functionality should just copy the version of asyncore they like into
> their own three and stick with that.

That was the very solution Chris came up with earlier today:  "stick a
fork in it, its done!"


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJraty+gerLs4ltQ4RAjBSAJ4niecZJusKY4XiioJ18mdhdMixxQCfWvcQ
Dwkh1ZBuxtGRbhUI4qy96Sc=
=ms0I
-----END PGP SIGNATURE-----


From guido at python.org  Tue Mar  3 23:20:13 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Mar 2009 14:20:13 -0800
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <20090303211757.12555.1049299398.divmod.xquotient.5145@weber.divmod.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>
	<5003.1223487033@parc.com>
	<e6511dbf0810081426q4c576bbak8ab9a7e1d6936254@mail.gmail.com>
	<29052.1223516380@parc.com>
	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>
	<48EE2C77.1050809@palladion.com> <49AD91B4.4070005@palladion.com>
	<ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
	<20090303211757.12555.1049299398.divmod.xquotient.5145@weber.divmod.com>
Message-ID: <ca471dc20903031420r62af26a1o93b6256c7d864acc@mail.gmail.com>

On Tue, Mar 3, 2009 at 1:17 PM,  <glyph at divmod.com> wrote:
>
> On 08:46 pm, guido at python.org wrote:
>>
>> This seems to be the crux of the problem with asyncore, ever since it
>> was added to the stdlib -- there's no real API, so every change
>> potentially breaks something. I wish we could start over with a proper
>> design under a new name.
>
> Might I suggest "reactor"... or possibly "twisted", as that new name? ;-)
>
> (Sorry, I was trying to avoid this thread, but that was an opening I could
> drive a truck through).
>
> In all seriousness, I seem to recall that Thomas Wouters was interested in
> doing integrating some portion of Twisted core into the standard library as
> of last PyCon. ?I mention him specifically by name in the hopes that it will
> jog his memory.
>
> At the very least, this might serve as a basis for an abstract API for
> asyncore:
>
> http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html

I hope we have learned from asyncore that a future-proof and maximally
flexible design of such an API is not easy. This is one of those cases
where I believe thinking hard up front will really pay off. In this
case I would recommend going through a full-blown PEP process before
committing to an API, to avoid making the same mistake twice. (I don't
mean this as a dig at Twisted, just as an observation about this
particular design problem. If Twisted solves all the problems already,
writing the PEP should be easy.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tjreedy at udel.edu  Tue Mar  3 23:33:27 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 03 Mar 2009 17:33:27 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <49AD9C5C.7040009@canterbury.ac.nz>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org>
	<49AD9C5C.7040009@canterbury.ac.nz>
Message-ID: <gokb7l$vvm$1@ger.gmane.org>

Greg Ewing wrote:
> Giovanni Bajo wrote:
> 
>> Just today, I was talking with a colleague (which is learning Python 
>> right now) about "ordered dict". His first thought was a dictionary 
>> that, when iterated, would return keys in sorted order.
> 
> I wonder whether "indexed list" would be a more appropriate
> name for what we're talking about here -- basically
> a sequence type that holds things in arbitrary order, but
> with the additional ability to look things up quickly by
> a key.

I almost agree, except that the API uses the dict, not list, API.  For 
instance, items are appended by adding a key, not with .append.  With 
sort not available and .popitem removing the last added item, 'indexed 
stack' would be a bit closer.  Indeed, I plan to try out odicts with 
graph algorithms that need keyed access to stacked items.

tjr



From Scott.Daniels at Acm.Org  Wed Mar  4 00:05:28 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Tue, 03 Mar 2009 15:05:28 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <54942cb29257e2f35aac571b03185ef5.squirrel@webmail.sonic.net>
References: <04fb076678f894104924b2289dbbbe5a.squirrel@webmail.sonic.net>
	<54942cb29257e2f35aac571b03185ef5.squirrel@webmail.sonic.net>
Message-ID: <gokcug$4a5$1@ger.gmane.org>

Forest wrote:
> On Tue, March 3, 2009 11:20 am, Forest wrote:
>> Okay, but I'd also like a convenient and fast way to find the oldest entry
>> in an OrderedDict, which I think I'd need for an LRU cache.  Skimming the
>> current patch (od7.diff), I didn't notice one.  Perhaps I simply missed
>> something.  Shouldn't popitem() allow the caller to choose which end from
>> which to pop?
> 
> Thinking it through a bit more, and LRU cache would actually need to
> access the oldest item before knowing whether to remove it.  Besides,
> popitem() should probably retain the signature of dict.popitem().  I
> therefore retract my suggestion about popitem().
> 
> Still, I think an LRU cache would be a very common use case for an ordered
> dict.  The current implementation already uses a list to keep track of
> order, which makes accessing the oldest key a simple matter of exposing
> it.  Perhaps a new method like getfirst() would be worth while here?

But you must decide if what you want really does LRU -- does accessing
the oldest entry make it the most recent entry?

--Scott David Daniels
Scott.Daniels at Acm.Org


From greg.ewing at canterbury.ac.nz  Wed Mar  4 01:51:03 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 04 Mar 2009 13:51:03 +1300
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <gokb7l$vvm$1@ger.gmane.org>
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<goi2i5$n0n$1@ger.gmane.org> <49AD9C5C.7040009@canterbury.ac.nz>
	<gokb7l$vvm$1@ger.gmane.org>
Message-ID: <49ADD077.2040307@canterbury.ac.nz>

Terry Reedy wrote:

> I almost agree, except that the API uses the dict, not list, API.

Yes, as long as the API is dict-like, it really needs to
be thought of as a kind of dict.

Perhaps the terminology should be

   ordereddict -- what we have here

   sorteddict -- hypothetical future type that keeps
                 itself sorted in key order

-- 
Greg

From guido at python.org  Wed Mar  4 01:49:52 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Mar 2009 16:49:52 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <gokcug$4a5$1@ger.gmane.org>
References: <04fb076678f894104924b2289dbbbe5a.squirrel@webmail.sonic.net>
	<54942cb29257e2f35aac571b03185ef5.squirrel@webmail.sonic.net>
	<gokcug$4a5$1@ger.gmane.org>
Message-ID: <ca471dc20903031649v7b19d46cq10745bb41283fa0f@mail.gmail.com>

On Tue, Mar 3, 2009 at 3:05 PM, Scott David Daniels
<Scott.Daniels at acm.org> wrote:
> Forest wrote:
>>
>> On Tue, March 3, 2009 11:20 am, Forest wrote:
>>>
>>> Okay, but I'd also like a convenient and fast way to find the oldest
>>> entry
>>> in an OrderedDict, which I think I'd need for an LRU cache. ?Skimming the
>>> current patch (od7.diff), I didn't notice one. ?Perhaps I simply missed
>>> something. ?Shouldn't popitem() allow the caller to choose which end from
>>> which to pop?
>>
>> Thinking it through a bit more, and LRU cache would actually need to
>> access the oldest item before knowing whether to remove it. ?Besides,
>> popitem() should probably retain the signature of dict.popitem(). ?I
>> therefore retract my suggestion about popitem().
>>
>> Still, I think an LRU cache would be a very common use case for an ordered
>> dict. ?The current implementation already uses a list to keep track of
>> order, which makes accessing the oldest key a simple matter of exposing
>> it. ?Perhaps a new method like getfirst() would be worth while here?
>
> But you must decide if what you want really does LRU -- does accessing
> the oldest entry make it the most recent entry?

Beware, deleting an item from an OrderedDict (in the current
implementation) is O(N). To implement an LRU cache you are probably
better off ignoring OrderedDict so that you can manipulate the list of
keys directly.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Wed Mar  4 02:15:34 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 3 Mar 2009 17:15:34 -0800
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
	currently conform
In-Reply-To: <Pine.LNX.4.64.0903030753060.12396@kimball.webabinitio.net>
References: <loom.20090301T182201-999@post.gmane.org>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<loom.20090302T230553-35@post.gmane.org>
	<E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1>
	<ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>
	<8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1>
	<ca471dc20903021547w69100e2ev1cf17e18480540c3@mail.gmail.com>
	<gohse2$797$1@ger.gmane.org>
	<D1F247C9-1D96-495C-841E-1DC6F8BBC2EB@solarsail.hcs.harvard.edu>
	<Pine.LNX.4.64.0903030753060.12396@kimball.webabinitio.net>
Message-ID: <bbaeab100903031715v1882585ah231881c3711ad688@mail.gmail.com>

On Tue, Mar 3, 2009 at 05:13, <rdmurray at bitdance.com> wrote:

> On Tue, 3 Mar 2009 at 06:01, Ivan Krsti?~G wrote:
>
>> On Mar 2, 2009, at 7:08 PM, Steve Holden wrote:
>>
>>> > >  PS.: so is datetime.datetime a builtin then? :)
>>> > >  Another historic accident. Like socket.socket. :-(
>>> >
>>>  A pity this stuff wasn't addressed for 3.0. Way too late now, though.
>>>
>>
>>
>> It may be too late to rename the existing accidents, but why not add
>> consistently-named aliases (socket.Socket, datetime.DateTime, etc) and
>> strongly encourage their use in new code?
>>
>
Or make the old names aliases for the new names and start a
PendingDeprecationWarning on the old names so they can be switched in the
distant future?


>
> As a user I'd be +1 on that.  In fact, I might even start using 'as'
> in my own code for that purpose right now.  I've always felt vaguely
> confused and disturbed whenever I imported 'datetime', but until this
> discussion I didn't realize why :)  Thinking about it, I know I've
> written 'from datetime import DateTime' a number of times and then had
> to go back and fix my code when I tried to run it.  And I'm sure that
> sometimes when that happens I've had to (re)read the docs (or do a 'dir')
> to find out why my import wasn't working.
>
> Having said all that out loud, I think I might be stronger than a +1 on
> this idea.  I'd be willing to help with doc and even code patches once
> I finish learning how to contribute properly.
>

+1 from me to fix these little mishaps in naming in both modules.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090303/0f45db7c/attachment.htm>

From guido at python.org  Wed Mar  4 02:25:21 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Mar 2009 17:25:21 -0800
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
	currently conform
In-Reply-To: <bbaeab100903031715v1882585ah231881c3711ad688@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>
	<loom.20090302T230553-35@post.gmane.org>
	<E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1>
	<ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>
	<8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1>
	<ca471dc20903021547w69100e2ev1cf17e18480540c3@mail.gmail.com>
	<gohse2$797$1@ger.gmane.org>
	<D1F247C9-1D96-495C-841E-1DC6F8BBC2EB@solarsail.hcs.harvard.edu>
	<Pine.LNX.4.64.0903030753060.12396@kimball.webabinitio.net>
	<bbaeab100903031715v1882585ah231881c3711ad688@mail.gmail.com>
Message-ID: <ca471dc20903031725h5a9ea4bfhc7167f6d2d9401dd@mail.gmail.com>

On Tue, Mar 3, 2009 at 5:15 PM, Brett Cannon <brett at python.org> wrote:
>
>
> On Tue, Mar 3, 2009 at 05:13, <rdmurray at bitdance.com> wrote:
>>
>> On Tue, 3 Mar 2009 at 06:01, Ivan Krsti?~G wrote:
>>>
>>> On Mar 2, 2009, at 7:08 PM, Steve Holden wrote:
>>>>
>>>> > > ?PS.: so is datetime.datetime a builtin then? :)
>>>> > > ?Another historic accident. Like socket.socket. :-(
>>>> >
>>>> ?A pity this stuff wasn't addressed for 3.0. Way too late now, though.

A pity indeed.

>>> It may be too late to rename the existing accidents, but why not add
>>> consistently-named aliases (socket.Socket, datetime.DateTime, etc) and
>>> strongly encourage their use in new code?
>
> Or make the old names aliases for the new names and start a
> PendingDeprecationWarning on the old names so they can be switched in the
> distant future?

+1, if it's not done in a rush and only for high-visibility modules --
let's start with socket and datetime.

We need a really long lead time before we can remove these. I
recommend starting with a *silent* deprecation in 3.1 combined with a
PR offensive for the new names.

>> As a user I'd be +1 on that. ?In fact, I might even start using 'as'
>> in my own code for that purpose right now. ?I've always felt vaguely
>> confused and disturbed whenever I imported 'datetime', but until this
>> discussion I didn't realize why :) ?Thinking about it, I know I've
>> written 'from datetime import DateTime' a number of times and then had
>> to go back and fix my code when I tried to run it. ?And I'm sure that
>> sometimes when that happens I've had to (re)read the docs (or do a 'dir')
>> to find out why my import wasn't working.
>>
>> Having said all that out loud, I think I might be stronger than a +1 on
>> this idea. ?I'd be willing to help with doc and even code patches once
>> I finish learning how to contribute properly.
>
> +1 from me to fix these little mishaps in naming in both modules.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Wed Mar  4 02:29:40 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 3 Mar 2009 17:29:40 -0800
Subject: [Python-Dev] Adding PEP consistent aliases for names that
	don'tcurrently conform
References: <loom.20090301T182201-999@post.gmane.org><loom.20090302T230553-35@post.gmane.org><E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1><ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com><8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1><ca471dc20903021547w69100e2ev1cf17e18480540c3@mail.gmail.com><gohse2$797$1@ger.gmane.org><D1F247C9-1D96-495C-841E-1DC6F8BBC2EB@solarsail.hcs.harvard.edu><Pine.LNX.4.64.0903030753060.12396@kimball.webabinitio.net><bbaeab100903031715v1882585ah231881c3711ad688@mail.gmail.com>
	<ca471dc20903031725h5a9ea4bfhc7167f6d2d9401dd@mail.gmail.com>
Message-ID: <290359B9F5F04DE48B39A2D2931F5727@RaymondLaptop1>


>>> It may be too late to rename the existing accidents, but why not add
>>> consistently-named aliases (socket.Socket, datetime.DateTime, etc) and
>>> strongly encourage their use in new code?
>
> Or make the old names aliases for the new names and start a
> PendingDeprecationWarning on the old names so they can be switched in the
> distant future?

Should the names in the __repr__ be changed now or later?

    >>> datetime(2008, 7, 31, 12, 0, 0)
    datetime.datetime(2008, 7, 31, 12, 0)


Raymond

From eric at trueblade.com  Wed Mar  4 02:30:41 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 03 Mar 2009 20:30:41 -0500
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
 currently conform
In-Reply-To: <ca471dc20903031725h5a9ea4bfhc7167f6d2d9401dd@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>	<loom.20090302T230553-35@post.gmane.org>	<E2975FCA2C704CA29C64E44FE748B649@RaymondLaptop1>	<ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>	<8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1>	<ca471dc20903021547w69100e2ev1cf17e18480540c3@mail.gmail.com>	<gohse2$797$1@ger.gmane.org>	<D1F247C9-1D96-495C-841E-1DC6F8BBC2EB@solarsail.hcs.harvard.edu>	<Pine.LNX.4.64.0903030753060.12396@kimball.webabinitio.net>	<bbaeab100903031715v1882585ah231881c3711ad688@mail.gmail.com>
	<ca471dc20903031725h5a9ea4bfhc7167f6d2d9401dd@mail.gmail.com>
Message-ID: <49ADD9C1.4020308@trueblade.com>

> We need a really long lead time before we can remove these. I
> recommend starting with a *silent* deprecation in 3.1 combined with a
> PR offensive for the new names.

I think the old names basically have to live forever in some way, due to 
loading old pickles. Remember the problems we had when we tried to 
restructure the library in 2.6?


From brett at python.org  Wed Mar  4 02:37:11 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 3 Mar 2009 17:37:11 -0800
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
	currently conform
In-Reply-To: <49ADD9C1.4020308@trueblade.com>
References: <loom.20090301T182201-999@post.gmane.org>
	<ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>
	<8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1>
	<ca471dc20903021547w69100e2ev1cf17e18480540c3@mail.gmail.com>
	<gohse2$797$1@ger.gmane.org>
	<D1F247C9-1D96-495C-841E-1DC6F8BBC2EB@solarsail.hcs.harvard.edu>
	<Pine.LNX.4.64.0903030753060.12396@kimball.webabinitio.net>
	<bbaeab100903031715v1882585ah231881c3711ad688@mail.gmail.com>
	<ca471dc20903031725h5a9ea4bfhc7167f6d2d9401dd@mail.gmail.com>
	<49ADD9C1.4020308@trueblade.com>
Message-ID: <bbaeab100903031737k751ab6dax67273f6ab874ecb9@mail.gmail.com>

On Tue, Mar 3, 2009 at 17:30, Eric Smith <eric at trueblade.com> wrote:

> We need a really long lead time before we can remove these. I
>> recommend starting with a *silent* deprecation in 3.1 combined with a
>> PR offensive for the new names.
>>
>
> I think the old names basically have to live forever in some way, due to
> loading old pickles. Remember the problems we had when we tried to
> restructure the library in 2.6?
>
>
Forever is a long time. =) If we keep the PendingDeprecationWarning for a
long time and really get the word out of the renames then people can migrate
their pickles over time. The real problem with the 2.6 reorg was that people
didn't want to have zero lead time to update their pickles and they way the
transition was being handled. In this case the old names and simply subclass
the new names and have no issues with old code.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090303/54af8ff4/attachment.htm>

From steve at holdenweb.com  Wed Mar  4 03:55:32 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 03 Mar 2009 21:55:32 -0500
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
 currently conform
In-Reply-To: <bbaeab100903031737k751ab6dax67273f6ab874ecb9@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>	<ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>	<8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1>	<ca471dc20903021547w69100e2ev1cf17e18480540c3@mail.gmail.com>	<gohse2$797$1@ger.gmane.org>	<D1F247C9-1D96-495C-841E-1DC6F8BBC2EB@solarsail.hcs.harvard.edu>	<Pine.LNX.4.64.0903030753060.12396@kimball.webabinitio.net>	<bbaeab100903031715v1882585ah231881c3711ad688@mail.gmail.com>	<ca471dc20903031725h5a9ea4bfhc7167f6d2d9401dd@mail.gmail.com>	<49ADD9C1.4020308@trueblade.com>
	<bbaeab100903031737k751ab6dax67273f6ab874ecb9@mail.gmail.com>
Message-ID: <49ADEDA4.8010903@holdenweb.com>

Brett Cannon wrote:
> 
> 
> On Tue, Mar 3, 2009 at 17:30, Eric Smith <eric at trueblade.com
> <mailto:eric at trueblade.com>> wrote:
> 
>         We need a really long lead time before we can remove these. I
>         recommend starting with a *silent* deprecation in 3.1 combined
>         with a
>         PR offensive for the new names.
> 
> 
>     I think the old names basically have to live forever in some way,
>     due to loading old pickles. Remember the problems we had when we
>     tried to restructure the library in 2.6?
> 
> 
> Forever is a long time. =) If we keep the PendingDeprecationWarning for
> a long time and really get the word out of the renames then people can
> migrate their pickles over time. The real problem with the 2.6 reorg was
> that people didn't want to have zero lead time to update their pickles
> and they way the transition was being handled. In this case the old
> names and simply subclass the new names and have no issues with old code.
> 
There's also no reason why someone couldn't write a pickle updater for
when such problems do rear their ugly heads.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at holdenweb.com  Wed Mar  4 03:55:32 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 03 Mar 2009 21:55:32 -0500
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
 currently conform
In-Reply-To: <bbaeab100903031737k751ab6dax67273f6ab874ecb9@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>	<ca471dc20903021538k42b8021ajb417bce911708841@mail.gmail.com>	<8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1>	<ca471dc20903021547w69100e2ev1cf17e18480540c3@mail.gmail.com>	<gohse2$797$1@ger.gmane.org>	<D1F247C9-1D96-495C-841E-1DC6F8BBC2EB@solarsail.hcs.harvard.edu>	<Pine.LNX.4.64.0903030753060.12396@kimball.webabinitio.net>	<bbaeab100903031715v1882585ah231881c3711ad688@mail.gmail.com>	<ca471dc20903031725h5a9ea4bfhc7167f6d2d9401dd@mail.gmail.com>	<49ADD9C1.4020308@trueblade.com>
	<bbaeab100903031737k751ab6dax67273f6ab874ecb9@mail.gmail.com>
Message-ID: <49ADEDA4.8010903@holdenweb.com>

Brett Cannon wrote:
> 
> 
> On Tue, Mar 3, 2009 at 17:30, Eric Smith <eric at trueblade.com
> <mailto:eric at trueblade.com>> wrote:
> 
>         We need a really long lead time before we can remove these. I
>         recommend starting with a *silent* deprecation in 3.1 combined
>         with a
>         PR offensive for the new names.
> 
> 
>     I think the old names basically have to live forever in some way,
>     due to loading old pickles. Remember the problems we had when we
>     tried to restructure the library in 2.6?
> 
> 
> Forever is a long time. =) If we keep the PendingDeprecationWarning for
> a long time and really get the word out of the renames then people can
> migrate their pickles over time. The real problem with the 2.6 reorg was
> that people didn't want to have zero lead time to update their pickles
> and they way the transition was being handled. In this case the old
> names and simply subclass the new names and have no issues with old code.
> 
There's also no reason why someone couldn't write a pickle updater for
when such problems do rear their ugly heads.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From benjamin at python.org  Wed Mar  4 05:09:18 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 3 Mar 2009 22:09:18 -0600
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
	currently conform
In-Reply-To: <bbaeab100903031737k751ab6dax67273f6ab874ecb9@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>
	<8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1>
	<ca471dc20903021547w69100e2ev1cf17e18480540c3@mail.gmail.com>
	<gohse2$797$1@ger.gmane.org>
	<D1F247C9-1D96-495C-841E-1DC6F8BBC2EB@solarsail.hcs.harvard.edu>
	<Pine.LNX.4.64.0903030753060.12396@kimball.webabinitio.net>
	<bbaeab100903031715v1882585ah231881c3711ad688@mail.gmail.com>
	<ca471dc20903031725h5a9ea4bfhc7167f6d2d9401dd@mail.gmail.com>
	<49ADD9C1.4020308@trueblade.com>
	<bbaeab100903031737k751ab6dax67273f6ab874ecb9@mail.gmail.com>
Message-ID: <1afaf6160903032009i440e36fdl9a97be3a78219ec6@mail.gmail.com>

2009/3/3 Brett Cannon <brett at python.org>:
>
>
> On Tue, Mar 3, 2009 at 17:30, Eric Smith <eric at trueblade.com> wrote:
>>>
>>> We need a really long lead time before we can remove these. I
>>> recommend starting with a *silent* deprecation in 3.1 combined with a
>>> PR offensive for the new names.
>>
>> I think the old names basically have to live forever in some way, due to
>> loading old pickles. Remember the problems we had when we tried to
>> restructure the library in 2.6?
>>
>
> Forever is a long time. =) If we keep the PendingDeprecationWarning for a
> long time and really get the word out of the renames then people can migrate
> their pickles over time. The real problem with the 2.6 reorg was that people
> didn't want to have zero lead time to update their pickles and they way the
> transition was being handled. In this case the old names and simply subclass
> the new names and have no issues with old code.

Yes, I'm already looking forward to Py4k now. :)



-- 
Regards,
Benjamin

From glyph at divmod.com  Wed Mar  4 06:31:07 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Wed, 04 Mar 2009 05:31:07 -0000
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of	medusa
In-Reply-To: <ca471dc20903031420r62af26a1o93b6256c7d864acc@mail.gmail.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>
	<5003.1223487033@parc.com>
	<e6511dbf0810081426q4c576bbak8ab9a7e1d6936254@mail.gmail.com>
	<29052.1223516380@parc.com>
	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>
	<48EE2C77.1050809@palladion.com> <49AD91B4.4070005@palladion.com>
	<ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
	<20090303211757.12555.1049299398.divmod.xquotient.5145@weber.divmod.com>
	<ca471dc20903031420r62af26a1o93b6256c7d864acc@mail.gmail.com>
Message-ID: <20090304053107.12555.475067956.divmod.xquotient.5237@weber.divmod.com>


On 3 Mar, 10:20 pm, guido at python.org wrote:
>On Tue, Mar 3, 2009 at 1:17 PM,  <glyph at divmod.com> wrote:

>>At the very least, this might serve as a basis for an abstract API for
>>asyncore:
>>
>>http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html

>I hope we have learned from asyncore that a future-proof and maximally
>flexible design of such an API is not easy. This is one of those cases
>where I believe thinking hard up front will really pay off. In this
>case I would recommend going through a full-blown PEP process before
>committing to an API, to avoid making the same mistake twice. (I don't
>mean this as a dig at Twisted, just as an observation about this
>particular design problem. If Twisted solves all the problems already,
>writing the PEP should be easy.)

I didn't see it as a dig at Twisted; you're right, this is a hard 
problem.  Twisted doesn't solve all the problems perfectly, either; much 
has been made of some rather unfortunate nuances of the IProducer and 
IConsumer interfaces.

I also regret that no Twisted people (myself included) seem to have the 
time to go through the PEP process and get something accepted.

Even if Twisted's interfaces aren't perfect, there's a lot of working 
application code which demonstrates that they are sufficient.  Perhaps 
more importantly, they are future-proof.  Every proposal we've come up 
with for addressing the more subtle shortcomings of these APIs has been 
completely backwards-compatible; there are no APIs with vague or broken 
semantics, just sub-optimal spellings.  The fact that there are already 
multiple reactor implementations which share very little code helps.

If someone who has PEP-writing skills and some free time is updating 
asyncore to be more modern, I'd be very happy to walk that person 
through Twisted's API design and explain how either portions of our 
implementation could be lifted for the stdlib, or how a "high level" 
layer could be written into asyncore so that the author of a particular 
chunk of low-level networking code could ignore whether they're using 
the stdlib mainloop or one of the Twisted ones.

From v+python at g.nevcal.com  Wed Mar  4 07:23:33 2009
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Tue, 03 Mar 2009 22:23:33 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <49ADD077.2040307@canterbury.ac.nz>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org>
	<49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org>
	<49ADD077.2040307@canterbury.ac.nz>
Message-ID: <49AE1E65.4000905@g.nevcal.com>

On approximately 3/3/2009 4:51 PM, came the following characters from 
the keyboard of Greg Ewing:
> Terry Reedy wrote:
> 
>> I almost agree, except that the API uses the dict, not list, API.
> 
> Yes, as long as the API is dict-like, it really needs to
> be thought of as a kind of dict.
> 
> Perhaps the terminology should be
> 
>   ordereddict -- what we have here
> 
>   sorteddict -- hypothetical future type that keeps
>                 itself sorted in key order
> 


FIFOdict ?  Yeah, that blows the capitalization scheme, way, way out.

The problem with the ordereddict/OrderedDict/odict is that there are way 
too many possible orderings, and without being more specific 
(InsertionSequenceOrderPreservingDictionary) people are doing to think 
"sort" when they hear "ordered".

I think FIFOdict is a reasonable abbreviation for 
InsertionSequenceOrderPreservingDictionary :)

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

From python at rcn.com  Wed Mar  4 08:22:22 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 3 Mar 2009 23:22:22 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>
	<49AE1E65.4000905@g.nevcal.com>
Message-ID: <D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>


>> Perhaps the terminology should be
>>
>>   ordereddict -- what we have here
>>
>>   sorteddict -- hypothetical future type that keeps
>>                 itself sorted in key order

+1


> FIFOdict ?  Yeah, that blows the capitalization scheme, way, way out.

Issues:
* The popitem() method is LIFO.
* In a non-popping context, there is no OUT.  It just stores.
* FIFO is more suggestive of queue behavior which does not apply here.
* Stores to existing keys don't go at the end; they leave the order unchanged.

FWIW, PEP 372 has links to seven other independent implementations and they all have names that are some variant spelling 
OrderedDict except for one which goes by the mysterious name of StableDict.

Am still +1 on painting the class green with pink polka dots, but I'm starting to appreciate why others are insisting on pink with 
green polka dots ;-)


Raymond 


From tjreedy at udel.edu  Wed Mar  4 08:47:25 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 04 Mar 2009 02:47:25 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <ca471dc20903031649v7b19d46cq10745bb41283fa0f@mail.gmail.com>
References: <04fb076678f894104924b2289dbbbe5a.squirrel@webmail.sonic.net>	<54942cb29257e2f35aac571b03185ef5.squirrel@webmail.sonic.net>	<gokcug$4a5$1@ger.gmane.org>
	<ca471dc20903031649v7b19d46cq10745bb41283fa0f@mail.gmail.com>
Message-ID: <golbm9$a9p$1@ger.gmane.org>

Guido van Rossum wrote:

> Beware, deleting an item from an OrderedDict (in the current
> implementation) is O(N).

Am I correct in presuming that that would not be true of .popitem?


From v+python at g.nevcal.com  Wed Mar  4 09:01:55 2009
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 04 Mar 2009 00:01:55 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>
	<49AE1E65.4000905@g.nevcal.com>
	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
Message-ID: <49AE3573.3020005@g.nevcal.com>

On approximately 3/3/2009 11:22 PM, came the following characters from 
the keyboard of Raymond Hettinger:
> 
>>> Perhaps the terminology should be
>>>
>>>   ordereddict -- what we have here
>>>
>>>   sorteddict -- hypothetical future type that keeps
>>>                 itself sorted in key order
> 
> +1

-1

Introducing the hypothetical sorteddict would serve to reduce the 
likelihood of ordereddict being interpreted as sorteddict among the 
small percentage of people that actually read the two lines that might 
mention it in the documentation, but wouldn't significantly aid the 
intuition of people who first encounter it in someone else's code.

And without an implementation, it would otherwise be documentation 
noise, not signal.


>> FIFOdict ?  Yeah, that blows the capitalization scheme, way, way out.
> 
> Issues:
> * The popitem() method is LIFO.


But traversal starts at the other end, if I understand correctly. 
popitem seems gratuitous (but handy and cheap)


> * In a non-popping context, there is no OUT.  It just stores.
> * FIFO is more suggestive of queue behavior which does not apply here.


It is suggestive of queue behavior, and the items are a queue if looked 
at from insertion, and traversal perspectives, if I understand 
correctly.  But without OUT, FIFO is a bit too aggressively suggestively 
  of queues... but not more so than ordereddict is a bit too suggestive 
of sorted behavior... And at least FIFO doesn't have the sorting 
connotation.


> * Stores to existing keys don't go at the end; they leave the order 
> unchanged.
> 
> FWIW, PEP 372 has links to seven other independent implementations and 
> they all have names that are some variant spelling OrderedDict except 
> for one which goes by the mysterious name of StableDict.


Well, just because six other independent implementations use a name with 
connotations that they don't live up to is no reason to perpetuate such 
foolishness, nor introduce it into the Python stdlib.

StableDict, eh?  That's not so mysterious, perhaps, if you think of 
stable sorts^H^H^H^H^H (whoops, there's the wrong connotation rearing 
its ugly head again, sorry).


> Am still +1 on painting the class green with pink polka dots, but I'm 
> starting to appreciate why others are insisting on pink with green polka 
> dots ;-)


Sure.  I didn't expect FIFOdict to be an extremely useful suggestion, 
but I wanted to make the point that if the name has an erroneous 
connotation, use a name that doesn't.  And to get the discussion above 
flowing, to find out more about your thinking in the matter.

InputOrderedDict might be more descriptive, yet not as long as that 
other atrocity I alluded to that you rightfully refused to quote :)

 From tree-walking, perhaps people would intuit the right connotations 
from InOrderDict which is no longer than ordereddict, but usually the 
tree is kept sorted too, so I'm afraid it might not be sufficient.

Maybe  SequenceDict ?  SeqPreservingDict ?  SeqDict ?  All of these talk 
about sequences, which are generally not implied to be sorted.  I like 
these well enough, and it is late enough, that I'm not going to think of 
more right now.

C'mon folks, brainstorm, don't complain about ordereddict if you can't 
come up with some alternatives for discussion!!!  (and some reasons why 
the suggestions might be good or bad)  Even your bad ideas might trigger 
a good name in someone else's head...


-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

From steve at pearwood.info  Wed Mar  4 08:58:09 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 4 Mar 2009 18:58:09 +1100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <49AE1E65.4000905@g.nevcal.com>
References: <loom.20090301T182201-999@post.gmane.org>
	<49ADD077.2040307@canterbury.ac.nz> <49AE1E65.4000905@g.nevcal.com>
Message-ID: <200903041858.09267.steve@pearwood.info>

On Wed, 4 Mar 2009 05:23:33 pm Glenn Linderman wrote:
> The problem with the ordereddict/OrderedDict/odict is that there
> are way too many possible orderings, and without being more
> specific (InsertionSequenceOrderPreservingDictionary) people are
> doing to think "sort" when they hear "ordered".

For what it's worth, the first time I heard the term "ordered 
dictionary", I assumed it would be a dict where the keys are kept in 
sorted order.

But so what? Making things easy is an admirable goal, but we shouldn't 
lose sight of the fact that Python is a programming language, not a 
door handle. There's no requirement that every last feature is easy 
to intuit for a naive user. It's okay if people sometimes guess 
wrong, so long as they have opportunity to learn better.

Speaking as an ignorant and lazy programmer, being user-friendly is 
one thing, but molly-coddling the ignorant and lazy is another. 
Especially when it takes just a few seconds to type "import 
collections; help(collections.odict)" and learn that the O stands for 
ordered, and that the order is specifically order of insertion rather 
than lexicographic order. Having good help text is user-friendly. 
Naming the class InsertionSequenceOrderPreservingDictionary is just 
dumbing down at the cost of usability. I trust this wasn't a serious 
suggestion, but just in case it was, I'll point out that we have dict 
instead of UnorderedKeyValueMapping. Does anyone think that people 
find Python harder to learn because of that choice?


> I think FIFOdict is a reasonable abbreviation for 
> InsertionSequenceOrderPreservingDictionary :)

I see your smiley, but in my opinion, the sort of programmer who can't 
work out what an OrderedDict (or odict) is, given the name, the doc 
string and the Internet, is going to have even more trouble working 
out what FIFOdict means.


I have no strong feelings either way between odict and OrderedDict. 
PEP 8 seems to demand OrderedDict, but I actually prefer odict on the 
basis that an ordered dictionary feels like a fundamental data 
structure like str, list and dict rather than a specialist class like 
HTTPBasicAuthHandler. (I realise that "fundamental data structure" is 
not a well-defined category.) I look forward to the day OrderedDict 
becomes a built-in so it can be renamed odict :)



-- 
Steven

From gisle at activestate.com  Wed Mar  4 10:42:30 2009
From: gisle at activestate.com (Gisle Aas)
Date: Wed, 4 Mar 2009 10:42:30 +0100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <49AE3573.3020005@g.nevcal.com>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>
	<49AE1E65.4000905@g.nevcal.com>
	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
	<49AE3573.3020005@g.nevcal.com>
Message-ID: <918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>

On Mar 4, 2009, at 9:01 , Glenn Linderman wrote:

> On approximately 3/3/2009 11:22 PM, came the following characters  
> from the keyboard of Raymond Hettinger:
>>>> Perhaps the terminology should be
>>>>
>>>>  ordereddict -- what we have here
>>>>
>>>>  sorteddict -- hypothetical future type that keeps
>>>>                itself sorted in key order
>> +1
>
> -1
>
> Introducing the hypothetical sorteddict would serve to reduce the  
> likelihood of ordereddict being interpreted as sorteddict among the  
> small percentage of people that actually read the two lines that  
> might mention it in the documentation, but wouldn't significantly  
> aid the intuition of people who first encounter it in someone else's  
> code.
>
> And without an implementation, it would otherwise be documentation  
> noise, not signal.

Instead of introducing a sorteddict I would instead suggest that the  
future should bring an odict with a sort method; possibly also  
keys_sorted and items_sorted methods.

I think this would simplify things and putting these methods into the  
odict documentation makes it clearer how it actually behaves for  
people that just scan the method index to get an impression of what  
the object is about.

Regards,
Gisle


From tleeuwenburg at gmail.com  Wed Mar  4 11:09:03 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Wed, 4 Mar 2009 21:09:03 +1100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>
References: <loom.20090301T182201-999@post.gmane.org>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<goi2i5$n0n$1@ger.gmane.org> <49AD9C5C.7040009@canterbury.ac.nz>
	<gokb7l$vvm$1@ger.gmane.org> <49ADD077.2040307@canterbury.ac.nz>
	<49AE1E65.4000905@g.nevcal.com>
	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
	<49AE3573.3020005@g.nevcal.com>
	<918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>
Message-ID: <43c8685c0903040209h2a164c81peeee0b723788806e@mail.gmail.com>

Hello all,

First a comment on-thread:
  I can't wait to get an ordered dictionary in the stdlib! The discussion
regarding suggestions for the name appears to be ongoing. What about the
name 'orderdict' instead of 'ordereddict'?. It doesn't contain the double-d,
is slightly shorter, and I think a little more typo-friendly. Just my 2c,
please feel free to ignore. OrderDict would of course be the
alternative-casing version.

Secondly, regarding this list:
  I couldn't find a lot of documentation regarding list culture, so I'm
quite nervous about the potential for stepping in where I'm not welcome
before I have spent a lot of time 'spinning up' on this issues of this list.
I am interested in learning more about how Python is written and, if I can,
providing some assistance to that task. However it's going to be a long,
slow process for me. At the same time, I do have firsthand knowledge of how
hard it can be to get people to contribute to anything to any degree, and as
such don't want to be a part of the problem by being too tentative in
introducing myself.

I would appreciate any pointers regarding what is appreciated on this list
and what is not. Hopefully, over time I will be able to make some form of
useful, concrete code contributions in the form of patches or documentation,
but I do realise it will take a lot of hands-on learning first. Trying to
grok the discussions on this list seems like a big part of that.

Thanks,
-Tennessee
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090304/5e7076b0/attachment.htm>

From steve at pearwood.info  Wed Mar  4 12:17:25 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 04 Mar 2009 22:17:25 +1100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
 to	collections ready for pronouncement
In-Reply-To: <918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<49AE3573.3020005@g.nevcal.com>
	<918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>
Message-ID: <49AE6345.5010703@pearwood.info>

Gisle Aas wrote:

> Instead of introducing a sorteddict I would instead suggest that the 
> future should bring an odict with a sort method; possibly also 
> keys_sorted and items_sorted methods.

Instead of odict.sorted(), that can be spelled:

sorted(odict)  # sort the keys
sorted(odict.values())  # sort the items
sorted(odict.items())  # sort the (key, value) pairs

More complex variations are also possible.

The idea of a SortedDict is that it should be sorted at all times, 
without needing an explicit sort method, e.g.:

D = SortedDict(d=1, a=1, b=1)
print D
=> SortedDict(a=1, b=1, d=1)
D['c'] = 1
print D
=> SortedDict(a=1, b=1, c=1, d=1)

If you need to call a sort method on the dict to generate the sorted 
version, you might as well just pass the values you want to sorted(). 
That's more flexible, as you can sort whatever you want by anything you 
like.

I only know one use-case for a SortedDict: doctests. It's hard to use 
dicts in doctests, because when you print the dict, the items appear in 
arbitrary order. If you had a SortedDict, you could always predict what 
the dict would look like and use it in a doctest. Possibly there are 
other use-cases, but if so I don't know what they are.



-- 
Steven


From ncoghlan at gmail.com  Wed Mar  4 12:23:26 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 04 Mar 2009 21:23:26 +1000
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
 currently conform
In-Reply-To: <1afaf6160903032009i440e36fdl9a97be3a78219ec6@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>	<8A3BAFF93D204229BF56BD194F2BF5B0@RaymondLaptop1>	<ca471dc20903021547w69100e2ev1cf17e18480540c3@mail.gmail.com>	<gohse2$797$1@ger.gmane.org>	<D1F247C9-1D96-495C-841E-1DC6F8BBC2EB@solarsail.hcs.harvard.edu>	<Pine.LNX.4.64.0903030753060.12396@kimball.webabinitio.net>	<bbaeab100903031715v1882585ah231881c3711ad688@mail.gmail.com>	<ca471dc20903031725h5a9ea4bfhc7167f6d2d9401dd@mail.gmail.com>	<49ADD9C1.4020308@trueblade.com>	<bbaeab100903031737k751ab6dax67273f6ab874ecb9@mail.gmail.com>
	<1afaf6160903032009i440e36fdl9a97be3a78219ec6@mail.gmail.com>
Message-ID: <49AE64AE.4090008@gmail.com>

Benjamin Peterson wrote:
> Yes, I'm already looking forward to Py4k now. :)

Shh, Guido will need at least 5 years before he's ready to contemplate
going through something like this again.

Or maybe a decade to be on the safe side ;)

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From python at rcn.com  Wed Mar  4 12:43:36 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 4 Mar 2009 03:43:36 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
References: <04fb076678f894104924b2289dbbbe5a.squirrel@webmail.sonic.net>	<54942cb29257e2f35aac571b03185ef5.squirrel@webmail.sonic.net>	<gokcug$4a5$1@ger.gmane.org><ca471dc20903031649v7b19d46cq10745bb41283fa0f@mail.gmail.com>
	<golbm9$a9p$1@ger.gmane.org>
Message-ID: <4537274C3387443BA6CBCEEA391769F1@RaymondLaptop1>


>> Beware, deleting an item from an OrderedDict (in the current
>> implementation) is O(N).
> 
> Am I correct in presuming that that would not be true of .popitem?

Yes.  The popitem() method is O(1).


Raymond

From steve at holdenweb.com  Wed Mar  4 12:44:56 2009
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 04 Mar 2009 06:44:56 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>
	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
Message-ID: <golpk2$mld$1@ger.gmane.org>

Raymond Hettinger wrote:
[...]
> FWIW, PEP 372 has links to seven other independent implementations and
> they all have names that are some variant spelling OrderedDict except
> for one which goes by the mysterious name of StableDict.
> 
> Am still +1 on painting the class green with pink polka dots, but I'm
> starting to appreciate why others are insisting on pink with green polka
> dots ;-)
> 
This will be no surprise to those who have seen the many discussions on
ordered dicts that c.l.py has spawned over the years.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From hrvoje.niksic at avl.com  Wed Mar  4 12:48:42 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Wed, 04 Mar 2009 12:48:42 +0100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
 to	collections ready for pronouncement
In-Reply-To: <32857794.2579598.1236165549728.JavaMail.xicrypt@atgrzls001>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<49AE3573.3020005@g.nevcal.com>	<918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>
	<32857794.2579598.1236165549728.JavaMail.xicrypt@atgrzls001>
Message-ID: <49AE6A9A.8000306@avl.com>

Steven D'Aprano wrote:
> Gisle Aas wrote:
> 
>> Instead of introducing a sorteddict I would instead suggest that the 
>> future should bring an odict with a sort method; possibly also 
>> keys_sorted and items_sorted methods.
> 
> Instead of odict.sorted(), that can be spelled:
> 
> sorted(odict)  # sort the keys
> sorted(odict.values())  # sort the items
> sorted(odict.items())  # sort the (key, value) pairs

All of these are useful, but not when you want to sort the odict 
in-place.  Since ordered dict is all about order, a method for changing 
the underlying key order seems quite useful.  An odict.sort() would be 
easy to implement both in the current code (where it would delegate to 
self._keys.sort()) and in an alternative implementation using a linked 
list of keys (variants of mergesort work very nicely with linked lists).

> The idea of a SortedDict is that it should be sorted at all times,

A SortedDict would be nice to have, but we might rather want to use a 
balanced tree for the its C implementation, i.e. not inherit from dict 
at all.

From steve at pearwood.info  Wed Mar  4 12:48:40 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Wed, 04 Mar 2009 22:48:40 +1100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <49AE3573.3020005@g.nevcal.com>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
	<49AE3573.3020005@g.nevcal.com>
Message-ID: <49AE6A98.5060701@pearwood.info>

Glenn Linderman wrote:

>>> FIFOdict ?  Yeah, that blows the capitalization scheme, way, way out.
[...]
> It is suggestive of queue behavior, and the items are a queue if looked 
> at from insertion, and traversal perspectives, if I understand 
> correctly.

Why is this relevant? Insertion and traversal are only two parts of dict 
behaviour, whether ordered, sorted or otherwise. You can lookup, delete 
or modify items anywhere in the dict, not just at the ends. Doesn't 
sound very queue-like to me.


[...]
>> FWIW, PEP 372 has links to seven other independent implementations and 
>> they all have names that are some variant spelling OrderedDict except 
>> for one which goes by the mysterious name of StableDict.
> 
> 
> Well, just because six other independent implementations use a name with 
> connotations that they don't live up to is no reason to perpetuate such 
> foolishness, nor introduce it into the Python stdlib.

I don't believe the name is any more misleading than "dict" itself, or 
"str". It is a standard well-known name. From Google:

Results 1 - 10 of about 264 for StableDict
Results 1 - 10 of about 6,880 for OrderedDict
Results 1 - 10 of about 21,700 for ODict

(I have made no effort to exclude false positives.)

Yes, if you are a Martian or somebody learning to program for the first 
time, you have to learn what an ordered dict is. So what? You had to 
learn what a "str" was the first time you came across one too.


> StableDict, eh?  That's not so mysterious, perhaps, if you think of 
> stable sorts^H^H^H^H^H (whoops, there's the wrong connotation rearing 
> its ugly head again, sorry).

What does it mean to say a dict is stable? That is doesn't decay and rot 
away if you leave it alone? Do items evaporate out of ordinary dicts if 
you leave them alone for long enough? That once you add an item, you 
can't remove it or mutate it? It's not clear what the Stable in 
StableDict could mean.


> I didn't expect FIFOdict to be an extremely useful suggestion, 
> but I wanted to make the point that if the name has an erroneous 
> connotation, use a name that doesn't.

FIFOdict is *far* more misleading, as it erroneously suggests that you 
can't (or at least shouldn't) access elements in the middle of the dict.


> From tree-walking, perhaps people would intuit the right connotations 
> from InOrderDict which is no longer than ordereddict, but usually the 
> tree is kept sorted too, so I'm afraid it might not be sufficient.

No, I'm sorry, that's wrong. Inorder traversal of a binary tree is 
nothing like order-of-insertion traversal of a dict.

node = 'a'
node.right = 'c'
node.left = 'b'
Inorder traversal of node gives: b, a, c.
Order-of-insertion traversal gives: a, c, b.


> C'mon folks, brainstorm, don't complain about ordereddict if you can't 
> come up with some alternatives for discussion!!!

There are two good alternatives: OrderedDict and odict, or as Raymond 
puts it, green with pink polka dots versus pink with green polka dots. I 
don't think there's much point in suggesting fluorescent orange with 
brown and grey stripes as well.



-- 
Steven

From ncoghlan at gmail.com  Wed Mar  4 12:49:51 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 04 Mar 2009 21:49:51 +1000
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
 to	collections ready for pronouncement
In-Reply-To: <43c8685c0903040209h2a164c81peeee0b723788806e@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org>
	<49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org>
	<49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<49AE3573.3020005@g.nevcal.com>	<918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>
	<43c8685c0903040209h2a164c81peeee0b723788806e@mail.gmail.com>
Message-ID: <49AE6ADF.9010507@gmail.com>

Tennessee Leeuwenburg wrote:
> Hello all,
> 
> First a comment on-thread:
>   I can't wait to get an ordered dictionary in the stdlib! The
> discussion regarding suggestions for the name appears to be ongoing.
> What about the name 'orderdict' instead of 'ordereddict'?. It doesn't
> contain the double-d, is slightly shorter, and I think a little more
> typo-friendly. Just my 2c, please feel free to ignore. OrderDict would
> of course be the alternative-casing version.

The naming discussion is largely shooting the breeze at this point - the
OrderedDict naming follows PEP 8 and has a decent history of use in this
context, and I don't believe the objections and alternatives are
compelling enough to get anyone to write the code necessary to change
it. For such a recent patch the "status quo wins by default" argument
isn't as strong as it can sometimes be, but it still carries some weight.

Someone might surprise me and come forward with a patch to change the
name, but I really doubt it at this point.

> Secondly, regarding this list:
> I would appreciate any pointers regarding what is appreciated on this
> list and what is not. Hopefully, over time I will be able to make some
> form of useful, concrete code contributions in the form of patches or
> documentation, but I do realise it will take a lot of hands-on learning
> first. Trying to grok the discussions on this list seems like a big part
> of that.

I'd say you're off to a good start - wanting to learn and understand the
existing culture rather than demanding that the current list members
adapt to *your* style is a great first step :)

As for the culture itself... 'respect' is the main word that comes to my
mind:
- respect for other people's time in trying to post messages that are
concise and to the point
- respect for other people's points of view in trying to resolve design
disagreements
- respect for other people's abilities in assuming that errors are
inadvertent mistakes or due to a small misunderstanding rather than a
result of sheer incompetence
- respect for Python's users in ensuring a variety of perspectives are
taken into account when considering changes

Also, you may have looked at this already, but if not, the developer
page has some useful pointers:
http://www.python.org/dev/
http://www.python.org/dev/culture/

Hope that helps!

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From steve at holdenweb.com  Wed Mar  4 12:48:39 2009
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 04 Mar 2009 06:48:39 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <43c8685c0903040209h2a164c81peeee0b723788806e@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org>
	<49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org>
	<49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<49AE3573.3020005@g.nevcal.com>	<918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>
	<43c8685c0903040209h2a164c81peeee0b723788806e@mail.gmail.com>
Message-ID: <49AE6A97.5010402@holdenweb.com>

Tennessee Leeuwenburg wrote:
> Hello all,
> 
> First a comment on-thread:
>   I can't wait to get an ordered dictionary in the stdlib! The
> discussion regarding suggestions for the name appears to be ongoing.
> What about the name 'orderdict' instead of 'ordereddict'?. It doesn't
> contain the double-d, is slightly shorter, and I think a little more
> typo-friendly. Just my 2c, please feel free to ignore. OrderDict would
> of course be the alternative-casing version.
> 
> Secondly, regarding this list:
>   I couldn't find a lot of documentation regarding list culture, so I'm
> quite nervous about the potential for stepping in where I'm not welcome
> before I have spent a lot of time 'spinning up' on this issues of this
> list. I am interested in learning more about how Python is written and,
> if I can, providing some assistance to that task. However it's going to
> be a long, slow process for me. At the same time, I do have firsthand
> knowledge of how hard it can be to get people to contribute to anything
> to any degree, and as such don't want to be a part of the problem by
> being too tentative in introducing myself.
> 
> I would appreciate any pointers regarding what is appreciated on this
> list and what is not. Hopefully, over time I will be able to make some
> form of useful, concrete code contributions in the form of patches or
> documentation, but I do realise it will take a lot of hands-on learning
> first. Trying to grok the discussions on this list seems like a big part
> of that.
> 
Just dive in. We'll savage you when you get out of line :^).

Seriously, as long as it's about development *of* rather than *with*
Python, you should be OK. A couple of weeks lurking (or reading the list
history) will tell you most things you need to know.

regards
 Steve

takes-all-sorts-to-make-a-world-ly y'rs  - steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From lie.1296 at gmail.com  Wed Mar  4 12:51:39 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Wed, 04 Mar 2009 22:51:39 +1100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<49AE3573.3020005@g.nevcal.com>
	<918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>
Message-ID: <golq0g$ooa$1@ger.gmane.org>

Gisle Aas wrote:
> On Mar 4, 2009, at 9:01 , Glenn Linderman wrote:
> 
>> On approximately 3/3/2009 11:22 PM, came the following characters from 
>> the keyboard of Raymond Hettinger:
>>>>> Perhaps the terminology should be
>>>>>
>>>>>  ordereddict -- what we have here
>>>>>
>>>>>  sorteddict -- hypothetical future type that keeps
>>>>>                itself sorted in key order
>>> +1
>>
>> -1
>>
>> Introducing the hypothetical sorteddict would serve to reduce the 
>> likelihood of ordereddict being interpreted as sorteddict among the 
>> small percentage of people that actually read the two lines that might 
>> mention it in the documentation, but wouldn't significantly aid the 
>> intuition of people who first encounter it in someone else's code.
>>
>> And without an implementation, it would otherwise be documentation 
>> noise, not signal.
> 
> Instead of introducing a sorteddict I would instead suggest that the 
> future should bring an odict with a sort method; possibly also 
> keys_sorted and items_sorted methods.
> 
> I think this would simplify things and putting these methods into the 
> odict documentation makes it clearer how it actually behaves for people 
> that just scan the method index to get an impression of what the object 
> is about.

How about making odict ordered by insertion order, then provide an 
optional argument for defining sorter? This optional argument must be a 
function/lambda/callable object and must be the first argument.

a = odict(bloh='foo', blah='faa')
a # odict([('bloh', 'foo'), ('blah', 'faa')])

b = odict(lambda a, b: (a[0] < b[0]), bloh='foo', blah='faa')
b # sorted by key: odict([('blah', 'faa'), ('bloh', 'foo')])

c = odict(lambda a, b: (a[1] < b[1]), bloh='foo', blah='faa')
c # sorted by value: odict([('blah', 'faa'), ('bloh', 'foo')])

b = odict(lambda a, b: (a[0] > b[0]), bloh='foo', blah='faa')
b # sorted by key, descending: odict([('bloh', 'foo'), ('blah', 'faa')])


From steve at holdenweb.com  Wed Mar  4 12:45:46 2009
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 04 Mar 2009 06:45:46 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>
	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
Message-ID: <golplk$mld$2@ger.gmane.org>

Raymond Hettinger wrote:
> 
>>> Perhaps the terminology should be
>>>
>>>   ordereddict -- what we have here
>>>
>>>   sorteddict -- hypothetical future type that keeps
>>>                 itself sorted in key order
> 
> +1
> 
> 
>> FIFOdict ?  Yeah, that blows the capitalization scheme, way, way out.
> 
> Issues:
> * The popitem() method is LIFO.
> * In a non-popping context, there is no OUT.  It just stores.
> * FIFO is more suggestive of queue behavior which does not apply here.
> * Stores to existing keys don't go at the end; they leave the order
> unchanged.
> 
> FWIW, PEP 372 has links to seven other independent implementations and
> they all have names that are some variant spelling OrderedDict except
> for one which goes by the mysterious name of StableDict.
> 
> Am still +1 on painting the class green with pink polka dots, but I'm
> starting to appreciate why others are insisting on pink with green polka
> dots ;-)
> 
historydict?

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From ncoghlan at gmail.com  Wed Mar  4 13:25:42 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 04 Mar 2009 22:25:42 +1000
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <golq0g$ooa$1@ger.gmane.org>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<49AE3573.3020005@g.nevcal.com>	<918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>
	<golq0g$ooa$1@ger.gmane.org>
Message-ID: <49AE7346.50206@gmail.com>

Lie Ryan wrote:
 How about making odict ordered by insertion order, then provide an
> optional argument for defining sorter? This optional argument must be a
> function/lambda/callable object and must be the first argument.

As the PEP mentions (and Hrvoje brought up again already in this
thread), a hash table (i.e. dict) probably isn't the right data
structure to use as the basis for an "always sorted" container.
In-memory databases, balanced trees, etc, etc.

Further, unlike a more general "sorted" dictionary, an insertion ordered
dict already has specific use cases in the standard library.
ConfigParser will use it by default in 2.7/3.1 and namedtuple._asdict()
is being changed in those versions to return an OrderedDict so that
iterating over the result of _asdict() will process the fields in the
same order as iterating over the tuple itself.

It is also being added because an insertion ordered dictionary was the
primary example for the new metaclass __prepare__ method introduced by
PEP 3115. Adapting the example from that PEP:

  # The metaclass
  class OrderedClass(type):

    @classmethod
    def __prepare__(metacls, name, bases): # No keywords in this case
      return collections.OrderedDict()

    def __new__(cls, name, bases, classdict):
      # Note that we replace the classdict with a regular
      # dict before passing it to the superclass, so that we
      # don't continue to record the order after the class
      # has been created.
      result = type.__new__(cls, name, bases, dict(classdict))
      result.member_names = list(classdict.keys())
      return result

  # An instance of the metaclass
  class StructDef(metaclass=OrderedClass):
    # This dummy example uses types directly, but something
    # like struct module format codes may make more sense
    field1 = int
    field2 = float
    field3 = customType
    trailingField = str

Unlike a normal class definition, the order of the field definitions in
structure matters, and in the example above, this information is
preserved by the metaclass. This can greatly simplify the process of
defining types where the order of the fields matters (e.g. so the values
can be serialised in the correct order for a binary translation of some
kind).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From lie.1296 at gmail.com  Wed Mar  4 13:37:11 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Wed, 04 Mar 2009 23:37:11 +1100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <golplk$mld$2@ger.gmane.org>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
	<golplk$mld$2@ger.gmane.org>
Message-ID: <golslt$1as$1@ger.gmane.org>

Steve Holden wrote:
> Raymond Hettinger wrote:
>>>> Perhaps the terminology should be
>>>>
>>>>   ordereddict -- what we have here
>>>>
>>>>   sorteddict -- hypothetical future type that keeps
>>>>                 itself sorted in key order
>> +1
>>
>>
>>> FIFOdict ?  Yeah, that blows the capitalization scheme, way, way out.
>> Issues:
>> * The popitem() method is LIFO.
>> * In a non-popping context, there is no OUT.  It just stores.
>> * FIFO is more suggestive of queue behavior which does not apply here.
>> * Stores to existing keys don't go at the end; they leave the order
>> unchanged.
>>
>> FWIW, PEP 372 has links to seven other independent implementations and
>> they all have names that are some variant spelling OrderedDict except
>> for one which goes by the mysterious name of StableDict.
>>
>> Am still +1 on painting the class green with pink polka dots, but I'm
>> starting to appreciate why others are insisting on pink with green polka
>> dots ;-)
>>
> historydict?

agedict?


From rdmurray at bitdance.com  Wed Mar  4 13:53:36 2009
From: rdmurray at bitdance.com (rdmurray at bitdance.com)
Date: Wed, 4 Mar 2009 07:53:36 -0500 (EST)
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <golslt$1as$1@ger.gmane.org>
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>
	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>
	<49AE1E65.4000905@g.nevcal.com>
	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
	<golplk$mld$2@ger.gmane.org> <golslt$1as$1@ger.gmane.org>
Message-ID: <Pine.LNX.4.64.0903040739590.12396@kimball.webabinitio.net>

On Wed, 4 Mar 2009 at 23:37, Lie Ryan wrote:
> Steve Holden wrote:
>>  Raymond Hettinger wrote:
>> > > >  Perhaps the terminology should be
>> > > > 
>> > > >    ordereddict -- what we have here
>> > > > 
>> > > >    sorteddict -- hypothetical future type that keeps
>> > > >                  itself sorted in key order
>> >  +1
>> > 
>> > 
>> > >  FIFOdict ?  Yeah, that blows the capitalization scheme, way, way out.
>> >  Issues:
>> >  * The popitem() method is LIFO.
>> >  * In a non-popping context, there is no OUT.  It just stores.
>> >  * FIFO is more suggestive of queue behavior which does not apply here.
>> >  * Stores to existing keys don't go at the end; they leave the order
>> >  unchanged.
>> > 
>> >  FWIW, PEP 372 has links to seven other independent implementations and
>> >  they all have names that are some variant spelling OrderedDict except
>> >  for one which goes by the mysterious name of StableDict.
>> > 
>> >  Am still +1 on painting the class green with pink polka dots, but I'm
>> >  starting to appreciate why others are insisting on pink with green polka
>> >  dots ;-)
>> >
>>  historydict?
>
> agedict?

I actually like StableDict best.  When I hear that I think, "ah, the
key order is stable in the face of insertions, unlike a regular dict".
Nor can I at the moment think of an alternative explanation of what a
"StableDict" might be.

That said, I have no problem with keeping OrderedDict as the name.
("Ordered does not mean sorted, it means insertion order preserving"
may become a FQA (Frequent Question Answer :), but it is short and
clear and takes no longer than explaining what a StableDict _is_.)

Although, that might be another argument in favor of StableDict: since
unless you think what I wrote above you aren't going to have a clue what
it is, someone reading code and encountering it would be more likely to
look it up, whereas with OrderedDict someone is more likely to assume they
know what it is and get confused for a little while before looking it up.

Do I feel strongly enough about this to write a patch?  No :)

--RDM

From p.f.moore at gmail.com  Wed Mar  4 14:17:13 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 4 Mar 2009 13:17:13 +0000
Subject: [Python-Dev] ABCs and MRO
In-Reply-To: <5d44f72f0903031007n376e9cb4w9dbdb1900dd2a36b@mail.gmail.com>
References: <79990c6b0903020241qe19ca49yaf3645f52a46912d@mail.gmail.com>
	<5d44f72f0903021037i14b68d97s3d86612024c97893@mail.gmail.com>
	<79990c6b0903021314g30ca852fjb39f0bc4d89ada63@mail.gmail.com>
	<5d44f72f0903031007n376e9cb4w9dbdb1900dd2a36b@mail.gmail.com>
Message-ID: <79990c6b0903040517r20639e57j10f34f77093fa9f0@mail.gmail.com>

2009/3/3 Jeffrey Yasskin <jyasskin at gmail.com>:
> Unfortunately, I think overloading functions on Number or Iterable
> would be really useful, and constraining it to only look at base
> classes would be unfortunate.

That could well be the case. So the question is, I guess, how would
you write such a function now? Because you could always continue to
write it like that, but make it generic for the other special cases
you didn't think of:

    @simplegeneric
    def foo(x, y):
        if isinstance(x, Iterable):
            do iterable processing
        else:
            do standard base processing

    @foo.register(int)
    def foo_int(x, y):
        do int processing

It's unfortunate that you have to cover it in the base processing, but
certainly no worse than at present.

Paul.

From luis94855510 at gmail.com  Wed Mar  4 14:18:41 2009
From: luis94855510 at gmail.com (Luis Saavedra)
Date: Wed, 04 Mar 2009 10:18:41 -0300
Subject: [Python-Dev] svn (403 Forbidden)
Message-ID: <49AE7FB1.3010208@gmail.com>

Hi,

when I run:

$ mkdir NewDir
$ cd NewDir
$ svn checkout http://svn.python.org/projects/python/trunk python
svn: Server sent unexpected return value (403 Forbidden) in response to 
REPORT request for '/projects/!svn/vcc/default'

how to fix this problem?, on another machine I don't have this problem...,

regards,

Luis.

From benji at benjiyork.com  Wed Mar  4 14:25:01 2009
From: benji at benjiyork.com (Benji York)
Date: Wed, 4 Mar 2009 08:25:01 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <49AE3573.3020005@g.nevcal.com>
References: <loom.20090301T182201-999@post.gmane.org>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<goi2i5$n0n$1@ger.gmane.org> <49AD9C5C.7040009@canterbury.ac.nz>
	<gokb7l$vvm$1@ger.gmane.org> <49ADD077.2040307@canterbury.ac.nz>
	<49AE1E65.4000905@g.nevcal.com>
	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
	<49AE3573.3020005@g.nevcal.com>
Message-ID: <e5fff6640903040525mfdc687lba745e024e909fd6@mail.gmail.com>

On Wed, Mar 4, 2009 at 3:01 AM, Glenn Linderman <v+python at g.nevcal.com> wrote:

> C'mon folks, brainstorm, don't complain about ordereddict if you can't come
> up with some alternatives for discussion!!! ?(and some reasons why the
> suggestions might be good or bad) ?Even your bad ideas might trigger a good
> name in someone else's head...

TemporalDict -- Since the order of insertion is important
SerialDict -- From Websters: of, relating to, consisting of, or
arranged in a series, rank, or row <serial order>
-- 
Benji York

From solipsis at pitrou.net  Wed Mar  4 14:29:36 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 4 Mar 2009 13:29:36 +0000 (UTC)
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>
	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>
	<49AE1E65.4000905@g.nevcal.com>
	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
	<golplk$mld$2@ger.gmane.org> <golslt$1as$1@ger.gmane.org>
	<Pine.LNX.4.64.0903040739590.12396@kimball.webabinitio.net>
Message-ID: <loom.20090304T132840-182@post.gmane.org>

<rdmurray <at> bitdance.com> writes:
> 
> I actually like StableDict best.  When I hear that I think, "ah, the
> key order is stable in the face of insertions, unlike a regular dict".
> Nor can I at the moment think of an alternative explanation of what a
> "StableDict" might be.
> 
> That said, I have no problem with keeping OrderedDict as the name.
> ("Ordered does not mean sorted, it means insertion order preserving"
> may become a FQA (Frequent Question Answer :), but it is short and
> clear and takes no longer than explaining what a StableDict _is_.)

Thanks to Python (and Raymond :-)), I now know what polka dots are.

Python-improves-my-English-skills'ly yours,

Antoine.



From lie.1296 at gmail.com  Wed Mar  4 15:02:46 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Thu, 05 Mar 2009 01:02:46 +1100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <49AE7346.50206@gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<49AE3573.3020005@g.nevcal.com>	<918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>	<golq0g$ooa$1@ger.gmane.org>
	<49AE7346.50206@gmail.com>
Message-ID: <gom1mc$jku$1@ger.gmane.org>

Nick Coghlan wrote:
> Lie Ryan wrote:
>  How about making odict ordered by insertion order, then provide an
>> optional argument for defining sorter? This optional argument must be a
>> function/lambda/callable object and must be the first argument.

or better yet, in the spirit of dumping cmp comparison like in list, the 
first optional argument would be a function that returns the sorting key 
of the object. If the optional argument is not specified, the current 
ordereddict semantic (by insertion order) will be used.

> As the PEP mentions (and Hrvoje brought up again already in this
> thread), a hash table (i.e. dict) probably isn't the right data
> structure to use as the basis for an "always sorted" container.
> In-memory databases, balanced trees, etc, etc.

Isn't ordered dictionary essentially also an "always sorted" container? 
It is always sorted depending on the order of insertion? I can't see any 
technical reason why the data structure can't accommodate them both. Can 
you point me to a discussion on this?


From chris at simplistix.co.uk  Wed Mar  4 15:56:24 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 04 Mar 2009 14:56:24 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49ABCFC9.1070508@gmail.com>
References: <49ABCFC9.1070508@gmail.com>
Message-ID: <49AE9698.1020806@simplistix.co.uk>

Vaibhav Mallya wrote:
> We do have HTMLParser, but that doesn't handle malformed pages well, and 
> just isn't as nice as BeautifulSoup.

Interesting, given that BeautifulSoup is built on HTMLParser ;-)

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From leif.walsh at gmail.com  Wed Mar  4 17:18:10 2009
From: leif.walsh at gmail.com (Leif Walsh)
Date: Wed, 4 Mar 2009 11:18:10 -0500
Subject: [Python-Dev] svn (403 Forbidden)
In-Reply-To: <49AE7FB1.3010208@gmail.com>
References: <49AE7FB1.3010208@gmail.com>
Message-ID: <cc7430500903040818j46ad2fdeqf179e460928f715c@mail.gmail.com>

On Wed, Mar 4, 2009 at 8:18 AM, Luis Saavedra <luis94855510 at gmail.com> wrote:
> how to fix this problem?, on another machine I don't have this problem...,

Which machines does it work/fail on?

I probably can't help you, but maybe that will help someone else figure it out.

-- 
Cheers,
Leif

From leif.walsh at gmail.com  Wed Mar  4 17:21:59 2009
From: leif.walsh at gmail.com (Leif Walsh)
Date: Wed, 4 Mar 2009 11:21:59 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <Pine.LNX.4.64.0903040739590.12396@kimball.webabinitio.net>
References: <loom.20090301T182201-999@post.gmane.org>
	<goi2i5$n0n$1@ger.gmane.org> <49AD9C5C.7040009@canterbury.ac.nz>
	<gokb7l$vvm$1@ger.gmane.org> <49ADD077.2040307@canterbury.ac.nz>
	<49AE1E65.4000905@g.nevcal.com>
	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
	<golplk$mld$2@ger.gmane.org> <golslt$1as$1@ger.gmane.org>
	<Pine.LNX.4.64.0903040739590.12396@kimball.webabinitio.net>
Message-ID: <cc7430500903040821t509b23a7uef27ae4febcf7d8b@mail.gmail.com>

On Wed, Mar 4, 2009 at 7:53 AM,  <rdmurray at bitdance.com> wrote:
> I actually like StableDict best. ?When I hear that I think, "ah, the
> key order is stable in the face of insertions, unlike a regular dict".
> Nor can I at the moment think of an alternative explanation of what a
> "StableDict" might be.

+1

-- 
Cheers,
Leif

From leif.walsh at gmail.com  Wed Mar  4 17:24:32 2009
From: leif.walsh at gmail.com (Leif Walsh)
Date: Wed, 4 Mar 2009 11:24:32 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <Pine.LNX.4.64.0903040739590.12396@kimball.webabinitio.net>
References: <loom.20090301T182201-999@post.gmane.org>
	<goi2i5$n0n$1@ger.gmane.org> <49AD9C5C.7040009@canterbury.ac.nz>
	<gokb7l$vvm$1@ger.gmane.org> <49ADD077.2040307@canterbury.ac.nz>
	<49AE1E65.4000905@g.nevcal.com>
	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
	<golplk$mld$2@ger.gmane.org> <golslt$1as$1@ger.gmane.org>
	<Pine.LNX.4.64.0903040739590.12396@kimball.webabinitio.net>
Message-ID: <cc7430500903040824w4e43d111u3a54b59b7c2b8cf8@mail.gmail.com>

On Wed, Mar 4, 2009 at 7:53 AM,  <rdmurray at bitdance.com> wrote:
> I actually like StableDict best. ?When I hear that I think, "ah, the
> key order is stable in the face of insertions, unlike a regular dict".
> Nor can I at the moment think of an alternative explanation of what a
> "StableDict" might be.

Hmm, perhaps a better explanation than a hasty +1:

"stabledict" makes me think of merge sort, being a stable sort.  In
the same way that merge sort doesn't needlessly swap elements while
sorting, stabledict might be thought to not "needlessly" swap elements
while {inserting, deleting}.  I also can't think of an alternative
explanation, so thus far, it's resistant to false positive semantics.

-- 
Cheers,
Leif

From tjreedy at udel.edu  Wed Mar  4 17:41:47 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 04 Mar 2009 11:41:47 -0500
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <gom1mc$jku$1@ger.gmane.org>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<49AE3573.3020005@g.nevcal.com>	<918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>	<golq0g$ooa$1@ger.gmane.org>	<49AE7346.50206@gmail.com>
	<gom1mc$jku$1@ger.gmane.org>
Message-ID: <gomb08$rgk$1@ger.gmane.org>

Lie Ryan wrote:

> Isn't ordered dictionary essentially also an "always sorted" container? 
> It is always sorted depending on the order of insertion? I can't see any 
> technical reason why the data structure can't accommodate them both. Can 
> you point me to a discussion on this?

Appending an item at the end of a sequence is O(1), no search required. 
  Inserting an item at a random 'sorted' point requires at best an 
O(logN) search.  Insertion itself is O(1) to O(N) depending on the 
structure.


From steve at pearwood.info  Wed Mar  4 18:21:50 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 05 Mar 2009 04:21:50 +1100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
 to	collections ready for pronouncement
In-Reply-To: <cc7430500903040824w4e43d111u3a54b59b7c2b8cf8@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>	<goi2i5$n0n$1@ger.gmane.org>
	<49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org>
	<49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<golplk$mld$2@ger.gmane.org>
	<golslt$1as$1@ger.gmane.org>	<Pine.LNX.4.64.0903040739590.12396@kimball.webabinitio.net>
	<cc7430500903040824w4e43d111u3a54b59b7c2b8cf8@mail.gmail.com>
Message-ID: <49AEB8AE.5060102@pearwood.info>

Leif Walsh wrote:
> On Wed, Mar 4, 2009 at 7:53 AM,  <rdmurray at bitdance.com> wrote:
>> I actually like StableDict best.  When I hear that I think, "ah, the
>> key order is stable in the face of insertions, unlike a regular dict".
>> Nor can I at the moment think of an alternative explanation of what a
>> "StableDict" might be.
> 
> Hmm, perhaps a better explanation than a hasty +1:
> 
> "stabledict" makes me think of merge sort, being a stable sort.

Why merge sort in particular? Why not bubble sort, heap sort, insertion 
sort or any one of many other stable sorts?

Is this analogy really simpler than merely learning the fact that the 
dict keys are kept in the order they are inserted? It's not a very 
difficult concept. Why are we complicating it by inventing obscure, 
complicated analogies with utterly unrelated functions?



 > In
> the same way that merge sort doesn't needlessly swap elements while
> sorting, stabledict might be thought to not "needlessly" swap elements
> while {inserting, deleting}.

You're drawing an awfully long bow here. One might equally argue that in 
the same way that bubble sort does lots and lots of swapping, stabledict 
might be thought to be horribly inefficient and slow.


> I also can't think of an alternative
> explanation, so thus far, it's resistant to false positive semantics.

"The keys don't expire with time."
"It's stable against accidental deletions."
"It's stable against accidentally over-writing values."



-- 
Steven

From Dennis.Benzinger at gmx.net  Wed Mar  4 18:54:58 2009
From: Dennis.Benzinger at gmx.net (Dennis Benzinger)
Date: Wed, 04 Mar 2009 18:54:58 +0100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
 to	collections ready for pronouncement
In-Reply-To: <e5fff6640903040525mfdc687lba745e024e909fd6@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org>
	<49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org>
	<49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<49AE3573.3020005@g.nevcal.com>
	<e5fff6640903040525mfdc687lba745e024e909fd6@mail.gmail.com>
Message-ID: <49AEC072.9020501@gmx.net>

Am 04.03.2009 14:25, Benji York schrieb:
> On Wed, Mar 4, 2009 at 3:01 AM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> 
>> C'mon folks, brainstorm, don't complain about ordereddict if you can't come
>> up with some alternatives for discussion!!!  (and some reasons why the
>> suggestions might be good or bad)  Even your bad ideas might trigger a good
>> name in someone else's head...
> 
> TemporalDict -- Since the order of insertion is important
> SerialDict -- From Websters: of, relating to, consisting of, or
> arranged in a series, rank, or row <serial order>

Because the class is designed to only support insertion order and not
any other sorting why choose a generic name that could also describe a
class that supports a different order?

I'd prefer encoding the order in the class name, therefore I suggest
(Ins|Insertion)(Order|Ordered)Dict, e.g. InsOrderDict. Abbreviating the
first group to simply I probably is too confusing because of the use of
I as a prefix for interfaces.


Dennis Benzinger

From foom at fuhm.net  Wed Mar  4 18:32:48 2009
From: foom at fuhm.net (James Y Knight)
Date: Wed, 4 Mar 2009 12:32:48 -0500
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49AE9698.1020806@simplistix.co.uk>
References: <49ABCFC9.1070508@gmail.com> <49AE9698.1020806@simplistix.co.uk>
Message-ID: <464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>


On Mar 4, 2009, at 9:56 AM, Chris Withers wrote:

> Vaibhav Mallya wrote:
>> We do have HTMLParser, but that doesn't handle malformed pages  
>> well, and just isn't as nice as BeautifulSoup.
>
> Interesting, given that BeautifulSoup is built on HTMLParser ;-)

I think html5lib would be a better candidate for an imrpoved HTML  
parser in the stdlib than BeautifulSoup.

James

From tonynelson at georgeanelson.com  Wed Mar  4 18:13:04 2009
From: tonynelson at georgeanelson.com (Tony Nelson)
Date: Wed, 4 Mar 2009 12:13:04 -0500
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49AE9698.1020806@simplistix.co.uk>
References: <49ABCFC9.1070508@gmail.com>
 <49AE9698.1020806@simplistix.co.uk>
Message-ID: <p04330101c5d46337e985@[192.168.123.162]>

At 2:56 PM +0000 3/4/09, Chris Withers wrote:
>Vaibhav Mallya wrote:
>> We do have HTMLParser, but that doesn't handle malformed pages well, and
>> just isn't as nice as BeautifulSoup.
>
>Interesting, given that BeautifulSoup is built on HTMLParser ;-)

In BeautifulSoup >= 3.1, yes.  Before that (<= 3.07a), it was based on the
more robust sgmllib.SGMLParser.  The current BeautifulSoup can't handle
'<foo a="bc"b="cd">', while the earlier SGMLParser versions can.  I don't
know quite how common that missing space is in the wild, but I've
personally made HTML with that problem.  Maybe this is the only problem
with using HTMLParser instead of SGMLParser; I don't know.  In the mean
time, if I have a need for BeautifulSoup in Python3.x, I'll port sgmllib
and use the older BeautifulSoup.
-- 
____________________________________________________________________
TonyN.:'                       <mailto:tonynelson at georgeanelson.com>
      '                              <http://www.georgeanelson.com/>

From guido at python.org  Wed Mar  4 19:04:21 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Mar 2009 10:04:21 -0800
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <20090304053107.12555.475067956.divmod.xquotient.5237@weber.divmod.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>
	<e6511dbf0810081426q4c576bbak8ab9a7e1d6936254@mail.gmail.com>
	<29052.1223516380@parc.com>
	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>
	<48EE2C77.1050809@palladion.com> <49AD91B4.4070005@palladion.com>
	<ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
	<20090303211757.12555.1049299398.divmod.xquotient.5145@weber.divmod.com>
	<ca471dc20903031420r62af26a1o93b6256c7d864acc@mail.gmail.com>
	<20090304053107.12555.475067956.divmod.xquotient.5237@weber.divmod.com>
Message-ID: <ca471dc20903041004u280872eal3b1ccceba6e235e4@mail.gmail.com>

On Tue, Mar 3, 2009 at 9:31 PM,  <glyph at divmod.com> wrote:
>
> On 3 Mar, 10:20 pm, guido at python.org wrote:
>>
>> On Tue, Mar 3, 2009 at 1:17 PM, ?<glyph at divmod.com> wrote:
>
>>> At the very least, this might serve as a basis for an abstract API for
>>> asyncore:
>>>
>>>
>>> http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html
>
>> I hope we have learned from asyncore that a future-proof and maximally
>> flexible design of such an API is not easy. This is one of those cases
>> where I believe thinking hard up front will really pay off. In this
>> case I would recommend going through a full-blown PEP process before
>> committing to an API, to avoid making the same mistake twice. (I don't
>> mean this as a dig at Twisted, just as an observation about this
>> particular design problem. If Twisted solves all the problems already,
>> writing the PEP should be easy.)
>
> I didn't see it as a dig at Twisted; you're right, this is a hard problem.
> ?Twisted doesn't solve all the problems perfectly, either; much has been
> made of some rather unfortunate nuances of the IProducer and IConsumer
> interfaces.
>
> I also regret that no Twisted people (myself included) seem to have the time
> to go through the PEP process and get something accepted.

Same here.

> Even if Twisted's interfaces aren't perfect, there's a lot of working
> application code which demonstrates that they are sufficient. ?Perhaps more
> importantly, they are future-proof. ?Every proposal we've come up with for
> addressing the more subtle shortcomings of these APIs has been completely
> backwards-compatible; there are no APIs with vague or broken semantics, just
> sub-optimal spellings. ?The fact that there are already multiple reactor
> implementations which share very little code helps.

Sounds like it's not so much the code that's future proof but the
process used for evolving it. That seems to be missing for asyncore.
:-(

> If someone who has PEP-writing skills and some free time is updating
> asyncore to be more modern, I'd be very happy to walk that person through
> Twisted's API design and explain how either portions of our implementation
> could be lifted for the stdlib, or how a "high level" layer could be written
> into asyncore so that the author of a particular chunk of low-level
> networking code could ignore whether they're using the stdlib mainloop or
> one of the Twisted ones.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Mar  4 19:13:47 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Mar 2009 10:13:47 -0800
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
	currently conform
In-Reply-To: <49AE64AE.4090008@gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>
	<gohse2$797$1@ger.gmane.org>
	<D1F247C9-1D96-495C-841E-1DC6F8BBC2EB@solarsail.hcs.harvard.edu>
	<Pine.LNX.4.64.0903030753060.12396@kimball.webabinitio.net>
	<bbaeab100903031715v1882585ah231881c3711ad688@mail.gmail.com>
	<ca471dc20903031725h5a9ea4bfhc7167f6d2d9401dd@mail.gmail.com>
	<49ADD9C1.4020308@trueblade.com>
	<bbaeab100903031737k751ab6dax67273f6ab874ecb9@mail.gmail.com>
	<1afaf6160903032009i440e36fdl9a97be3a78219ec6@mail.gmail.com>
	<49AE64AE.4090008@gmail.com>
Message-ID: <ca471dc20903041013p6a56726dhebeb6cc68ad85a09@mail.gmail.com>

On Wed, Mar 4, 2009 at 3:23 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Benjamin Peterson wrote:
>> Yes, I'm already looking forward to Py4k now. :)
>
> Shh, Guido will need at least 5 years before he's ready to contemplate
> going through something like this again.
>
> Or maybe a decade to be on the safe side ;)

Actually Py4k will have to be on the next BDFL's watch. :)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From sidnei.da.silva at gmail.com  Wed Mar  4 19:14:05 2009
From: sidnei.da.silva at gmail.com (Sidnei da Silva)
Date: Wed, 4 Mar 2009 15:14:05 -0300
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <ca471dc20903041004u280872eal3b1ccceba6e235e4@mail.gmail.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>
	<29052.1223516380@parc.com>
	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>
	<48EE2C77.1050809@palladion.com> <49AD91B4.4070005@palladion.com>
	<ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
	<20090303211757.12555.1049299398.divmod.xquotient.5145@weber.divmod.com>
	<ca471dc20903031420r62af26a1o93b6256c7d864acc@mail.gmail.com>
	<20090304053107.12555.475067956.divmod.xquotient.5237@weber.divmod.com>
	<ca471dc20903041004u280872eal3b1ccceba6e235e4@mail.gmail.com>
Message-ID: <a7a2b76b0903041014q13c6b489lcfe2c297e19ce8a3@mail.gmail.com>

On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum <guido at python.org> wrote:
> Sounds like it's not so much the code that's future proof but the
> process used for evolving it. That seems to be missing for asyncore.
> :-(

Turning the issue around a bit, has anyone considered polishing up the
current fix to restore it's backwards compatibility, instead of
starting a discussion about a full-blown replacement?

I think that would be enough for most asyncore users (or even the
couple few affected) for the moment, and then we can think about a
possible future replacement.

-- 
Sidnei da Silva
Canonical Ltd.
?Landscape ? Changing the way you manage your systems
http://landscape.canonical.com

From g.brandl at gmx.net  Wed Mar  4 19:10:03 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 04 Mar 2009 19:10:03 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
References: <49ABCFC9.1070508@gmail.com> <49AE9698.1020806@simplistix.co.uk>
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
Message-ID: <gomg5r$fhi$2@ger.gmane.org>

James Y Knight schrieb:
> On Mar 4, 2009, at 9:56 AM, Chris Withers wrote:
> 
>> Vaibhav Mallya wrote:
>>> We do have HTMLParser, but that doesn't handle malformed pages  
>>> well, and just isn't as nice as BeautifulSoup.
>>
>> Interesting, given that BeautifulSoup is built on HTMLParser ;-)
> 
> I think html5lib would be a better candidate for an imrpoved HTML  
> parser in the stdlib than BeautifulSoup.

I second that.

Georg


From guido at python.org  Wed Mar  4 19:21:26 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Mar 2009 10:21:26 -0800
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <a7a2b76b0903041014q13c6b489lcfe2c297e19ce8a3@mail.gmail.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>
	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>
	<48EE2C77.1050809@palladion.com> <49AD91B4.4070005@palladion.com>
	<ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
	<20090303211757.12555.1049299398.divmod.xquotient.5145@weber.divmod.com>
	<ca471dc20903031420r62af26a1o93b6256c7d864acc@mail.gmail.com>
	<20090304053107.12555.475067956.divmod.xquotient.5237@weber.divmod.com>
	<ca471dc20903041004u280872eal3b1ccceba6e235e4@mail.gmail.com>
	<a7a2b76b0903041014q13c6b489lcfe2c297e19ce8a3@mail.gmail.com>
Message-ID: <ca471dc20903041021s67660577qb5b5be415c117bad@mail.gmail.com>

On Wed, Mar 4, 2009 at 10:14 AM, Sidnei da Silva
<sidnei.da.silva at gmail.com> wrote:
> On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum <guido at python.org> wrote:
>> Sounds like it's not so much the code that's future proof but the
>> process used for evolving it. That seems to be missing for asyncore.
>> :-(
>
> Turning the issue around a bit, has anyone considered polishing up the
> current fix to restore it's backwards compatibility, instead of
> starting a discussion about a full-blown replacement?
>
> I think that would be enough for most asyncore users (or even the
> couple few affected) for the moment, and then we can think about a
> possible future replacement.

If it can be done while maintaining backwards compatibility with both
the 2.6 version and the pre-2.6 version, that would be great of
course! But can it?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From exarkun at divmod.com  Wed Mar  4 19:27:38 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Wed, 4 Mar 2009 13:27:38 -0500
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
 of medusa
In-Reply-To: <ca471dc20903041021s67660577qb5b5be415c117bad@mail.gmail.com>
Message-ID: <20090304182738.12853.1774369899.divmod.quotient.18049@henry.divmod.com>

On Wed, 4 Mar 2009 10:21:26 -0800, Guido van Rossum <guido at python.org> wrote:
>On Wed, Mar 4, 2009 at 10:14 AM, Sidnei da Silva
><sidnei.da.silva at gmail.com> wrote:
>> On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum <guido at python.org> wrote:
>>> Sounds like it's not so much the code that's future proof but the
>>> process used for evolving it. That seems to be missing for asyncore.
>>> :-(
>>
>> Turning the issue around a bit, has anyone considered polishing up the
>> current fix to restore it's backwards compatibility, instead of
>> starting a discussion about a full-blown replacement?
>>
>> I think that would be enough for most asyncore users (or even the
>> couple few affected) for the moment, and then we can think about a
>> possible future replacement.
>
>If it can be done while maintaining backwards compatibility with both
>the 2.6 version and the pre-2.6 version, that would be great of
>course! But can it?
>

Is it really necessary to retain compatibility with the Python 2.6
version?  Python 2.6.0 and Python 2.6.1 contain a regression (as
compared to basically all previous versions of Python) which prevents
asyncore-based programs which are years old from working on them.
Restoring the pre-2.6 behavior will fix these old, presumably stable,
widely used programs for users who install 2.6.2 and newer.

The downside (which you were imagining, I'm sure) is that any new
software developed against the Python 2.6.0 or 2.6.1 behavior will
then break in 2.6.2 and later.  While this is unfortunate, it is
clearly the far lesser of two evils.  The choice must be made, though.
Either leave old software broken or break new software.  Just because
the "leave old software broken" choice is made through inaction doesn't
make it the better choice (though obviously since it requires action,
someone will have to do it, and I'm not volunteering - if inaction is
the choice because no one wants to do the work, fine, but that's a
different motivation than avoiding breaking newly written software).

So, as a disinterested party in this specific case, I'd say revert to
the pre-2.6 behavior.  It does less harm than leaving the current
behavior.

Jean-Paul

From guido at python.org  Wed Mar  4 19:46:28 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Mar 2009 10:46:28 -0800
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <20090304182738.12853.1774369899.divmod.quotient.18049@henry.divmod.com>
References: <ca471dc20903041021s67660577qb5b5be415c117bad@mail.gmail.com>
	<20090304182738.12853.1774369899.divmod.quotient.18049@henry.divmod.com>
Message-ID: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>

On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> - Show quoted text -
> On Wed, 4 Mar 2009 10:21:26 -0800, Guido van Rossum <guido at python.org>
> wrote:
>>
>> On Wed, Mar 4, 2009 at 10:14 AM, Sidnei da Silva
>> <sidnei.da.silva at gmail.com> wrote:
>>>
>>> On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum <guido at python.org>
>>> wrote:
>>>>
>>>> Sounds like it's not so much the code that's future proof but the
>>>> process used for evolving it. That seems to be missing for asyncore.
>>>> :-(
>>>
>>> Turning the issue around a bit, has anyone considered polishing up the
>>> current fix to restore it's backwards compatibility, instead of
>>> starting a discussion about a full-blown replacement?
>>>
>>> I think that would be enough for most asyncore users (or even the
>>> couple few affected) for the moment, and then we can think about a
>>> possible future replacement.
>>
>> If it can be done while maintaining backwards compatibility with both
>> the 2.6 version and the pre-2.6 version, that would be great of
>> course! But can it?
>>
>
> Is it really necessary to retain compatibility with the Python 2.6
> version? ?Python 2.6.0 and Python 2.6.1 contain a regression (as
> compared to basically all previous versions of Python) which prevents
> asyncore-based programs which are years old from working on them.
> Restoring the pre-2.6 behavior will fix these old, presumably stable,
> widely used programs for users who install 2.6.2 and newer.
>
> The downside (which you were imagining, I'm sure) is that any new
> software developed against the Python 2.6.0 or 2.6.1 behavior will
> then break in 2.6.2 and later. ?While this is unfortunate, it is
> clearly the far lesser of two evils. ?The choice must be made, though.
> Either leave old software broken or break new software. ?Just because
> the "leave old software broken" choice is made through inaction doesn't
> make it the better choice (though obviously since it requires action,
> someone will have to do it, and I'm not volunteering - if inaction is
> the choice because no one wants to do the work, fine, but that's a
> different motivation than avoiding breaking newly written software).
>
> So, as a disinterested party in this specific case, I'd say revert to
> the pre-2.6 behavior. ?It does less harm than leaving the current
> behavior.

Sorry, but I really do think that we should maintain backward
compatibility *within* the 2.6 series as well. If that makes it
impossible to also maintain the 2.5 behavior, perhaps some flag could
be added to restore 2.5 compatibility, e.g.

import asyncore
asyncore.python_25_compat = True

Note that this "API" is designed to work in 2.5 as well. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From exarkun at divmod.com  Wed Mar  4 19:51:06 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Wed, 4 Mar 2009 13:51:06 -0500
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
 of medusa
In-Reply-To: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
Message-ID: <20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>

On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum <guido at python.org> wrote:
>On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> [snip]
>>
>> So, as a disinterested party in this specific case, I'd say revert to
>> the pre-2.6 behavior. ?It does less harm than leaving the current
>> behavior.
>
>Sorry, but I really do think that we should maintain backward
>compatibility *within* the 2.6 series as well. If that makes it
>impossible to also maintain the 2.5 behavior, perhaps some flag could
>be added to restore 2.5 compatibility, e.g.
>
>import asyncore
>asyncore.python_25_compat = True
>
>Note that this "API" is designed to work in 2.5 as well. :-)
>

But why?  The argument I made had the objective of minimizing developer
effort.  What's the objective of maintaining backward compatibility within
the 2.6 series in this case (sorry if it appeared earlier in this thread
and I missed it)?

Jean-Paul

From guido at python.org  Wed Mar  4 19:54:19 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Mar 2009 10:54:19 -0800
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
Message-ID: <ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>

On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum <guido at python.org>
> wrote:
>>
>> On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone <exarkun at divmod.com>
>> wrote:
>> [snip]
>>>
>>> So, as a disinterested party in this specific case, I'd say revert to
>>> the pre-2.6 behavior. ?It does less harm than leaving the current
>>> behavior.
>>
>> Sorry, but I really do think that we should maintain backward
>> compatibility *within* the 2.6 series as well. If that makes it
>> impossible to also maintain the 2.5 behavior, perhaps some flag could
>> be added to restore 2.5 compatibility, e.g.
>>
>> import asyncore
>> asyncore.python_25_compat = True
>>
>> Note that this "API" is designed to work in 2.5 as well. :-)

> But why? ?The argument I made had the objective of minimizing developer
> effort. ?What's the objective of maintaining backward compatibility within
> the 2.6 series in this case (sorry if it appeared earlier in this thread
> and I missed it)?

The same as always. We don't change APIs in bugfix releases.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From exarkun at divmod.com  Wed Mar  4 19:56:15 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Wed, 4 Mar 2009 13:56:15 -0500
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
 of medusa
In-Reply-To: <ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
Message-ID: <20090304185615.12853.1400676125.divmod.quotient.18065@henry.divmod.com>

On Wed, 4 Mar 2009 10:54:19 -0800, Guido van Rossum <guido at python.org> wrote:
>On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone <exarkun at divmod.com> wrote:
>> On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum <guido at python.org>
>> wrote:
>>>
>>> On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone <exarkun at divmod.com>
>>> wrote:
>>> [snip]
>>>>
>>>> So, as a disinterested party in this specific case, I'd say revert to
>>>> the pre-2.6 behavior. ?It does less harm than leaving the current
>>>> behavior.
>>>
>>> Sorry, but I really do think that we should maintain backward
>>> compatibility *within* the 2.6 series as well. If that makes it
>>> impossible to also maintain the 2.5 behavior, perhaps some flag could
>>> be added to restore 2.5 compatibility, e.g.
>>>
>>> import asyncore
>>> asyncore.python_25_compat = True
>>>
>>> Note that this "API" is designed to work in 2.5 as well. :-)
>
>> But why? ?The argument I made had the objective of minimizing developer
>> effort. ?What's the objective of maintaining backward compatibility within
>> the 2.6 series in this case (sorry if it appeared earlier in this thread
>> and I missed it)?
>
>The same as always. We don't change APIs in bugfix releases.
>

Okay.  Thanks for explaining.

Jean-Paul

From steve at holdenweb.com  Wed Mar  4 20:14:10 2009
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 04 Mar 2009 14:14:10 -0500
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
Message-ID: <gomju8$ujd$1@ger.gmane.org>

Guido van Rossum wrote:
> On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone <exarkun at divmod.com> wrote:
>> On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum <guido at python.org>
>> wrote:
>>> On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone <exarkun at divmod.com>
>>> wrote:
>>> [snip]
>>>> So, as a disinterested party in this specific case, I'd say revert to
>>>> the pre-2.6 behavior.  It does less harm than leaving the current
>>>> behavior.
>>> Sorry, but I really do think that we should maintain backward
>>> compatibility *within* the 2.6 series as well. If that makes it
>>> impossible to also maintain the 2.5 behavior, perhaps some flag could
>>> be added to restore 2.5 compatibility, e.g.
>>>
>>> import asyncore
>>> asyncore.python_25_compat = True
>>>
>>> Note that this "API" is designed to work in 2.5 as well. :-)
> 
>> But why?  The argument I made had the objective of minimizing developer
>> effort.  What's the objective of maintaining backward compatibility within
>> the 2.6 series in this case (sorry if it appeared earlier in this thread
>> and I missed it)?
> 
> The same as always. We don't change APIs in bugfix releases.
> 
Which unfortunately stops us from categorizing API breakages as bugs.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From glyph at divmod.com  Wed Mar  4 20:44:46 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Wed, 04 Mar 2009 19:44:46 -0000
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of	medusa
In-Reply-To: <20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
Message-ID: <20090304194446.12555.1053607582.divmod.xquotient.5310@weber.divmod.com>


On 06:51 pm, exarkun at divmod.com wrote:
>On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum <guido at python.org> 
>wrote:
>>On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone 
>><exarkun at divmod.com> wrote:

>>>So, as a disinterested party in this specific case, I'd say revert to
>>>the pre-2.6 behavior.

>>Sorry, but I really do think that we should maintain backward
>>compatibility *within* the 2.6 series as well.

>But why?  The argument I made had the objective of minimizing developer
>effort.  What's the objective of maintaining backward compatibility 
>within
>the 2.6 series in this case (sorry if it appeared earlier in this 
>thread
>and I missed it)?

Python's compatibility policy dictates that compatibility between micro- 
version releases be maintained *extremely* carefully.

As you all well know, normally I'd be strongly in favor of adhering to 
such a policy.  However, the ostensible purpose of the policy here is 
that users can *always* install micro-version upgrades to get bugfixes 
and security fixes, without worrying about compatibility upgrades.

Maintaining compatibility with the 2.6.x version of asyncore presupposes 
that *someone* has written some software against that version of 
asyncore and it might break if they installed an upgrade, though.  If 
that's the case - if there's even one person who has written or run any 
asyncore software against the version in 2.6 - then I think maintaining 
bug-for-bug compatibility is important.

However, my guess - and I'm assuming that JP was thinking the same thing 
- is that literally nobody has done that, or even *would* ever do that, 
so there's no software anywhere in the world that would break.  asyncore 
is relatively unpopular (thanks in part to the excellent alternatives 
:-)); its major users are the same people who have already complained.

Maybe it's too late to do something like this for 2.6, or even 3.0, but 
I've thought on a few occasions that projects (both Python and Twisted, 
at least) could use a special very-low-traffic policy-deviations mailing 
list for "We *REALLY* don't think this is going to ever break anything 
anyone has actually done, but just to be sure..." notifications for 
situations like this.  The implications being that (A) _everyone_ who 
uses the software should subscribe, and (B) if anyone ever responded to 
a message, the deviation wouldn't take place.

Anyway, I'm also a disinterested observer, so please don't take this as 
a strong endorsement of doing the fix in 2.6; the set-an-attribute-to- 
fix-it idea is fine by me too.  I thought I'd lay out the reasoning for 
violating an otherwise extremely reasonable policy, though.

From barry at python.org  Wed Mar  4 21:28:57 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 4 Mar 2009 15:28:57 -0500
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of	medusa
In-Reply-To: <20090304194446.12555.1053607582.divmod.xquotient.5310@weber.divmod.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<20090304194446.12555.1053607582.divmod.xquotient.5310@weber.divmod.com>
Message-ID: <92A276C3-3FE3-4060-BCE4-F8865E2E4295@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 4, 2009, at 2:44 PM, glyph at divmod.com wrote:

> Maintaining compatibility with the 2.6.x version of asyncore  
> presupposes that *someone* has written some software against that  
> version of asyncore and it might break if they installed an upgrade,  
> though.  If that's the case - if there's even one person who has  
> written or run any asyncore software against the version in 2.6 -  
> then I think maintaining bug-for-bug compatibility is important.
>
> However, my guess - and I'm assuming that JP was thinking the same  
> thing - is that literally nobody has done that, or even *would* ever  
> do that, so there's no software anywhere in the world that would  
> break.  asyncore is relatively unpopular (thanks in part to the  
> excellent alternatives :-)); its major users are the same people who  
> have already complained.

FWIW, I use smtpd.py and a few of the asyncore APIs  
(.loop(), .socket_map.clear(), and .close_all()) in the Mailman 3 test  
suite.  That only works on Python 2.6 and I don't recall even a hiccup  
when moving from 2.5 to 2.6.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSa7kiXEjvBPtnXfVAQI2BQP/XD71ipSv5OhtdOJt+OF2jbfOyFU/2vPR
iJ33g9JJ2A0CvDWXmc81t9hMH3U8waagkXaDUF4r2MNM3JbONSjQGGZqX6X/1UOB
XCT+jNOrbRw2VQR0qjrweQ5A8u0Y1zsDo/rD4CXe0K1pFFWLubaU3lVgqEBMBL8r
xvf77EMoUuA=
=vmM5
-----END PGP SIGNATURE-----

From janssen at parc.com  Wed Mar  4 21:28:15 2009
From: janssen at parc.com (Bill Janssen)
Date: Wed, 4 Mar 2009 12:28:15 PST
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
Message-ID: <54761.1236198495@parc.com>

Guido van Rossum <guido at python.org> wrote:

> On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> > On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum <guido at python.org>
> > wrote:
> >>
> >> On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone <exarkun at divmod.com>
> >> wrote:
> >> [snip]
> >>>
> >>> So, as a disinterested party in this specific case, I'd say revert to
> >>> the pre-2.6 behavior. ?It does less harm than leaving the current
> >>> behavior.
> >>
> >> Sorry, but I really do think that we should maintain backward
> >> compatibility *within* the 2.6 series as well. If that makes it
> >> impossible to also maintain the 2.5 behavior, perhaps some flag could
> >> be added to restore 2.5 compatibility, e.g.
> >>
> >> import asyncore
> >> asyncore.python_25_compat = True
> >>
> >> Note that this "API" is designed to work in 2.5 as well. :-)
> 
> > But why? ?The argument I made had the objective of minimizing developer
> > effort. ?What's the objective of maintaining backward compatibility within
> > the 2.6 series in this case (sorry if it appeared earlier in this thread
> > and I missed it)?
> 
> The same as always. We don't change APIs in bugfix releases.

OK, seems reasonable.

But in this case, isn't the broken API the bug that's being fixed?  Do
we need a different way to fix broken APIs in bugfix releases?

Bill

From daniel at stutzbachenterprises.com  Wed Mar  4 21:31:38 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Wed, 4 Mar 2009 14:31:38 -0600
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
Message-ID: <eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>

On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum <guido at python.org> wrote:

> The same as always. We don't change APIs in bugfix releases.
>

This question is actually for the Zope folks and others who have had
problems with the 2.6 asyncore/asynchat:

Are any of the problems due to a change in the documented API... or are they
all due to changes in undocumented internals that your code relied on?

Myself, the only change that I noticed going from 2.5 to 2.6 was that the
undocumented signature of asynchat.__init__() changed from taking a
parameter called "conn" to one with the same meaning called "sock".

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090304/c809a931/attachment-0001.htm>

From krstic at solarsail.hcs.harvard.edu  Wed Mar  4 22:14:28 2009
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?Q?Ivan_Krsti=C4=87?=)
Date: Wed, 4 Mar 2009 16:14:28 -0500
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of	medusa
In-Reply-To: <20090304053107.12555.475067956.divmod.xquotient.5237@weber.divmod.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>
	<5003.1223487033@parc.com>
	<e6511dbf0810081426q4c576bbak8ab9a7e1d6936254@mail.gmail.com>
	<29052.1223516380@parc.com>
	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>
	<48EE2C77.1050809@palladion.com> <49AD91B4.4070005@palladion.com>
	<ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
	<20090303211757.12555.1049299398.divmod.xquotient.5145@weber.divmod.com>
	<ca471dc20903031420r62af26a1o93b6256c7d864acc@mail.gmail.com>
	<20090304053107.12555.475067956.divmod.xquotient.5237@weber.divmod.com>
Message-ID: <5A50A6EE-1DF9-413D-9FCC-29EA4007B79F@solarsail.hcs.harvard.edu>

Glyph,

On Mar 4, 2009, at 12:31 AM, glyph at divmod.com wrote:
> If someone who has PEP-writing skills and some free time is updating  
> asyncore to be more modern, I'd be very happy to walk that person  
> through Twisted's API design and explain how either portions of our  
> implementation could be lifted for the stdlib, or how a "high level"  
> layer could be written into asyncore so that the author of a  
> particular chunk of low-level networking code could ignore whether  
> they're using the stdlib mainloop or one of the Twisted ones.

I spent about a half hour sometime in the last month talking this  
through with Itamar, though not in great detail. I'd be interested in  
sitting down with both of you and trying to establish more precisely  
how much work is necessary to get something to actually happen here. I  
won't outright promise a PEP, but I'll promise at the very least to  
write down elaborate notes that someone could turn into a PEP  
relatively straightforwardly. Deal?

Cheers,

--
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | http://radian.org


From krstic at solarsail.hcs.harvard.edu  Wed Mar  4 22:17:52 2009
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?Q?Ivan_Krsti=C4=87?=)
Date: Wed, 4 Mar 2009 16:17:52 -0500
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
References: <49ABCFC9.1070508@gmail.com> <49AE9698.1020806@simplistix.co.uk>
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
Message-ID: <784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>

On Mar 4, 2009, at 12:32 PM, James Y Knight wrote:
> I think html5lib would be a better candidate for an imrpoved HTML  
> parser in the stdlib than BeautifulSoup.


While we're talking about alternatives, Ian Bicking appears to swear  
by lxml:

<http://blog.ianbicking.org/2008/12/10/lxml-an-underappreciated-web-scraping-library/ 
 >

Cheers,

--
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | http://radian.org


From ncoghlan at gmail.com  Wed Mar  4 22:25:20 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 05 Mar 2009 07:25:20 +1000
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <gom1mc$jku$1@ger.gmane.org>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<49AE3573.3020005@g.nevcal.com>	<918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>	<golq0g$ooa$1@ger.gmane.org>	<49AE7346.50206@gmail.com>
	<gom1mc$jku$1@ger.gmane.org>
Message-ID: <49AEF1C0.7000902@gmail.com>

Lie Ryan wrote:
> Isn't ordered dictionary essentially also an "always sorted" container?
> It is always sorted depending on the order of insertion? I can't see any
> technical reason why the data structure can't accommodate them both. Can
> you point me to a discussion on this?

My phrasing was a little ambiguous - "always sorted for an arbitrary key
function" is better handled with something other than a hash map +
additional bookkeeping due to the effect on the speed of insertion and
deletion.

With a specifically insertion-ordered dict, only deletion is really
slowed down by the additional bookkeeping: it drops to O(n) due to the
need to find and remove the key being deleted from the sequence of keys
as well as from the hash map).

As Terry noted, supporting arbitrary sort orders would slow down
insertion as well.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From chrism at plope.com  Wed Mar  4 22:27:35 2009
From: chrism at plope.com (Chris McDonough)
Date: Wed, 04 Mar 2009 16:27:35 -0500
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
 of medusa
In-Reply-To: <eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
Message-ID: <49AEF247.9060800@plope.com>

Daniel Stutzbach wrote:
> On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum <guido at python.org> wrote:
> 
>> The same as always. We don't change APIs in bugfix releases.
>>
> 
> This question is actually for the Zope folks and others who have had
> problems with the 2.6 asyncore/asynchat:
> 
> Are any of the problems due to a change in the documented API... or are they
> all due to changes in undocumented internals that your code relied on?

As far as I can tell, asyncore/asynchat is all "undocumented internals".  Any
use of asyncore in anger will use internals; there never was any well-understood
API to these modules.  Medusa itself (from which asyncore and asynchat were
derived) appears to itself break with the changes to asyncore/asynchat in 2.6
(at least it appears to use attributes like "ac_out_buffer" which were removed
in 2.6; this is not "Zope's version"; there is no such animal; this is plain old
Medusa 0.5.4).

Count me in as one who believes that it would be the lesser of two evils to
revert to the older (2.5 and prior) asyncore/asynchat implementations in 2.6.2
rather than soldiering on with the 2.6 and 2.6.1 implementation (which almost
certainly has fewer, if any, consumers); somebody messed up in 2.6 by making its
asyncore/asynchat more forward compatible with 3.0's than backwards compatible
with 2.5's and prior; this was just a mistake and keeping old code running
should trump any theoretical or puritanical benefit in a dot release.

- C


From guido at python.org  Wed Mar  4 22:31:27 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Mar 2009 13:31:27 -0800
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <49AEF247.9060800@plope.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com>
Message-ID: <ca471dc20903041331l7922d796x5028ebf85692e8ea@mail.gmail.com>

On Wed, Mar 4, 2009 at 1:27 PM, Chris McDonough <chrism at plope.com> wrote:
> Daniel Stutzbach wrote:
>> On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum <guido at python.org> wrote:
>>
>>> The same as always. We don't change APIs in bugfix releases.
>>>
>>
>> This question is actually for the Zope folks and others who have had
>> problems with the 2.6 asyncore/asynchat:
>>
>> Are any of the problems due to a change in the documented API... or are they
>> all due to changes in undocumented internals that your code relied on?
>
> As far as I can tell, asyncore/asynchat is all "undocumented internals". ?Any
> use of asyncore in anger will use internals; there never was any well-understood
> API to these modules. ?Medusa itself (from which asyncore and asynchat were
> derived) appears to itself break with the changes to asyncore/asynchat in 2.6
> (at least it appears to use attributes like "ac_out_buffer" which were removed
> in 2.6; this is not "Zope's version"; there is no such animal; this is plain old
> Medusa 0.5.4).
>
> Count me in as one who believes that it would be the lesser of two evils to
> revert to the older (2.5 and prior) asyncore/asynchat implementations in 2.6.2
> rather than soldiering on with the 2.6 and 2.6.1 implementation (which almost
> certainly has fewer, if any, consumers); somebody messed up in 2.6 by making its
> asyncore/asynchat more forward compatible with 3.0's than backwards compatible
> with 2.5's and prior; this was just a mistake and keeping old code running
> should trump any theoretical or puritanical benefit in a dot release.

Then I'd like to hear from the folks who made and reviewed those
changes to asyncore in 2.6. I can't imagine the changes were done
without good intentions.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Wed Mar  4 22:34:17 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 05 Mar 2009 07:34:17 +1000
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
 to	collections ready for pronouncement
In-Reply-To: <49AEC072.9020501@gmx.net>
References: <loom.20090301T182201-999@post.gmane.org>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org>	<49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org>	<49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<49AE3573.3020005@g.nevcal.com>	<e5fff6640903040525mfdc687lba745e024e909fd6@mail.gmail.com>
	<49AEC072.9020501@gmx.net>
Message-ID: <49AEF3D9.1060303@gmail.com>

Dennis Benzinger wrote:
> I'd prefer encoding the order in the class name, therefore I suggest
> (Ins|Insertion)(Order|Ordered)Dict, e.g. InsOrderDict. Abbreviating the
> first group to simply I probably is too confusing because of the use of
> I as a prefix for interfaces.

Except I just don't see this proliferation of dict types with different
sort orders ever happening.

The distinction between OrderedDict and dict is that the order of
keys()/values()/items() isn't arbitrary the way it is in a regular dict
- there's a defined order that will always be used. That's all the name
tells you - if someone assumes they know what that ordering is without
actually looking at the documentation (and gets it wrong as a result),
then I don't see how that is any different from the fact that someone
might mistakenly assume that list.sort() puts the items in descending
order instead of ascending order.

For other sort orders, it seems far more likely to me that a
collections.SortedMap type would be added at some point in the future
that accepts a key function like the one accepted by sorted() and
list.sort(). Such a data type would make different trade-offs between
insertion, deletion and lookup speeds than those made in the hash map
based OrderedDict.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From daniel at stutzbachenterprises.com  Wed Mar  4 22:36:49 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Wed, 4 Mar 2009 15:36:49 -0600
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <5A50A6EE-1DF9-413D-9FCC-29EA4007B79F@solarsail.hcs.harvard.edu>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>
	<29052.1223516380@parc.com>
	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>
	<48EE2C77.1050809@palladion.com> <49AD91B4.4070005@palladion.com>
	<ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
	<20090303211757.12555.1049299398.divmod.xquotient.5145@weber.divmod.com>
	<ca471dc20903031420r62af26a1o93b6256c7d864acc@mail.gmail.com>
	<20090304053107.12555.475067956.divmod.xquotient.5237@weber.divmod.com>
	<5A50A6EE-1DF9-413D-9FCC-29EA4007B79F@solarsail.hcs.harvard.edu>
Message-ID: <eae285400903041336h3be07800sdaa8afe276014c5@mail.gmail.com>

On Wed, Mar 4, 2009 at 3:14 PM, Ivan Krsti? <
krstic at solarsail.hcs.harvard.edu> wrote:

> I spent about a half hour sometime in the last month talking this through
> with Itamar, though not in great detail. I'd be interested in sitting down
> with both of you and trying to establish more precisely how much work is
> necessary to get something to actually happen here. I won't outright promise
> a PEP, but I'll promise at the very least to write down elaborate notes that
> someone could turn into a PEP relatively straightforwardly. Deal?
>

Will any or all of you be at PyCon?  I'd be willing to put in the extra work
to turn your notes into a PEP.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090304/67e04ffc/attachment.htm>

From greg.ewing at canterbury.ac.nz  Wed Mar  4 22:44:59 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 05 Mar 2009 10:44:59 +1300
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <Pine.LNX.4.64.0903040739590.12396@kimball.webabinitio.net>
References: <loom.20090301T182201-999@post.gmane.org>
	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>
	<gohj2l$63r$1@ger.gmane.org>
	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>
	<49AC5A9C.3030507@gmail.com>
	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>
	<goi2i5$n0n$1@ger.gmane.org> <49AD9C5C.7040009@canterbury.ac.nz>
	<gokb7l$vvm$1@ger.gmane.org> <49ADD077.2040307@canterbury.ac.nz>
	<49AE1E65.4000905@g.nevcal.com>
	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
	<golplk$mld$2@ger.gmane.org> <golslt$1as$1@ger.gmane.org>
	<Pine.LNX.4.64.0903040739590.12396@kimball.webabinitio.net>
Message-ID: <49AEF65B.1010709@canterbury.ac.nz>

rdmurray at bitdance.com wrote:

> I actually like StableDict best.  When I hear that I think, "ah, the
> key order is stable in the face of insertions, unlike a regular dict".

But it still doesn't convey what the ordering actually *is*.

-- 
Greg

From guido at python.org  Wed Mar  4 22:52:59 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Mar 2009 13:52:59 -0800
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
	collections ready for pronouncement
In-Reply-To: <49AEF65B.1010709@canterbury.ac.nz>
References: <loom.20090301T182201-999@post.gmane.org>
	<49AD9C5C.7040009@canterbury.ac.nz> <gokb7l$vvm$1@ger.gmane.org>
	<49ADD077.2040307@canterbury.ac.nz> <49AE1E65.4000905@g.nevcal.com>
	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>
	<golplk$mld$2@ger.gmane.org> <golslt$1as$1@ger.gmane.org>
	<Pine.LNX.4.64.0903040739590.12396@kimball.webabinitio.net>
	<49AEF65B.1010709@canterbury.ac.nz>
Message-ID: <ca471dc20903041352i742a43abi467e029cfa3092fd@mail.gmail.com>

On Wed, Mar 4, 2009 at 1:44 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> rdmurray at bitdance.com wrote:
>
>> I actually like StableDict best. ?When I hear that I think, "ah, the
>> key order is stable in the face of insertions, unlike a regular dict".
>
> But it still doesn't convey what the ordering actually *is*.

Please, stick with OrderedDict. That's the name used historically by
most people who independently reinvented this functionality.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From p.f.moore at gmail.com  Thu Mar  5 00:24:30 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 4 Mar 2009 23:24:30 +0000
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
	currently conform
In-Reply-To: <ca471dc20903041013p6a56726dhebeb6cc68ad85a09@mail.gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>
	<D1F247C9-1D96-495C-841E-1DC6F8BBC2EB@solarsail.hcs.harvard.edu>
	<Pine.LNX.4.64.0903030753060.12396@kimball.webabinitio.net>
	<bbaeab100903031715v1882585ah231881c3711ad688@mail.gmail.com>
	<ca471dc20903031725h5a9ea4bfhc7167f6d2d9401dd@mail.gmail.com>
	<49ADD9C1.4020308@trueblade.com>
	<bbaeab100903031737k751ab6dax67273f6ab874ecb9@mail.gmail.com>
	<1afaf6160903032009i440e36fdl9a97be3a78219ec6@mail.gmail.com>
	<49AE64AE.4090008@gmail.com>
	<ca471dc20903041013p6a56726dhebeb6cc68ad85a09@mail.gmail.com>
Message-ID: <79990c6b0903041524qa456b4el12d9c70fd640a1d@mail.gmail.com>

2009/3/4 Guido van Rossum <guido at python.org>:
> On Wed, Mar 4, 2009 at 3:23 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Benjamin Peterson wrote:
>>> Yes, I'm already looking forward to Py4k now. :)
>>
>> Shh, Guido will need at least 5 years before he's ready to contemplate
>> going through something like this again.
>>
>> Or maybe a decade to be on the safe side ;)
>
> Actually Py4k will have to be on the next BDFL's watch. :)

Somebody warn Orlijn now!!! :-)

Paul.

From python at rcn.com  Thu Mar  5 02:01:48 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 4 Mar 2009 17:01:48 -0800
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
Message-ID: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>

Just notices that the empty() and full() methods were still there.
IIRC, we agreed to take them out (but leaving qsize() exposed).
The docs entries and test cases were taken out, but the actual
methods were accidentally left in.

Raymond

From benjamin at python.org  Thu Mar  5 02:04:45 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 4 Mar 2009 19:04:45 -0600
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
In-Reply-To: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>
Message-ID: <1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>

2009/3/4 Raymond Hettinger <python at rcn.com>:
> Just notices that the empty() and full() methods were still there.
> IIRC, we agreed to take them out (but leaving qsize() exposed).
> The docs entries and test cases were taken out, but the actual
> methods were accidentally left in.

If so, the only thing to do is deprecate it in 3.1 for removal in 3.2.



-- 
Regards,
Benjamin

From tleeuwenburg at gmail.com  Thu Mar  5 02:25:51 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Thu, 5 Mar 2009 12:25:51 +1100
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
In-Reply-To: <1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>
	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
Message-ID: <43c8685c0903041725m792e108dwc5839035d64ab7f4@mail.gmail.com>

That code doesn't look toooo scary... is trying to add a DeprecationWarning
something that I could try to learn how to do and try my hand at? Maybe if
someone else would like to address that more quickly, they'd be able to keep
me in the loop so I can start learning how Things are Done?

It looks as though all that is required is to add a warn message indicating
the planned removal?

I'm not sure if it's better for me to have a go at things quietly or to be
more interactive about getting to grips with the code...

Cheers,
-T

On Thu, Mar 5, 2009 at 12:04 PM, Benjamin Peterson <benjamin at python.org>wrote:

> 2009/3/4 Raymond Hettinger <python at rcn.com>:
> > Just notices that the empty() and full() methods were still there.
> > IIRC, we agreed to take them out (but leaving qsize() exposed).
> > The docs entries and test cases were taken out, but the actual
> > methods were accidentally left in.
>
> If so, the only thing to do is deprecate it in 3.1 for removal in 3.2.
>
>
>
> --
> Regards,
> Benjamin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/tleeuwenburg%40gmail.com
>



-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090305/4fbdc223/attachment-0001.htm>

From python at rcn.com  Thu Mar  5 02:30:50 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 4 Mar 2009 17:30:50 -0800
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>
	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
Message-ID: <9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>

>> Just noticed that the empty() and full() methods were still there.
>> IIRC, we agreed to take them out (but leaving qsize() exposed).
>> The docs entries and test cases were taken out, but the actual
>> methods were accidentally left in.
> 
> If so, the only thing to do is deprecate it in 3.1 for removal in 3.2.

I recommend adding a warning to 3.0.2 and removing in 3.1.
Waiting for more 3.x uptake doesn't serve our users well.
IIRC, that was the rationale for cmp() removal in 3.0.1.
Even in 2.x, these methods were documented as being 
unreliable and were removed from the 3.0 docs entirely.
We discussed removing them and most of the work was done.  


Guido, any thoughts?


Raymond

From tleeuwenburg at gmail.com  Thu Mar  5 02:39:24 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Thu, 5 Mar 2009 12:39:24 +1100
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
In-Reply-To: <9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>
	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
Message-ID: <43c8685c0903041739m46f11513yef9c1e923097d77d@mail.gmail.com>

Hi all, just FYI... sorry for any list pollution

I went ahead and tried adding a warning to my local checkout. The test suite
raised a DeprecationWarning -- it appears those methods are currently in use
by other Lib code:


test_multibytecodec_support
test_multiprocessing
/home/tjl/python3/lib/python3.1/queue.py:98: DeprecationWarning:
Queue.empty() is deprecated, and won't be in 3.2.
  DeprecationWarning)
/home/tjl/python3/lib/python3.1/queue.py:109: DeprecationWarning:
Queue.full() is deprecated, and won't be in 3.2.
  DeprecationWarning)
/home/tjl/python3/lib/python3.1/queue.py:98: DeprecationWarning:
Queue.empty() is deprecated, and won't be in 3.2.
  DeprecationWarning)
/home/tjl/python3/lib/python3.1/queue.py:109: DeprecationWarning:
Queue.full() is deprecated, and won't be in 3.2.
  DeprecationWarning)
test_mutants
test_netrc
test_nis

Regards,
-T

On Thu, Mar 5, 2009 at 12:30 PM, Raymond Hettinger <python at rcn.com> wrote:

> Just noticed that the empty() and full() methods were still there.
>>> IIRC, we agreed to take them out (but leaving qsize() exposed).
>>> The docs entries and test cases were taken out, but the actual
>>> methods were accidentally left in.
>>>
>>
>> If so, the only thing to do is deprecate it in 3.1 for removal in 3.2.
>>
>
> I recommend adding a warning to 3.0.2 and removing in 3.1.
> Waiting for more 3.x uptake doesn't serve our users well.
> IIRC, that was the rationale for cmp() removal in 3.0.1.
> Even in 2.x, these methods were documented as being unreliable and were
> removed from the 3.0 docs entirely.
> We discussed removing them and most of the work was done.
>
> Guido, any thoughts?
>
>
> Raymond
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/tleeuwenburg%40gmail.com
>



-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090305/b2e63afb/attachment.htm>

From jnoller at gmail.com  Thu Mar  5 02:41:30 2009
From: jnoller at gmail.com (jnoller at gmail.com)
Date: Thu, 05 Mar 2009 01:41:30 +0000
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
In-Reply-To: <43c8685c0903041739m46f11513yef9c1e923097d77d@mail.gmail.com>
Message-ID: <000e0cd47bb2710f5e04645544f5@google.com>

Yup, I'd need to remove support in multiprocessing too.

On Mar 4, 2009 8:39pm, Tennessee Leeuwenburg <tleeuwenburg at gmail.com> wrote:
> Hi all, just FYI... sorry for any list pollution

> I went ahead and tried adding a warning to my local checkout. The test  
> suite raised a DeprecationWarning -- it appears those methods are  
> currently in use by other Lib code:



> test_multibytecodec_support
> test_multiprocessing
> /home/tjl/python3/lib/python3.1/queue.py:98: DeprecationWarning:  
> Queue.empty() is deprecated, and won't be in 3.2.
> DeprecationWarning)
> /home/tjl/python3/lib/python3.1/queue.py:109: DeprecationWarning:  
> Queue.full() is deprecated, and won't be in 3.2.

> DeprecationWarning)
> /home/tjl/python3/lib/python3.1/queue.py:98: DeprecationWarning:  
> Queue.empty() is deprecated, and won't be in 3.2.
> DeprecationWarning)
> /home/tjl/python3/lib/python3.1/queue.py:109: DeprecationWarning:  
> Queue.full() is deprecated, and won't be in 3.2.

> DeprecationWarning)
> test_mutants
> test_netrc
> test_nis

> Regards,
> -T

> On Thu, Mar 5, 2009 at 12:30 PM, Raymond Hettinger python at rcn.com> wrote:



> Just noticed that the empty() and full() methods were still there.

> IIRC, we agreed to take them out (but leaving qsize() exposed).

> The docs entries and test cases were taken out, but the actual

> methods were accidentally left in.





> If so, the only thing to do is deprecate it in 3.1 for removal in 3.2.





> I recommend adding a warning to 3.0.2 and removing in 3.1.

> Waiting for more 3.x uptake doesn't serve our users well.

> IIRC, that was the rationale for cmp() removal in 3.0.1.

> Even in 2.x, these methods were documented as being unreliable and were  
> removed from the 3.0 docs entirely.

> We discussed removing them and most of the work was done.



> Guido, any thoughts?





> Raymond


> _______________________________________________

> Python-Dev mailing list

> Python-Dev at python.org

> http://mail.python.org/mailman/listinfo/python-dev

> Unsubscribe:  
> http://mail.python.org/mailman/options/python-dev/tleeuwenburg%40gmail.com






> --
> --------------------------------------------------
> Tennessee Leeuwenburg
> http://myownhat.blogspot.com/
> "Don't believe everything you think"

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090305/1a65c9db/attachment.htm>

From python at rcn.com  Thu Mar  5 02:45:08 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 4 Mar 2009 17:45:08 -0800
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
References: <000e0cd47bb2710f5e04645544f5@google.com>
Message-ID: <84998BC6982B4B51BFC86D2DB7500B09@RaymondLaptop1>

> Yup, I'd need to remove support in multiprocessing too.

>> ahead and tried adding a warning to my local checkout. 

Thanks for quickly checking this out.

Will be nice to get the API cleaned-up.
Leaving out part of the clean-up was a mistake.


Raymond

From tleeuwenburg at gmail.com  Thu Mar  5 02:49:10 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Thu, 5 Mar 2009 12:49:10 +1100
Subject: [Python-Dev] What to do about failing tests
Message-ID: <43c8685c0903041749g626b40d3o9e5584beb14c413@mail.gmail.com>

On my local checkout I notice a number of failing tests (test_fileio
test_grp test_io test_urllib2_localnet). Is there anything that I should
attempt to do regarding these?

-Tennessee
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090305/c04ec5b1/attachment-0001.htm>

From tleeuwenburg at gmail.com  Thu Mar  5 03:04:29 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Thu, 5 Mar 2009 13:04:29 +1100
Subject: [Python-Dev] forgive my patch submission
Message-ID: <43c8685c0903041804x74f6dc14hf84090c0be7c72c4@mail.gmail.com>

I am trying to learn the systems... I'm not trying to force a particular
approach to the solution but I want to learn how to correctly follow the
process. Please feel free to reject this -- I'm not trying to jump the gun
given there isn't even an agreed requirement at this stage.

Regards,
-Tennessee

-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090305/0f85f919/attachment.htm>

From benjamin at python.org  Thu Mar  5 05:08:50 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 4 Mar 2009 22:08:50 -0600
Subject: [Python-Dev] What to do about failing tests
In-Reply-To: <43c8685c0903041749g626b40d3o9e5584beb14c413@mail.gmail.com>
References: <43c8685c0903041749g626b40d3o9e5584beb14c413@mail.gmail.com>
Message-ID: <1afaf6160903042008v7f71203cp49196f2a9467915b@mail.gmail.com>

2009/3/4 Tennessee Leeuwenburg <tleeuwenburg at gmail.com>:
> On my local checkout I notice a number of failing tests (test_fileio
> test_grp test_io test_urllib2_localnet). Is there anything that I should
> attempt to do regarding these?

Try "svn up". Those should be fixed now.



-- 
Regards,
Benjamin

From lie.1296 at gmail.com  Thu Mar  5 05:16:50 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Thu, 05 Mar 2009 15:16:50 +1100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <49AEB8AE.5060102@pearwood.info>
References: <loom.20090301T182201-999@post.gmane.org>	<goi2i5$n0n$1@ger.gmane.org>	<49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org>	<49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<golplk$mld$2@ger.gmane.org>	<golslt$1as$1@ger.gmane.org>	<Pine.LNX.4.64.0903040739590.12396@kimball.webabinitio.net>	<cc7430500903040824w4e43d111u3a54b59b7c2b8cf8@mail.gmail.com>
	<49AEB8AE.5060102@pearwood.info>
Message-ID: <gonjno$21e$1@ger.gmane.org>

Steven D'Aprano wrote:
>> I also can't think of an alternative
>> explanation, so thus far, it's resistant to false positive semantics.
> 
> "The keys don't expire with time."
> "It's stable against accidental deletions."
> "It's stable against accidentally over-writing values."

Add to that:
"The StableDict is stable because it has no bugs, unlike the buggy dict"


From lie.1296 at gmail.com  Thu Mar  5 05:32:58 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Thu, 05 Mar 2009 15:32:58 +1100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <gomb08$rgk$1@ger.gmane.org>
References: <loom.20090301T182201-999@post.gmane.org>	<1afaf6160903021228h40321987j5046946cdd825518@mail.gmail.com>	<gohj2l$63r$1@ger.gmane.org>	<ca471dc20903021346j4cd2c29as38b8b844bc4b8e1e@mail.gmail.com>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org><49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org><49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<49AE3573.3020005@g.nevcal.com>	<918214AD-5612-4254-A28B-F66CE6E1C6DA@activestate.com>	<golq0g$ooa$1@ger.gmane.org>	<49AE7346.50206@gmail.com>	<gom1mc$jku$1@ger.gmane.org>
	<gomb08$rgk$1@ger.gmane.org>
Message-ID: <gonkm0$3r7$1@ger.gmane.org>

Terry Reedy wrote:
> Lie Ryan wrote:
> 
>> Isn't ordered dictionary essentially also an "always sorted" 
>> container? It is always sorted depending on the order of insertion? I 
>> can't see any technical reason why the data structure can't 
>> accommodate them both. Can you point me to a discussion on this?
> 
> Appending an item at the end of a sequence is O(1), no search required. 
>  Inserting an item at a random 'sorted' point requires at best an 
> O(logN) search.  Insertion itself is O(1) to O(N) depending on the 
> structure.

Yeah, but with a proper algorithm[1] it is possible to get a O(1) append 
(which is the characteristic we want for insertion order dictionary, 
while falling back to O(log n) if we explicitly give comparer function 
(or comparison key extractor).

[1] The insertion algorithm will test for where to insert from the end 
of the list. This way, insertion-order dict will still be O(1) (with an 
increased constant), else if custom order is specified insertion it will 
be O(n)

#UNTESTED BECAUSE I DON'T HAVE PYTHON CURRENTLY

# Note that it derives from OrderDict
class MyOrderDict(OrderDict):
     def __init__(*args, **kwds):
         if len(args) > 2:
             raise TypeError('expected at most 2 arguments')
         if len(args) == 2:
             self._cmp, args = args[0], args[1:]
         else:
             self._cmp = lambda a, b: True
         if not hasattr(self, '_keys'):
             self._keys = []
         self.update(*args, **kwds)

     def __setitem__(self, key, value):
         if key not in self:
             self._key.append(None)
             for i, k in enumerate(reversed(self._key)):
                 i = -i - 1
                 if self._cmp((k, self[k]), (key, value)):
                     self._key[i], self._key[i - 1] = k, key
                 else:
                     self._key[i] = k
         dict.__setitem__(self, key, value)


From steve at holdenweb.com  Thu Mar  5 06:04:36 2009
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 05 Mar 2009 00:04:36 -0500
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <eae285400903041336h3be07800sdaa8afe276014c5@mail.gmail.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>	<29052.1223516380@parc.com>	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>	<48EE2C77.1050809@palladion.com>
	<49AD91B4.4070005@palladion.com>	<ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>	<20090303211757.12555.1049299398.divmod.xquotient.5145@weber.divmod.com>	<ca471dc20903031420r62af26a1o93b6256c7d864acc@mail.gmail.com>	<20090304053107.12555.475067956.divmod.xquotient.5237@weber.divmod.com>	<5A50A6EE-1DF9-413D-9FCC-29EA4007B79F@solarsail.hcs.harvard.edu>
	<eae285400903041336h3be07800sdaa8afe276014c5@mail.gmail.com>
Message-ID: <49AF5D64.7010701@holdenweb.com>

Daniel Stutzbach wrote:
> On Wed, Mar 4, 2009 at 3:14 PM, Ivan Krsti?
> <krstic at solarsail.hcs.harvard.edu
> <mailto:krstic at solarsail.hcs.harvard.edu>> wrote:
> 
>     I spent about a half hour sometime in the last month talking this
>     through with Itamar, though not in great detail. I'd be interested
>     in sitting down with both of you and trying to establish more
>     precisely how much work is necessary to get something to actually
>     happen here. I won't outright promise a PEP, but I'll promise at the
>     very least to write down elaborate notes that someone could turn
>     into a PEP relatively straightforwardly. Deal?
> 
> 
> Will any or all of you be at PyCon?  I'd be willing to put in the extra
> work to turn your notes into a PEP.
> 
OPEN SPACE!
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at holdenweb.com  Thu Mar  5 06:04:36 2009
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 05 Mar 2009 00:04:36 -0500
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <eae285400903041336h3be07800sdaa8afe276014c5@mail.gmail.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>	<29052.1223516380@parc.com>	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>	<48EE2C77.1050809@palladion.com>
	<49AD91B4.4070005@palladion.com>	<ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>	<20090303211757.12555.1049299398.divmod.xquotient.5145@weber.divmod.com>	<ca471dc20903031420r62af26a1o93b6256c7d864acc@mail.gmail.com>	<20090304053107.12555.475067956.divmod.xquotient.5237@weber.divmod.com>	<5A50A6EE-1DF9-413D-9FCC-29EA4007B79F@solarsail.hcs.harvard.edu>
	<eae285400903041336h3be07800sdaa8afe276014c5@mail.gmail.com>
Message-ID: <49AF5D64.7010701@holdenweb.com>

Daniel Stutzbach wrote:
> On Wed, Mar 4, 2009 at 3:14 PM, Ivan Krsti?
> <krstic at solarsail.hcs.harvard.edu
> <mailto:krstic at solarsail.hcs.harvard.edu>> wrote:
> 
>     I spent about a half hour sometime in the last month talking this
>     through with Itamar, though not in great detail. I'd be interested
>     in sitting down with both of you and trying to establish more
>     precisely how much work is necessary to get something to actually
>     happen here. I won't outright promise a PEP, but I'll promise at the
>     very least to write down elaborate notes that someone could turn
>     into a PEP relatively straightforwardly. Deal?
> 
> 
> Will any or all of you be at PyCon?  I'd be willing to put in the extra
> work to turn your notes into a PEP.
> 
OPEN SPACE!
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From tleeuwenburg at gmail.com  Thu Mar  5 07:43:16 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Thu, 5 Mar 2009 17:43:16 +1100
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
In-Reply-To: <000e0cd47bb2710f5e04645544f5@google.com>
References: <43c8685c0903041739m46f11513yef9c1e923097d77d@mail.gmail.com>
	<000e0cd47bb2710f5e04645544f5@google.com>
Message-ID: <43c8685c0903042243t8955312x8abdace628de3eb7@mail.gmail.com>

Hi Jesse,

I'm not sure what the most appropriate thing to do is. I could:
  (a) leave any multiprocessing changes to you,
  (b) alter the functioning of the method queue_empty inside
test_multiprocessing to test for emptiness in a different manner

I'm happy to go ahead and try my hand at making the appropriate changes but
I don't want to tread in areas of the code that other people have ownership
of.

It appears as though the only place in multiprocessing which uses the
deprecated call is in the test_multiprocessing file.

I also found a call to Queue.empty in wsgui.py. I don't see any authorship
tags at the top of this file but the last commiter was Andrew Kuchling. Do I
need to contact him regarding this or is it appropriate for me to make the
appropriate modifications without consulting him?

Apologies to anyone if I appear to be overly tentative OR overly pushy -- I
am aware that some people take a great deal of personal ownership of their
works, while others are impatient with a soft approach.

Regards,
-Tennessee

On Thu, Mar 5, 2009 at 12:41 PM, <jnoller at gmail.com> wrote:

> Yup, I'd need to remove support in multiprocessing too.
>
>
> On Mar 4, 2009 8:39pm, Tennessee Leeuwenburg <tleeuwenburg at gmail.com>
> wrote:
> > Hi all, just FYI... sorry for any list pollution
> >
> > I went ahead and tried adding a warning to my local checkout. The test
> suite raised a DeprecationWarning -- it appears those methods are currently
> in use by other Lib code:
> >
> >
> >
> > test_multibytecodec_support
> > test_multiprocessing
> > /home/tjl/python3/lib/python3.1/queue.py:98: DeprecationWarning:
> Queue.empty() is deprecated, and won't be in 3.2.
> >   DeprecationWarning)
> > /home/tjl/python3/lib/python3.1/queue.py:109: DeprecationWarning:
> Queue.full() is deprecated, and won't be in 3.2.
> >
> >   DeprecationWarning)
> > /home/tjl/python3/lib/python3.1/queue.py:98: DeprecationWarning:
> Queue.empty() is deprecated, and won't be in 3.2.
> >   DeprecationWarning)
> > /home/tjl/python3/lib/python3.1/queue.py:109: DeprecationWarning:
> Queue.full() is deprecated, and won't be in 3.2.
> >
> >   DeprecationWarning)
> > test_mutants
> > test_netrc
> > test_nis
> >
> > Regards,
> > -T
> >
> > On Thu, Mar 5, 2009 at 12:30 PM, Raymond Hettinger python at rcn.com>
> wrote:
> >
> >
> >
> > Just noticed that the empty() and full() methods were still there.
> >
> > IIRC, we agreed to take them out (but leaving qsize() exposed).
> >
> > The docs entries and test cases were taken out, but the actual
> >
> > methods were accidentally left in.
> >
> >
> >
> >
> >
> > If so, the only thing to do is deprecate it in 3.1 for removal in 3.2.
> >
> >
> >
> >
> >
> > I recommend adding a warning to 3.0.2 and removing in 3.1.
> >
> > Waiting for more 3.x uptake doesn't serve our users well.
> >
> > IIRC, that was the rationale for cmp() removal in 3.0.1.
> >
> > Even in 2.x, these methods were documented as being unreliable and were
> removed from the 3.0 docs entirely.
> >
> > We discussed removing them and most of the work was done.
> >
> >
> >
> > Guido, any thoughts?
> >
> >
> >
> >
> >
> > Raymond
> >
> >
> > _______________________________________________
> >
> > Python-Dev mailing list
> >
> > Python-Dev at python.org
> >
> > http://mail.python.org/mailman/listinfo/python-dev
> >
> > Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/tleeuwenburg%40gmail.com
> >
> >
> >
> >
> >
> >
> > --
> > --------------------------------------------------
> > Tennessee Leeuwenburg
> > http://myownhat.blogspot.com/
> > "Don't believe everything you think"
> >
>



-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090305/27fbd09b/attachment-0001.htm>

From Dennis.Benzinger at gmx.net  Thu Mar  5 08:11:37 2009
From: Dennis.Benzinger at gmx.net (Dennis Benzinger)
Date: Thu, 05 Mar 2009 08:11:37 +0100
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory
 to	collections ready for pronouncement
In-Reply-To: <49AEF3D9.1060303@gmail.com>
References: <loom.20090301T182201-999@post.gmane.org>	<49AC5A9C.3030507@gmail.com>	<A69E6DE76ADE4B668FDA88F35DC32E40@RaymondLaptop1>	<goi2i5$n0n$1@ger.gmane.org>	<49AD9C5C.7040009@canterbury.ac.nz>	<gokb7l$vvm$1@ger.gmane.org>	<49ADD077.2040307@canterbury.ac.nz>	<49AE1E65.4000905@g.nevcal.com>	<D444028CA8DC4FEAAAF526112EAB8BF8@RaymondLaptop1>	<49AE3573.3020005@g.nevcal.com>	<e5fff6640903040525mfdc687lba745e024e909fd6@mail.gmail.com>
	<49AEC072.9020501@gmx.net> <49AEF3D9.1060303@gmail.com>
Message-ID: <49AF7B29.8050700@gmx.net>

Hi Nick!

Am 04.03.2009 22:34, Nick Coghlan schrieb:
> Dennis Benzinger wrote:
>> I'd prefer encoding the order in the class name, therefore I suggest
>> (Ins|Insertion)(Order|Ordered)Dict, e.g. InsOrderDict. Abbreviating the
>> first group to simply I probably is too confusing because of the use of
>> I as a prefix for interfaces.
> 
> Except I just don't see this proliferation of dict types with different
> sort orders ever happening.

Maybe there's a misunderstanding because I don't see it either. I was
trying to suggest four alternative names for the OrderedDict class. I
don't prefer encoding every possible sort order into the class name. I
just wanted to improve the name of OrderedDict.

> The distinction between OrderedDict and dict is that the order of
> keys()/values()/items() isn't arbitrary the way it is in a regular dict
> - there's a defined order that will always be used.

Yes, the insertion order.

> That's all the name
> tells you - if someone assumes they know what that ordering is without
> actually looking at the documentation (and gets it wrong as a result),
> then I don't see how that is any different from the fact that someone
> might mistakenly assume that list.sort() puts the items in descending
> order instead of ascending order.

And because that's all the name tells you I suggested to make the name
more clear by prepending Ins or Insertion.

> For other sort orders, it seems far more likely to me that a
> collections.SortedMap type would be added at some point in the future
> that accepts a key function like the one accepted by sorted() and
> list.sort(). Such a data type would make different trade-offs between
> insertion, deletion and lookup speeds than those made in the hash map
> based OrderedDict.
> [...]

Yes.


Dennis Benzinger

From josiah.carlson at gmail.com  Thu Mar  5 08:46:18 2009
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Wed, 4 Mar 2009 23:46:18 -0800
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <ca471dc20903041331l7922d796x5028ebf85692e8ea@mail.gmail.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com>
	<ca471dc20903041331l7922d796x5028ebf85692e8ea@mail.gmail.com>
Message-ID: <e6511dbf0903042346g45b5723x8b097768153e9f66@mail.gmail.com>

On Wed, Mar 4, 2009 at 1:31 PM, Guido van Rossum <guido at python.org> wrote:
> On Wed, Mar 4, 2009 at 1:27 PM, Chris McDonough <chrism at plope.com> wrote:
>> Daniel Stutzbach wrote:
>>> On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum <guido at python.org> wrote:
>>>
>>>> The same as always. We don't change APIs in bugfix releases.
>>>>
>>>
>>> This question is actually for the Zope folks and others who have had
>>> problems with the 2.6 asyncore/asynchat:
>>>
>>> Are any of the problems due to a change in the documented API... or are they
>>> all due to changes in undocumented internals that your code relied on?
>>
>> As far as I can tell, asyncore/asynchat is all "undocumented internals". ?Any
>> use of asyncore in anger will use internals; there never was any well-understood
>> API to these modules. ?Medusa itself (from which asyncore and asynchat were
>> derived) appears to itself break with the changes to asyncore/asynchat in 2.6
>> (at least it appears to use attributes like "ac_out_buffer" which were removed
>> in 2.6; this is not "Zope's version"; there is no such animal; this is plain old
>> Medusa 0.5.4).
>>
>> Count me in as one who believes that it would be the lesser of two evils to
>> revert to the older (2.5 and prior) asyncore/asynchat implementations in 2.6.2
>> rather than soldiering on with the 2.6 and 2.6.1 implementation (which almost
>> certainly has fewer, if any, consumers); somebody messed up in 2.6 by making its
>> asyncore/asynchat more forward compatible with 3.0's than backwards compatible
>> with 2.5's and prior; this was just a mistake and keeping old code running
>> should trump any theoretical or puritanical benefit in a dot release.
>
> Then I'd like to hear from the folks who made and reviewed those
> changes to asyncore in 2.6. I can't imagine the changes were done
> without good intentions.

Interestingly enough, I just accidentally found this thread with an
unrelated web search, but it makes sense to reply.

Why the changes to asyncore:
1) the internals of asyncore were inefficient
2) the API and general behavior of the handle_*(), readable() and
writable() methods are unchanged, which are *exactly* what a user
should be overriding with subclassing
3) anyone modifying the *internals* of the implementation for the sake
of speed, shouldn't need to anymore

Maybe I have misread the posts in this thread, but does anyone have
any examples of their code breaking when only overriding handle_*(),
readable(), and writable() methods?

 - Josiah

From josiah.carlson at gmail.com  Thu Mar  5 09:00:31 2009
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Thu, 5 Mar 2009 00:00:31 -0800
Subject: [Python-Dev] What type of object mmap.read_byte should return
	on py3k?
In-Reply-To: <49AAD7D8.9080301@m2.ccsnet.ne.jp>
References: <49A91DA4.5080903@m2.ccsnet.ne.jp>
	<200902281521.48100.victor.stinner@haypocalc.com>
	<49A956F0.9050700@m2.ccsnet.ne.jp> <49AAD7D8.9080301@m2.ccsnet.ne.jp>
Message-ID: <e6511dbf0903050000n27c1ff2bq2c82c3e60ee3d736@mail.gmail.com>

On Sun, Mar 1, 2009 at 10:45 AM, Hirokazu Yamamoto
<ocean-city at m2.ccsnet.ne.jp> wrote:
> I uploaded the patch with choice (a)
> http://bugs.python.org/file13215/py3k_mmap_and_bytes.patch
> If (b) is suitable, I'll rewrite the patch.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/josiah.carlson%40gmail.com
>

Has anyone been using mmap in python 3k to know what is more
intuitive?  When I was using mmap in python 2.4, I never used the
read/write methods, I stuck with slicing, which was very convenient
with 2.4 non-unicode strings.  I don't really have an intuition on 3.x
bytes.

 - Josiah

From chris at simplistix.co.uk  Thu Mar  5 10:08:02 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Thu, 05 Mar 2009 09:08:02 +0000
Subject: [Python-Dev] How do I get commit access?
In-Reply-To: <ca471dc20903021050g6b356795ndd1c8341d87360c@mail.gmail.com>
References: <49A7A906.8000008@simplistix.co.uk>	
	<0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1>	
	<49A7AD8C.5030209@simplistix.co.uk>	
	<A8EBDAFE66D843DBBAFFD2B942917A5D@RaymondLaptop1>	
	<49A7B501.4070508@simplistix.co.uk> <go8ed0$vnq$1@ger.gmane.org>	
	<18856.9294.373185.493616@montanaro.dyndns.org>	
	<49ABCFCA.5020402@simplistix.co.uk>	
	<18859.64200.859197.70559@montanaro.dyndns.org>	
	<49ABFA94.3090807@simplistix.co.uk>
	<ca471dc20903021050g6b356795ndd1c8341d87360c@mail.gmail.com>
Message-ID: <49AF9672.9010309@simplistix.co.uk>

Guido van Rossum wrote:
> I'd like to jump in and vouch for Chris. I've known him for many years
> and while we haven't worked closely I expect he'd be a valuable
> contributor. So +1 from me for giving Chris commit privileges for core
> Python. 

Thanks :-) I can't promise how *much* time I'll be able to give, but 
when I do have itches, I'll certainly be scratching them...

> (Chris, I assume you'll go through an apprentice phase where
> you'll be letting another committer review your changes before you
> commit them yourself. 

How does Python work w.r.t. to dev? I'm used to the branch/merge pattern:

- create branch, say chris.withers-issue4308
- do work on that branch
- request code review
- merge branch to appropriate release branches and trunk
- delete branch

If this isn't the pattern Python uses, why isn't it? ;-)

> Rietveld at codereview.appspot.com should be
> helpful for getting your code reviewed. (Just use upload.py instead of
> the Create Issue form. :-)

OK, although I'd prefer the branch/merge pattern, less toolage required...

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Thu Mar  5 10:26:43 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Thu, 05 Mar 2009 09:26:43 +0000
Subject: [Python-Dev] patch commit policies (was [issue4308] repr of
 httplib.IncompleteRead is stupid)
In-Reply-To: <49AC394E.2000903@v.loewis.de>
References: <49AC394E.2000903@v.loewis.de>
Message-ID: <49AF9AD3.2090405@simplistix.co.uk>

Martin v. L?wis wrote:
> Martin v. L?wis <martin at v.loewis.de> added the comment:
> 
>> So all Chris has to do to get this applied to 2.5 is craft an exploit based
>> on the current behavior, right? ;-)
> 
> Right :-) Of course, security patches should see a much more careful
> review than regular bug fixes.

Well, it's funny you say that, since where I bumped into this, the bug 
was effectively DOS'ing a couple of mailservers as a result of 
mailinglogger sending out log entries of uncaught exceptions such as 
this and so emitting 100Mb emails whenever the foreign server chose not 
to deliver the whole chunk requested...

That aside, is it actually a python-wide policy to *forbid* patching 
older releases where the patch isn't security-related?

I can understand the "no more releases unless there are security 
problems", but what's the harm in applying a patch to an old version 
branch on the off chance that a security release might be made some time?

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Thu Mar  5 10:35:07 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Thu, 05 Mar 2009 09:35:07 +0000
Subject: [Python-Dev] running the tests...
Message-ID: <49AF9CCB.2050509@simplistix.co.uk>

Hi All,

I found the very brief snippet on test-running at:

http://python.org/dev/faq/#how-to-test-a-patch

...so thought I'd ask here:

- what's the canonical way to run "all the tests"?

- what's the canonical way to run the tests for just the package being 
patched? (I'm assuming it's a standard library package here...)

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From glyph at divmod.com  Thu Mar  5 10:57:15 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Thu, 05 Mar 2009 09:57:15 -0000
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of	medusa
In-Reply-To: <5A50A6EE-1DF9-413D-9FCC-29EA4007B79F@solarsail.hcs.harvard.edu>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>
	<5003.1223487033@parc.com>
	<e6511dbf0810081426q4c576bbak8ab9a7e1d6936254@mail.gmail.com>
	<29052.1223516380@parc.com>
	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>
	<48EE2C77.1050809@palladion.com> <49AD91B4.4070005@palladion.com>
	<ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
	<20090303211757.12555.1049299398.divmod.xquotient.5145@weber.divmod.com>
	<ca471dc20903031420r62af26a1o93b6256c7d864acc@mail.gmail.com>
	<20090304053107.12555.475067956.divmod.xquotient.5237@weber.divmod.com>
	<5A50A6EE-1DF9-413D-9FCC-29EA4007B79F@solarsail.hcs.harvard.edu>
Message-ID: <20090305095715.12555.1989619352.divmod.xquotient.5319@weber.divmod.com>

On 4 Mar, 09:14 pm, krstic at solarsail.hcs.harvard.edu wrote:
>I spent about a half hour sometime in the last month talking this 
>through with Itamar, though not in great detail. I'd be interested in 
>sitting down with both of you and trying to establish more precisely 
>how much work is necessary to get something to actually happen here. I 
>won't outright promise a PEP, but I'll promise at the very least to 
>write down elaborate notes that someone could turn into a PEP 
>relatively straightforwardly. Deal?

Absolutely.  I really appreciate the offer.  As the other gentlemen 
suggested, PyCon would be an ideal venue for doing this.  Are you going 
to be there?

I'll hopefully be seeing your talk this evening, but I suspect that 
would be a pretty bad venue to try to get this done ;-).

From glyph at divmod.com  Thu Mar  5 10:58:54 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Thu, 05 Mar 2009 09:58:54 -0000
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of	medusa
In-Reply-To: <92A276C3-3FE3-4060-BCE4-F8865E2E4295@python.org>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<20090304194446.12555.1053607582.divmod.xquotient.5310@weber.divmod.com>
	<92A276C3-3FE3-4060-BCE4-F8865E2E4295@python.org>
Message-ID: <20090305095854.12555.74540015.divmod.xquotient.5321@weber.divmod.com>


On 4 Mar, 08:28 pm, barry at python.org wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>On Mar 4, 2009, at 2:44 PM, glyph at divmod.com wrote:
>>Maintaining compatibility with the 2.6.x version of asyncore 
>>presupposes that *someone* has written some software against that 
>>version of asyncore and it might break if they installed an upgrade, 
>>though.

>FWIW, I use smtpd.py and a few of the asyncore APIs  (.loop(), 
>.socket_map.clear(), and .close_all()) in the Mailman 3 test  suite. 
>That only works on Python 2.6 and I don't recall even a hiccup  when 
>moving from 2.5 to 2.6.

Well, in that case, I withdraw even my disinterested suggestion :).

From glyph at divmod.com  Thu Mar  5 11:00:23 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Thu, 05 Mar 2009 10:00:23 -0000
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of	medusa
In-Reply-To: <eae285400903041336h3be07800sdaa8afe276014c5@mail.gmail.com>
References: <a7a2b76b0810081015p55710cd1i589a45bf78fde0fb@mail.gmail.com>
	<29052.1223516380@parc.com>
	<ca471dc20810081918q29e6bf1cx651827abe35ae3d7@mail.gmail.com>
	<48EE2C77.1050809@palladion.com> <49AD91B4.4070005@palladion.com>
	<ca471dc20903031246h78161bd2p3ca80da46c2c3b01@mail.gmail.com>
	<20090303211757.12555.1049299398.divmod.xquotient.5145@weber.divmod.com>
	<ca471dc20903031420r62af26a1o93b6256c7d864acc@mail.gmail.com>
	<20090304053107.12555.475067956.divmod.xquotient.5237@weber.divmod.com>
	<5A50A6EE-1DF9-413D-9FCC-29EA4007B79F@solarsail.hcs.harvard.edu>
	<eae285400903041336h3be07800sdaa8afe276014c5@mail.gmail.com>
Message-ID: <20090305100023.12555.1946371741.divmod.xquotient.5323@weber.divmod.com>


On 4 Mar, 09:36 pm, daniel at stutzbachenterprises.com wrote:
>Will any or all of you be at PyCon?  I'd be willing to put in the extra 
>work
>to turn your notes into a PEP.

I definitely will be.  I'll see you there!

From g.brandl at gmx.net  Thu Mar  5 10:58:17 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 05 Mar 2009 10:58:17 +0100
Subject: [Python-Dev] running the tests...
In-Reply-To: <49AF9CCB.2050509@simplistix.co.uk>
References: <49AF9CCB.2050509@simplistix.co.uk>
Message-ID: <goo7nq$jh7$1@ger.gmane.org>

Chris Withers schrieb:
> Hi All,
> 
> I found the very brief snippet on test-running at:
> 
> http://python.org/dev/faq/#how-to-test-a-patch
> 
> ....so thought I'd ask here:
> 
> - what's the canonical way to run "all the tests"?

Assuming UNIXy OSes: make test, or if you want to save a bit of time,
make quicktest.

> - what's the canonical way to run the tests for just the package being 
> patched? (I'm assuming it's a standard library package here...)

In 90% of all cases, the test suite is called like the module, so

   ./python Lib/test/regrtest.py test_foo

where foo is the module name should do it.  In the other 10%, you'll have
to look around a bit for the tests.  But since patching should always
include adding a test, it's necessary anyway ;)

cheers,
Georg


From stefan_ml at behnel.de  Thu Mar  5 11:39:20 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 05 Mar 2009 11:39:20 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
References: <49ABCFC9.1070508@gmail.com>
	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
Message-ID: <gooa4o$rrh$1@ger.gmane.org>

Ivan Krsti? wrote:
> On Mar 4, 2009, at 12:32 PM, James Y Knight wrote:
>> I think html5lib would be a better candidate for an imrpoved HTML
>> parser in the stdlib than BeautifulSoup.
> 
> While we're talking about alternatives, Ian Bicking appears to swear by
> lxml:
> 
> <http://blog.ianbicking.org/2008/12/10/lxml-an-underappreciated-web-scraping-library/>

I second that. ;)

And, BTW, I wouldn't mind getting lxml into the stdlib either.

Stefan


From ncoghlan at gmail.com  Thu Mar  5 12:19:30 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 05 Mar 2009 21:19:30 +1000
Subject: [Python-Dev] patch commit policies (was [issue4308] repr of
 httplib.IncompleteRead is stupid)
In-Reply-To: <49AF9AD3.2090405@simplistix.co.uk>
References: <49AC394E.2000903@v.loewis.de> <49AF9AD3.2090405@simplistix.co.uk>
Message-ID: <49AFB542.9010101@gmail.com>

Chris Withers wrote:
> That aside, is it actually a python-wide policy to *forbid* patching
> older releases where the patch isn't security-related?
> 
> I can understand the "no more releases unless there are security
> problems", but what's the harm in applying a patch to an old version
> branch on the off chance that a security release might be made some time?

Precisely because if we're doing a security release, we want to be sure
that that the *only* change that gets included is the fix to the
security problem. If there are other changes hanging around on the
branch, then the release would need to go at least a single release
candidate cycle (which we don't want to need to do for security releases
on old branches).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Thu Mar  5 12:23:53 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 05 Mar 2009 21:23:53 +1000
Subject: [Python-Dev] running the tests...
In-Reply-To: <goo7nq$jh7$1@ger.gmane.org>
References: <49AF9CCB.2050509@simplistix.co.uk> <goo7nq$jh7$1@ger.gmane.org>
Message-ID: <49AFB649.1020801@gmail.com>

Georg Brandl wrote:
> Chris Withers schrieb:
>> Hi All,
>>
>> I found the very brief snippet on test-running at:
>>
>> http://python.org/dev/faq/#how-to-test-a-patch
>>
>> ....so thought I'd ask here:
>>
>> - what's the canonical way to run "all the tests"?
> 
> Assuming UNIXy OSes: make test, or if you want to save a bit of time,
> make quicktest.
> 
>> - what's the canonical way to run the tests for just the package being 
>> patched? (I'm assuming it's a standard library package here...)
> 
> In 90% of all cases, the test suite is called like the module, so
> 
>    ./python Lib/test/regrtest.py test_foo
> 
> where foo is the module name should do it.  In the other 10%, you'll have
> to look around a bit for the tests.  But since patching should always
> include adding a test, it's necessary anyway ;)

My personal preferences:

Thorough: ./python -m test.regrtest -uall
Typical: ./python -m test.regrtest
Specific: ./python -m test.regrtest test_mod1 test_mod2

(enabling the relevant test resources via -uall or something more
specific is especially important when working on things like the
networking code or the audio support - many of the relevant tests are
skipped by default)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From chris at simplistix.co.uk  Thu Mar  5 13:42:18 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Thu, 05 Mar 2009 12:42:18 +0000
Subject: [Python-Dev] running the tests...
In-Reply-To: <49AFB649.1020801@gmail.com>
References: <49AF9CCB.2050509@simplistix.co.uk> <goo7nq$jh7$1@ger.gmane.org>
	<49AFB649.1020801@gmail.com>
Message-ID: <49AFC8AA.5010103@simplistix.co.uk>

Nick Coghlan wrote:
> My personal preferences:
> 
> Thorough: ./python -m test.regrtest -uall
> Typical: ./python -m test.regrtest
> Specific: ./python -m test.regrtest test_mod1 test_mod2

This looks good, I assume this would work on Windows too?

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From millbr02 at luther.edu  Thu Mar  5 14:53:04 2009
From: millbr02 at luther.edu (Brad Miller)
Date: Thu, 5 Mar 2009 07:53:04 -0600
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <49AC5C0B.4070806@v.loewis.de>
References: <mailman.29.1235991618.1178.python-dev@python.org>
	<p06240803c5d1f82dbc4b@10.0.1.221> <49AC5C0B.4070806@v.loewis.de>
Message-ID: <e45086040903050553m44f7bce7xca474d658e0c977f@mail.gmail.com>

On Mon, Mar 2, 2009 at 4:22 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> > Would whoever is responsible for IDLE please take a look at the patches
> > I submitted for Python 2 & 3 [tracker IDs 5233 and 5234 respectively].
> > These change the behavior of IDLE so that IDLESTARTUP or PYTHONSTARTUP
> > files are executed with each restart. This allows loading frequently
> > used packages, personal utilities, etc. automatically at each restart. I
> > consider this a very important problem in IDLE, especially when using it
> > to teach.
>
> Just to put this into perspective: I personally don't see that as a very
> important problem. I didn't know IDLESTARTUP existed, and I use
> PYTHONSTARTUP only for the command line (to setup readline and history).
> I think there are many more open issues that are *way* more important.


Martin,

  No disrespect intended but I don't see how this puts things into
perspective.  I'm writing to you from the annual computer science education
conference (SIGCSE)  where Python is clearly gaining ground as an important
language for teaching computer science.

It seems logical to me that the committers are high powered Python users who
don't think much about Python being used in education.  I'm just as
frustrated as Mitchell about a patch for displaying ranges and
dict_keys/values objects in a more user friendly way.  I submitted this
patch during the 3.0 alpha phase and it is still sitting around.  For me
this is a serious problem, but I can understand how it seems pretty minor to
others, who are not teaching new programmers.

So what is the solution?  The obvious solution is for one of us, that is
someone who uses Python as an education tool, to become a committer.  This
seems problematic to me.  Although I'm willing to be a committer, and I'm
confident I have the development skills necessary to be a committer I don't
have the time to develop the resume of patches needed to earn that
privilege.

It would be nice if we could find some solution to this.

Brad


>
> This is not to say that the patch should not applied - I haven't even
> looked at it. It's just a warning that, if no other committer feels this
> is as important as you fell it is, it may not be committed reviewed and
> committed before 3.1.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/bmiller%40luther.edu
>



-- 
Brad Miller
Assistant Professor, Computer Science
Luther College
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090305/f874bcc9/attachment.htm>

From skip at pobox.com  Thu Mar  5 15:47:13 2009
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 5 Mar 2009 08:47:13 -0600
Subject: [Python-Dev] running the tests...
In-Reply-To: <49AFC8AA.5010103@simplistix.co.uk>
References: <49AF9CCB.2050509@simplistix.co.uk> <goo7nq$jh7$1@ger.gmane.org>
	<49AFB649.1020801@gmail.com> <49AFC8AA.5010103@simplistix.co.uk>
Message-ID: <18863.58865.813615.413929@montanaro.dyndns.org>


    >> My personal preferences:
    >> 
    >> Thorough: ./python -m test.regrtest -uall
    >> Typical: ./python -m test.regrtest
    >> Specific: ./python -m test.regrtest test_mod1 test_mod2

    Chris> This looks good, I assume this would work on Windows too?

I believe so, but <wink><jab>you should still get a real OS.</jab></wink>

Skip

From ggpolo at gmail.com  Thu Mar  5 15:53:23 2009
From: ggpolo at gmail.com (Guilherme Polo)
Date: Thu, 5 Mar 2009 11:53:23 -0300
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <e45086040903050553m44f7bce7xca474d658e0c977f@mail.gmail.com>
References: <mailman.29.1235991618.1178.python-dev@python.org>
	<p06240803c5d1f82dbc4b@10.0.1.221> <49AC5C0B.4070806@v.loewis.de>
	<e45086040903050553m44f7bce7xca474d658e0c977f@mail.gmail.com>
Message-ID: <ac2200130903050653g59728facpe9526dacb795db51@mail.gmail.com>

On Thu, Mar 5, 2009 at 10:53 AM, Brad Miller <millbr02 at luther.edu> wrote:
>
>
> On Mon, Mar 2, 2009 at 4:22 PM, "Martin v. L?wis" <martin at v.loewis.de>
> wrote:
>>
>> > Would whoever is responsible for IDLE please take a look at the patches
>> > I submitted for Python 2 & 3 [tracker IDs 5233 and 5234 respectively].
>> > These change the behavior of IDLE so that IDLESTARTUP or PYTHONSTARTUP
>> > files are executed with each restart. This allows loading frequently
>> > used packages, personal utilities, etc. automatically at each restart. I
>> > consider this a very important problem in IDLE, especially when using it
>> > to teach.
>>
>> Just to put this into perspective: I personally don't see that as a very
>> important problem. I didn't know IDLESTARTUP existed, and I use
>> PYTHONSTARTUP only for the command line (to setup readline and history).
>> I think there are many more open issues that are *way* more important.
>
> Martin,
> ??No disrespect intended but I don't see how this puts things into
> perspective. ?I'm writing to you from the annual computer science education
> conference (SIGCSE) ?where Python is clearly gaining ground as an important
> language for teaching computer science.
> It seems logical to me that the committers are high powered Python users who
> don't think much about Python being used in education. ?I'm just as
> frustrated as Mitchell about a patch for displaying ranges and
> dict_keys/values objects in a more user friendly way. ?I submitted this
> patch during the 3.0 alpha phase and it is still sitting around. ?For me
> this is a serious problem, but I can understand how it seems pretty minor to
> others, who are not teaching new programmers.
> So what is the solution? ?The obvious solution is for one of us, that is
> someone who uses Python as an education tool, to become a committer. ?This
> seems problematic to me. ?Although I'm willing to be a committer, and I'm
> confident I have the development skills necessary to be a committer I don't
> have the time to develop the resume of patches needed to earn that
> privilege.
> It would be nice if we could find some solution to this.

Or... IDLE could be taken out from Python. Tkinter is following the
same path too, sadly.
My hope is that by removing IDLE from Python it would bring new
developers that are not necessary python developers (by this I mean
developers of python itself). I changed IDLE quite a bit last year,
but I'm not sure if anyone cared enough to look at it (added tabs, ttk
support, themes, window relayout, and some other things), and I don't
think continuing with it in the stdlib is bringing any benefits.

I have commit access, and although I have been inactive for two or
three weeks (maybe a bit more) now, I have submitted plenty of fixes
for tkinter which are mostly reviewed by Martin, and only, Martin --
when he has time to review or when the fix hits some level of
"important enough" to be looked at. I could just commit these fixes,
but some people would hate me then because I didn't let anyone review,
so I don't really think adding more new committers will bring the
benefits you are expecting.

A different problem also present in both tkinter and IDLE is the lack of tests.

> Brad
>>
>>
>> This is not to say that the patch should not applied - I haven't even
>> looked at it. It's just a warning that, if no other committer feels this
>> is as important as you fell it is, it may not be committed reviewed and
>> committed before 3.1.
>>
>> Regards,
>> Martin
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/bmiller%40luther.edu
>
>
>
> --
> Brad Miller
> Assistant Professor, Computer Science
> Luther College
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com
>
>



-- 
-- Guilherme H. Polo Goncalves

From chris at simplistix.co.uk  Thu Mar  5 16:15:53 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Thu, 05 Mar 2009 15:15:53 +0000
Subject: [Python-Dev] running the tests...
In-Reply-To: <18863.58865.813615.413929@montanaro.dyndns.org>
References: <49AF9CCB.2050509@simplistix.co.uk> <goo7nq$jh7$1@ger.gmane.org>
	<49AFB649.1020801@gmail.com> <49AFC8AA.5010103@simplistix.co.uk>
	<18863.58865.813615.413929@montanaro.dyndns.org>
Message-ID: <49AFECA9.4000004@simplistix.co.uk>

skip at pobox.com wrote:
>     >> My personal preferences:
>     >> 
>     >> Thorough: ./python -m test.regrtest -uall
>     >> Typical: ./python -m test.regrtest
>     >> Specific: ./python -m test.regrtest test_mod1 test_mod2
> 
>     Chris> This looks good, I assume this would work on Windows too?
> 
> I believe so, but <wink><jab>you should still get a real OS.</jab></wink>

I do use plenty of real operating systems, but one of python's biggest 
strengths for me is that it is totally cross platform...

...that and the soundcard in my desktop machine only has windows drivers :-(

*cough*

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From brett at python.org  Thu Mar  5 16:45:41 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 5 Mar 2009 07:45:41 -0800
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
In-Reply-To: <9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>
	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
Message-ID: <bbaeab100903050745q761e7998t5c309df2a93a4202@mail.gmail.com>

On Wed, Mar 4, 2009 at 17:30, Raymond Hettinger <python at rcn.com> wrote:

> Just noticed that the empty() and full() methods were still there.
>>> IIRC, we agreed to take them out (but leaving qsize() exposed).
>>> The docs entries and test cases were taken out, but the actual
>>> methods were accidentally left in.
>>>
>>
>> If so, the only thing to do is deprecate it in 3.1 for removal in 3.2.
>>
>
> I recommend adding a warning to 3.0.2 and removing in 3.1.
> Waiting for more 3.x uptake doesn't serve our users well.
> IIRC, that was the rationale for cmp() removal in 3.0.1.


Right, but that's because 3.0.1 was released within four months of 3.0.0.
Who knows when 3.0.2 will be released. Having some users suddenly get a new
deprecation for some code just because they upgraded their interpreter
doesn't sound very backwards-compatible to me. This is true even for a
PendingDeprecationWarning.

-1 on tossing a warning into 3.0.2 and going +1 with the 3.1 deprecation.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090305/f1813974/attachment.htm>

From brett at python.org  Thu Mar  5 16:48:54 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 5 Mar 2009 07:48:54 -0800
Subject: [Python-Dev] How do I get commit access?
In-Reply-To: <49AF9672.9010309@simplistix.co.uk>
References: <49A7A906.8000008@simplistix.co.uk>
	<A8EBDAFE66D843DBBAFFD2B942917A5D@RaymondLaptop1>
	<49A7B501.4070508@simplistix.co.uk> <go8ed0$vnq$1@ger.gmane.org>
	<18856.9294.373185.493616@montanaro.dyndns.org>
	<49ABCFCA.5020402@simplistix.co.uk>
	<18859.64200.859197.70559@montanaro.dyndns.org>
	<49ABFA94.3090807@simplistix.co.uk>
	<ca471dc20903021050g6b356795ndd1c8341d87360c@mail.gmail.com>
	<49AF9672.9010309@simplistix.co.uk>
Message-ID: <bbaeab100903050748y28576592ka29ff77ec7ac7839@mail.gmail.com>

On Thu, Mar 5, 2009 at 01:08, Chris Withers <chris at simplistix.co.uk> wrote:

> Guido van Rossum wrote:
>
>> I'd like to jump in and vouch for Chris. I've known him for many years
>> and while we haven't worked closely I expect he'd be a valuable
>> contributor. So +1 from me for giving Chris commit privileges for core
>> Python.
>>
>
> Thanks :-) I can't promise how *much* time I'll be able to give, but when I
> do have itches, I'll certainly be scratching them...
>
>  (Chris, I assume you'll go through an apprentice phase where
>> you'll be letting another committer review your changes before you
>> commit them yourself.
>>
>
> How does Python work w.r.t. to dev? I'm used to the branch/merge pattern:
>
> - create branch, say chris.withers-issue4308
> - do work on that branch
> - request code review
> - merge branch to appropriate release branches and trunk
> - delete branch
>
> If this isn't the pattern Python uses, why isn't it? ;-)
>

Because Python started out on CVS and then moved to svn and just never
picked up the branching religion. But there is nothing wrong with creating a
branch in the sandbox and doing work there; just isn't common practice for
anything that isn't going to be a long term project.


>
>
>  Rietveld at codereview.appspot.com should be
>> helpful for getting your code reviewed. (Just use upload.py instead of
>> the Create Issue form. :-)
>>
>
> OK, although I'd prefer the branch/merge pattern, less toolage required...


Right, but that still doesn't preclude the usefulness of being able to leave
comments in Rietveld for a reviewer.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090305/475e47b9/attachment.htm>

From ajaksu at gmail.com  Thu Mar  5 18:17:19 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Thu, 5 Mar 2009 14:17:19 -0300
Subject: [Python-Dev] Tracker cleanup - Roundup hacking report :)
Message-ID: <2d75d7660903050917n340ede0u4746bc8395e07b12@mail.gmail.com>

Hi,
Here's a progress report on the "let's make the tracker a bit better" tasks.

Note: if you make use of saved queries, I recommend reading the
'anyone can remove any queries' issue:
http://psf.upfronthosting.co.za/roundup/meta/issue244

Feedback on meta-tracker open issues, as well as new RFEs and replies
to tracker-discuss threads should make the tracker meeting your needs
a bit more likely :)

Thanks Martin, Stefan, Jeroen, Ezio, Brett, Georg and others that
helped along the way.

Real tracker cleanup will be back soon ;)

Best regards,
Daniel

Open WIP tickets:
    add # of comments to default issue list
        Track the counts of messages and nosy users on a given issue.
        http://psf.upfronthosting.co.za/roundup/meta/issue87
(screenshot: http://psf.upfronthosting.co.za/roundup/meta/file110/msg_nosy_counts.png
)

    Add multiselects for searches
        This allows one to search, e.g., for issues with "Version in
['2.5', '2.6', '2.7']" or "Stage in ['patch review', 'not selected']".
        http://psf.upfronthosting.co.za/roundup/meta/issue243
(example: http://psf.upfronthosting.co.za/roundup/meta/file105/issue_search.html
)

    Add 'Mail me' button to messages and issues
        Similar to becoming nosy retroactively, by getting a past
message after the fact. This (supposedly) allows one to reply to any
message in an issue from email.
        http://psf.upfronthosting.co.za/roundup/meta/issue245

    Have issue ID search work from "search tracker" box
        Passing an ID into the "search tracker" box opens the issue
with the matching ID.
        http://psf.upfronthosting.co.za/roundup/meta/issue123

    Add 'Stage' to results page
        Adds 'Stage' as a column to the search results when 'Display'
is selected.
        http://psf.upfronthosting.co.za/roundup/meta/issue242

    "Search in All Issues" button
        http://psf.upfronthosting.co.za/roundup/meta/issue224

    Have CVS download of issue list put name rather than ID
        http://issues.roundup-tracker.org/issue955070

    Failed logins do not produce an error message
        Add a 'Login as %(user) successful' green-banner on successful logins
        http://psf.upfronthosting.co.za/roundup/meta/issue230

Tickets/ideas on sketching or local testing stage:

    notation to filter by logged-in user
        Allows 'reflexive' queries, like 'issues I've created',
'issues I follow', 'issues assigned to me'. Pending Query security
resolution.
        http://issues.roundup-tracker.org/issue1525113

    Shortcuts for selecting self in user lists: "Add me" for nosy and
"Claim" for assignee.

    Provide prev/next/show list links when going through a list of bugs
        http://psf.upfronthosting.co.za/roundup/meta/issue4

    Ability to specify non-matching criteria in filters/searches
        http://issues.roundup-tracker.org/issue678900

    Email me my current changes without submitting (a.k.a. 'Save')

    Allow searching for ranges for Number attributes
        http://issues.roundup-tracker.org/issue1182919

    input fields should have HTML id attributes
        http://issues.roundup-tracker.org/issue1513369

    Full text "AND" search has message scope, not issue scope
        http://issues.roundup-tracker.org/issue1155657

    Set attributes in the body
        http://psf.upfronthosting.co.za/roundup/meta/issue198

Closed tickets:
    Edit Queries screen should include a link to the search page
        http://psf.upfronthosting.co.za/roundup/meta/issue202
    Denote which dependencies issues have been closed
        http://psf.upfronthosting.co.za/roundup/meta/issue207
    Fix 'not closed' search
        http://psf.upfronthosting.co.za/roundup/meta/issue240
    Add 'Stage' to search page
        http://psf.upfronthosting.co.za/roundup/meta/issue239
    Add creator/assignee to search view
        http://psf.upfronthosting.co.za/roundup/meta/issue180

From jnoller at gmail.com  Thu Mar  5 18:21:34 2009
From: jnoller at gmail.com (jnoller at gmail.com)
Date: Thu, 05 Mar 2009 17:21:34 +0000
Subject: [Python-Dev] Tracker cleanup - Roundup hacking report :)
In-Reply-To: <2d75d7660903050917n340ede0u4746bc8395e07b12@mail.gmail.com>
Message-ID: <0016e644cf3261684904646266b2@google.com>

On Mar 5, 2009 12:17pm, "Daniel (ajax) Diniz" <ajaksu at gmail.com> wrote:
> Hi,

> Here's a progress report on the "let's make the tracker a bit better"  
> tasks.

...snip

Slightly off topic Daniel, but if you see any multiprocessing bugs lurking  
out there, can you make me (jnoller) the assignee?

jesse
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090305/28fdb4cb/attachment-0001.htm>

From jnoller at gmail.com  Thu Mar  5 18:25:55 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 5 Mar 2009 12:25:55 -0500
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
In-Reply-To: <43c8685c0903042243t8955312x8abdace628de3eb7@mail.gmail.com>
References: <43c8685c0903041739m46f11513yef9c1e923097d77d@mail.gmail.com>
	<000e0cd47bb2710f5e04645544f5@google.com>
	<43c8685c0903042243t8955312x8abdace628de3eb7@mail.gmail.com>
Message-ID: <4222a8490903050925qc23a528u820207e9658410d7@mail.gmail.com>

On Thu, Mar 5, 2009 at 1:43 AM, Tennessee Leeuwenburg
<tleeuwenburg at gmail.com> wrote:
> Hi Jesse,
>
> I'm not sure what the most appropriate thing to do is. I could:
> ? (a) leave any multiprocessing changes to you,
> ? (b) alter the functioning of the method queue_empty inside
> test_multiprocessing to test for emptiness in a different manner
>
> I'm happy to go ahead and try my hand at making the appropriate changes but
> I don't want to tread in areas of the code that other people have ownership
> of.
>
> It appears as though the only place in multiprocessing which uses the
> deprecated call is in the test_multiprocessing file.
>
> I also found a call to Queue.empty in wsgui.py. I don't see any authorship
> tags at the top of this file but the last commiter was Andrew Kuchling. Do I
> need to contact him regarding this or is it appropriate for me to make the
> appropriate modifications without consulting him?
>
> Apologies to anyone if I appear to be overly tentative OR overly pushy -- I
> am aware that some people take a great deal of personal ownership of their
> works, while others are impatient with a soft approach.
>
> Regards,
> -Tennessee

I'm an equal opportunity patch employer: I'll take it from anyone.

That being said, I would open a new issue on the tracker
(bugs.python.org) outlining the issue (removing these/deprecating
them) and add me to the +noisy. There you can upload patches as you
create them, and we can all coordinate work and discussion there.

For the multiprocessing support - the first step is to identify why
it's being used and remove it (I'm head-deep in pycon talk prep, so I
haven't been able to look) and then replace the code. Second, I need
to make sure the multiprocessing.queue API does not have it's own APIs
for these (mp.queue is a clone of queue.queue) and deprecate those.

-jesse

From guido at python.org  Thu Mar  5 18:32:29 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Mar 2009 09:32:29 -0800
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <gooa4o$rrh$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com> <49AE9698.1020806@simplistix.co.uk>
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
	<gooa4o$rrh$1@ger.gmane.org>
Message-ID: <ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>

On Thu, Mar 5, 2009 at 2:39 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Ivan Krsti? wrote:
>> On Mar 4, 2009, at 12:32 PM, James Y Knight wrote:
>>> I think html5lib would be a better candidate for an imrpoved HTML
>>> parser in the stdlib than BeautifulSoup.
>>
>> While we're talking about alternatives, Ian Bicking appears to swear by
>> lxml:
>>
>> <http://blog.ianbicking.org/2008/12/10/lxml-an-underappreciated-web-scraping-library/>
>
> I second that. ;)
>
> And, BTW, I wouldn't mind getting lxml into the stdlib either.

No matter how beautiful and fast lxml is, it has one downside where it
comes to installing it into the stdlib: it is based on large, complex
3rd party libraries, libxml2 and libxslt.

Based on the sad example of BerkeleyDB, which was initially welcomed
into the stdlib but more recently booted out for reasons having to do
with the release cycle of the external dependency and other issues
typical for large external dependencies, I think we should be very
careful with including it in the standard library.

Instead, let's hope Linux distros pick it up (and if anyone knows how
to encourage that, let us know).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Thu Mar  5 19:30:25 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 5 Mar 2009 13:30:25 -0500
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
References: <49ABCFC9.1070508@gmail.com> <49AE9698.1020806@simplistix.co.uk>
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
	<gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
Message-ID: <687FDB81-61F5-419B-A7F5-3F5EC8CDE68E@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 5, 2009, at 12:32 PM, Guido van Rossum wrote:

> Instead, let's hope Linux distros pick it up (and if anyone knows how
> to encourage that, let us know).


Gentoo: emerge lxml
Ubuntu (and probably Debian): apt-get install python-lxml

Guido, do you know where your time machine keys are? :)

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSbAaRHEjvBPtnXfVAQK3nQP/Uz6CF7zxIbTJWHyGyPBr1+pUTESzryNs
SKnBwcyIjYw/+7whtdfp31jbgsv+FcZ9YmMx7haUzPS/lKaRClvfUlirXepDCQt/
Z44nxvjEbbpQPmvlmf9SAIgvk7AumWcigXth2cvMJedHz0fVA9jXA1f/bnGxdTA6
/DUrqxruwZo=
=R5FW
-----END PGP SIGNATURE-----

From list8a.forest at tibit.com  Thu Mar  5 19:40:12 2009
From: list8a.forest at tibit.com (Forest)
Date: Thu, 5 Mar 2009 10:40:12 -0800 (PST)
Subject: [Python-Dev] PEP 372 -- Adding an ordered directory to
 collections ready for pronouncement
In-Reply-To: <mailman.4658.1236216354.11745.python-dev@python.org>
References: <mailman.4658.1236216354.11745.python-dev@python.org>
Message-ID: <da327e3ceb0034932ee532be6d430737.squirrel@webmail.sonic.net>

On Wed, Wed, 4 Mar 2009 13:52:59 -0800, Guido van Rossum wrote:
> On Wed, Mar 4, 2009 at 1:44 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>
> wrote:
>> rdmurray at bitdance.com wrote:
>>
>>> I actually like StableDict best. ?When I hear that I think, "ah, the
>>> key order is stable in the face of insertions, unlike a regular dict".
>>
>> But it still doesn't convey what the ordering actually *is*.
>
> Please, stick with OrderedDict. That's the name used historically by
> most people who independently reinvented this functionality.

It's also what I typed into google and PYPI when I went looking for this
functionality.

+1 for odereddict or odict or OrderedDict.



From stefan_ml at behnel.de  Thu Mar  5 20:22:50 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 05 Mar 2009 20:22:50 +0100
Subject: [Python-Dev] Integrate lxml into the stdlib? (was: Integrate
 BeautifulSoup into stdlib?)
In-Reply-To: <ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
References: <49ABCFC9.1070508@gmail.com>
	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
Message-ID: <gop8qa$csc$1@ger.gmane.org>

Hi Guido,

I'm happy to see you jump onto this.

Guido van Rossum wrote:
> No matter how beautiful and fast lxml is, it has one downside where it
> comes to installing it into the stdlib: it is based on large, complex
> 3rd party libraries, libxml2 and libxslt.

I actually had a recent discussion with other lxml developers and we were
fast to agree that that would be the main problem. lxml itself is only some
18K lines of Cython code (which translates into 180K lines of C code) and
less than 7K lines of Python code, but libxml2 and libxslt add up to about
230K lines of C code just by themselves. That is definitely far from
trivial and it's hard to guarantee that bugs in these libraries will never
lead to security holes in a Python release, for example.

Still, it does provide an awful lot of things that the stdlib currently
fails to deliver in one way or another, some even completely. XPath, XSLT,
XML validation and (above all) real-world HTML parsing come to mind. I
definitely stopped counting the posts on c.l.py about HTMLParser not being
able to parse a specific web page.

It's good that (c)ElementTree is part of the stdlib, and it's also good
that there is a rather smooth upgrade path towards lxml. But lxml is by
itself becoming more and more a critical dependency of web related packages
and applications, simply because it provides everything in one tool. And
even if I wasn't the author of lxml, I would have a hard time feeling happy
if a real-world HTML parser was added to the stdlib that provides a totally
different interface than the best (and fastest) XML library that the stdlib
currently has.


> Instead, let's hope Linux distros pick it up (and if anyone knows how
> to encourage that, let us know).

At least all Debian based distros (such as Ubuntu) have it available. Not
the latest, greatest version, but that will come. That said, it's never
been a real problem to EasyInstall lxml directly from PyPI onto any decent
Linux distribution. MacOS-X is a far more tricky target here, not to say a
word about Windows (C-compiler? anyone?).

I would expect that even if lxml itself was in the stdlib, Linux
distributions would (want to) build it against their system libraries.
Static builds would only be required on MacOS-X and Windows.

Stefan


From steve at holdenweb.com  Thu Mar  5 20:27:16 2009
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 05 Mar 2009 14:27:16 -0500
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
In-Reply-To: <4222a8490903050925qc23a528u820207e9658410d7@mail.gmail.com>
References: <43c8685c0903041739m46f11513yef9c1e923097d77d@mail.gmail.com>	<000e0cd47bb2710f5e04645544f5@google.com>	<43c8685c0903042243t8955312x8abdace628de3eb7@mail.gmail.com>
	<4222a8490903050925qc23a528u820207e9658410d7@mail.gmail.com>
Message-ID: <gop92k$dbp$1@ger.gmane.org>

Jesse Noller wrote:
[...]I'll take it from anyone.
> 
And we can *quote* you on that?

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From nas at arctrix.com  Thu Mar  5 20:30:52 2009
From: nas at arctrix.com (Neil Schemenauer)
Date: Thu, 5 Mar 2009 19:30:52 +0000 (UTC)
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com>
Message-ID: <gop99c$et3$1@ger.gmane.org>

Chris McDonough <chrism at plope.com> wrote:
> As far as I can tell, asyncore/asynchat is all "undocumented
> internals".  Any use of asyncore in anger will use internals;
> there never was any well-understood API to these modules.

What I would like to see is a module that provides a low-level API
for doing cross-platform asynchronous IO.  The two necessary parts
are:

    * a wrapper that allows non-blocking reads and writes on
      channels (sockets, file descriptors, serial ports, etc) 

    * a select() or epoll like interface that allows waiting on
      multiple channels

The implementation requires some intricate and platform specific
code which is why it would be nice to be a standard library feature.

I'm sure that Twisted has the necessary parts but the problem, IMHO,
is that it does so much more else.

  Neil


From guido at python.org  Thu Mar  5 20:35:39 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Mar 2009 11:35:39 -0800
Subject: [Python-Dev] Integrate lxml into the stdlib? (was: Integrate
	BeautifulSoup into stdlib?)
In-Reply-To: <gop8qa$csc$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com> <49AE9698.1020806@simplistix.co.uk>
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
	<gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<gop8qa$csc$1@ger.gmane.org>
Message-ID: <ca471dc20903051135j35ef8bdax30be5f85c09f0ed4@mail.gmail.com>

On Thu, Mar 5, 2009 at 11:22 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> I'm happy to see you jump onto this.

I'm not sure why you say that -- all I am doing is advising *against* inclusion.

> Guido van Rossum wrote:
>> No matter how beautiful and fast lxml is, it has one downside where it
>> comes to installing it into the stdlib: it is based on large, complex
>> 3rd party libraries, libxml2 and libxslt.
>
> I actually had a recent discussion with other lxml developers and we were
> fast to agree that that would be the main problem. lxml itself is only some
> 18K lines of Cython code (which translates into 180K lines of C code) and
> less than 7K lines of Python code, but libxml2 and libxslt add up to about
> 230K lines of C code just by themselves. That is definitely far from
> trivial and it's hard to guarantee that bugs in these libraries will never
> lead to security holes in a Python release, for example.
>
> Still, it does provide an awful lot of things that the stdlib currently
> fails to deliver in one way or another, some even completely. XPath, XSLT,
> XML validation and (above all) real-world HTML parsing come to mind. I
> definitely stopped counting the posts on c.l.py about HTMLParser not being
> able to parse a specific web page.

There's *waaaay* too much stuff in the XML world to ever hope to have
comprehensive support in the stdlib. Heck, XmlPlus hasn't even been
incorporated into the stdlib.

> It's good that (c)ElementTree is part of the stdlib, and it's also good
> that there is a rather smooth upgrade path towards lxml.

And yet it worries me that lxml claims to be "mostly compatible" with
ElementTree. What's keeping it from being completely (backwards)
compatible?

> But lxml is by
> itself becoming more and more a critical dependency of web related packages
> and applications, simply because it provides everything in one tool.

That depends on how XML-centric your thinking is. Personally I *don't*
like putting everything in XML, and so far I have been able to keep my
code 99% XML-free.

> And
> even if I wasn't the author of lxml, I would have a hard time feeling happy
> if a real-world HTML parser was added to the stdlib that provides a totally
> different interface than the best (and fastest) XML library that the stdlib
> currently has.

That sounds like a completely different argument and one you should
have with the proponents of inclusion of that other parser. I can only
assume you're talking about html5lib or BeautifulSoup. I have no
knowledge of any of these, and prefer to stay out of that discussion.

>> Instead, let's hope Linux distros pick it up (and if anyone knows how
>> to encourage that, let us know).
>
> At least all Debian based distros (such as Ubuntu) have it available. Not
> the latest, greatest version, but that will come. That said, it's never
> been a real problem to EasyInstall lxml directly from PyPI onto any decent
> Linux distribution. MacOS-X is a far more tricky target here, not to say a
> word about Windows (C-compiler? anyone?).
>
> I would expect that even if lxml itself was in the stdlib, Linux
> distributions would (want to) build it against their system libraries.
> Static builds would only be required on MacOS-X and Windows.

And that in itself is one of the main arguments against inclusion in
the stdlib, since it adds a whole new level of complexity to the
compatibility matrix. E.g. assume that some newer version of libxml2
has a new feature. You can wrap that feature with an API in lxml, but
now you require that newer libxml2 version as a dependency. Since the
distros don't support that they either are prevented from providing
the corresponding newer version of Python or you will have to make the
lxml code conditional on the presence or absence of that API. The
latter is preferable, but now it means that Python users can't rely on
that API being present even if they have the right version of Python.
It's a mess. Requiring a 3rd party download makes this cleaner,
because you decouple the llibxml2/lxml versioning from the Python
version.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From daniel at stutzbachenterprises.com  Thu Mar  5 20:43:52 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Thu, 5 Mar 2009 13:43:52 -0600
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <gop99c$et3$1@ger.gmane.org>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
Message-ID: <eae285400903051143q416b6ab0sce0f1a92a3039cad@mail.gmail.com>

On Thu, Mar 5, 2009 at 1:30 PM, Neil Schemenauer <nas at arctrix.com> wrote:

> What I would like to see is a module that provides a low-level API
> for doing cross-platform asynchronous IO.  The two necessary parts
> are:
>
>    * a wrapper that allows non-blocking reads and writes on
>      channels (sockets, file descriptors, serial ports, etc)
>
>    * a select() or epoll like interface that allows waiting on
>      multiple channels
>

Two thoughts:

If you have a working select(), it will tell you the sockets on which read()
and write() won't block, so non-blocking reads and writes are not necessary.

On Windows, sockets, pipes, and files are each completely distinct types
with their own functions and unifying them under one select()-like interface
requires faking it using threads behind-the-scenes AFAIK.  Personally, I'd
be happy with continuing to only support socket objects on Windows.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090305/8272c184/attachment.htm>

From jnoller at gmail.com  Thu Mar  5 20:50:00 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Thu, 5 Mar 2009 14:50:00 -0500
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
In-Reply-To: <gop92k$dbp$1@ger.gmane.org>
References: <43c8685c0903041739m46f11513yef9c1e923097d77d@mail.gmail.com>
	<000e0cd47bb2710f5e04645544f5@google.com>
	<43c8685c0903042243t8955312x8abdace628de3eb7@mail.gmail.com>
	<4222a8490903050925qc23a528u820207e9658410d7@mail.gmail.com>
	<gop92k$dbp$1@ger.gmane.org>
Message-ID: <4222a8490903051150h3f796b4aw9c5de62d08284200@mail.gmail.com>

On Thu, Mar 5, 2009 at 2:27 PM, Steve Holden <steve at holdenweb.com> wrote:
> Jesse Noller wrote:
> [...]I'll take it from anyone.
>>
> And we can *quote* you on that?
>
> regards
> ?Steve

As long as it's not on a t-shirt, I should be OK.

From collinw at gmail.com  Thu Mar  5 21:07:11 2009
From: collinw at gmail.com (Collin Winter)
Date: Thu, 5 Mar 2009 12:07:11 -0800
Subject: [Python-Dev] Pickler/Unpickler API clarification
Message-ID: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>

I'm working on some performance patches for cPickle, and one of the
bigger wins so far has been replacing the Pickler's memo dict with a
custom hashtable (and hence removing memo's getters and setters). In
looking over this, Jeffrey Yasskin commented that this would break
anyone who was accessing the memo attribute.

I've found a few examples of code using the memo attribute ([1], [2],
[3]), and there are probably more out there, but the memo attribute
doesn't look like part of the API to me. It's only documented in
http://docs.python.org/library/pickle.html as "you used to need this
before Python 2.3, but don't anymore". However: I don't believe you
should ever need this attribute.

The usages of memo I've seen break down into two camps: clearing the
memo, and wanting to explicitly populate the memo with predefined
values. Clearing the memo is recommended as part of reusing Pickler
objects, but I can't fathom when you would want to reuse a Pickler
*without* clearing the memo. Reusing the Pickler without clearing the
memo will produce pickles that are, as best I can see, invalid -- at
least, pickletools.dis() rejects this, which is the closest thing we
have to a validator. Explicitly setting memo values has the same
problem: an easy, very brittle way to produce invalid data.

So my questions are these:
1) Should Pickler/Unpickler objects automatically clear their memos
when dumping/loading?
2) Is memo an intentionally exposed, supported part of the
Pickler/Unpickler API, despite the lack of documentation and tests?

Thanks,
Collin

[1] - http://google.com/codesearch/p?hl=en#Qx8E-7HUBTk/trunk/google/appengine/api/memcache/__init__.py&q=lang:py%20%5C.memo
[2] - http://google.com/codesearch/p?hl=en#M-DDI-lCOgE/lib/python2.4/site-packages/cvs2svn_lib/primed_pickle.py&q=lang:py%20%5C.memo
[3] - http://google.com/codesearch/p?hl=en#l_w_cA4dKMY/AtlasAnalysis/2.0.3-LST-1/PhysicsAnalysis/PyAnalysis/PyAnalysisUtils/python/root_pickle.py&q=lang:py%20pick.*%5C.memo%5Cb

From benjamin at python.org  Thu Mar  5 21:10:07 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 5 Mar 2009 15:10:07 -0500
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
References: <49ABCFC9.1070508@gmail.com> <49AE9698.1020806@simplistix.co.uk>
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
	<gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
Message-ID: <1afaf6160903051210x7dc45b07s60620261468f1c31@mail.gmail.com>

2009/3/5 Guido van Rossum <guido at python.org>:
> On Thu, Mar 5, 2009 at 2:39 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>> And, BTW, I wouldn't mind getting lxml into the stdlib either.
>
> No matter how beautiful and fast lxml is, it has one downside where it
> comes to installing it into the stdlib: it is based on large, complex
> 3rd party libraries, libxml2 and libxslt.

And it depends on Cython, which is wonderful normally, but maybe
difficult to deal with in language evolution since we wouldn't have
direct control over the C sources.


-- 
Regards,
Benjamin

From stefan_ml at behnel.de  Thu Mar  5 21:34:41 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 05 Mar 2009 21:34:41 +0100
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <ca471dc20903051135j35ef8bdax30be5f85c09f0ed4@mail.gmail.com>
References: <49ABCFC9.1070508@gmail.com>
	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<gop8qa$csc$1@ger.gmane.org>
	<ca471dc20903051135j35ef8bdax30be5f85c09f0ed4@mail.gmail.com>
Message-ID: <gopd12$t0k$1@ger.gmane.org>

Guido van Rossum wrote:
> On Thu, Mar 5, 2009 at 11:22 AM, Stefan Behnel wrote:
>> I'm happy to see you jump onto this.
> 
> I'm not sure why you say that -- all I am doing is advising *against* inclusion.

I understand that. What worth is a discussion where everyone just nods for
good? :)


>> Guido van Rossum wrote:
> There's *waaaay* too much stuff in the XML world to ever hope to have
> comprehensive support in the stdlib.

Definitely. But lxml was born because some Dutch guy thought that there was
way too little easy-to-master XML support in the overall Python world.

http://codespeak.net/lxml/intro.html

There is some space to look for a trade-off here.


>> It's good that (c)ElementTree is part of the stdlib, and it's also good
>> that there is a rather smooth upgrade path towards lxml.
> 
> And yet it worries me that lxml claims to be "mostly compatible" with
> ElementTree. What's keeping it from being completely (backwards)
> compatible?

The underlying tree model. An Element in lxml.etree knows it's parent,
which isn't the case in ET. That's the main difference. Most people call
that a feature in lxml, but it's fundamental and it does have the
implication that you can't keep the same Element object in more than one place.

Some other (minor) differences are described here:

http://codespeak.net/lxml/dev/compatibility.html


>> But lxml is by
>> itself becoming more and more a critical dependency of web related packages
>> and applications, simply because it provides everything in one tool.
> 
> That depends on how XML-centric your thinking is. Personally I *don't*
> like putting everything in XML, and so far I have been able to keep my
> code 99% XML-free.

That's totally fine. I used Python for years without ever feeling the need
to deploy any of the dbm databases in my projects. Nor curses, nor tk. And
lxml.objectify only supports pickle because one of the developers thought
it was a good idea to pickle trees. And yet all of these modules are part
of the stdlib, and I bet there are a whole lot of applications by now that
wouldn't work without them.


>> I would expect that even if lxml itself was in the stdlib, Linux
>> distributions would (want to) build it against their system libraries.
>> Static builds would only be required on MacOS-X and Windows.
> 
> And that in itself is one of the main arguments against inclusion in
> the stdlib, since it adds a whole new level of complexity to the
> compatibility matrix. E.g. assume that some newer version of libxml2
> has a new feature.

That happens. So far, I have managed to keep lxml backwards compatible over
more than three years of libxml2 releases. However:

> You can wrap that feature with an API in lxml, but
> now you require that newer libxml2 version as a dependency. Since the
> distros don't support that they either are prevented from providing
> the corresponding newer version of Python or you will have to make the
> lxml code conditional on the presence or absence of that API. The
> latter is preferable, but now it means that Python users can't rely on
> that API being present even if they have the right version of Python.
> It's a mess. Requiring a 3rd party download makes this cleaner,
> because you decouple the llibxml2/lxml versioning from the Python
> version.

A good example is actually (once again) parsing broken HTML. libxml2
handles this a lot better since 2.6.21, so if you use 2.6.20, you will
simply not get the same results as with a later version.

I do see the point you are making here. Even if lxml gets mature and
static, that doesn't necessarily apply to the external libraries it uses.
However, I should note that exactly the same argument also applies to
sqlite3 and gdbm, which, again, are in the stdlib today, with sqlite3 being
a fairly recent addition.

Stefan


From greg.ewing at canterbury.ac.nz  Thu Mar  5 21:39:58 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Mar 2009 09:39:58 +1300
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
 of medusa
In-Reply-To: <gop99c$et3$1@ger.gmane.org>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
Message-ID: <49B0389E.5070004@canterbury.ac.nz>

Neil Schemenauer wrote:

> What I would like to see is a module that provides a low-level API
> for doing cross-platform asynchronous IO.  The two necessary parts
> are:
> 
>     * a wrapper that allows non-blocking reads and writes on
>       channels (sockets, file descriptors, serial ports, etc) 
> 
>     * a select() or epoll like interface that allows waiting on
>       multiple channels

+1

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Mar  5 21:46:47 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Mar 2009 09:46:47 +1300
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
 of medusa
In-Reply-To: <eae285400903051143q416b6ab0sce0f1a92a3039cad@mail.gmail.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<eae285400903051143q416b6ab0sce0f1a92a3039cad@mail.gmail.com>
Message-ID: <49B03A37.4070705@canterbury.ac.nz>

Daniel Stutzbach wrote:

> If you have a working select(), it will tell you the sockets on which 
> read() and write() won't block, so non-blocking reads and writes are not 
> necessary.

No, but there should be an interface that lets you say
"when something comes in on this fd, call this function
for me".

In other words it should be a light wrapper around
select/poll/whatever that provides a callback interface.

-- 
Greg

From josiah.carlson at gmail.com  Thu Mar  5 21:49:47 2009
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Thu, 5 Mar 2009 12:49:47 -0800
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <49B03A37.4070705@canterbury.ac.nz>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<eae285400903051143q416b6ab0sce0f1a92a3039cad@mail.gmail.com>
	<49B03A37.4070705@canterbury.ac.nz>
Message-ID: <e6511dbf0903051249w5275518dyc1e6175984d3271b@mail.gmail.com>

On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Daniel Stutzbach wrote:
>
>> If you have a working select(), it will tell you the sockets on which
>> read() and write() won't block, so non-blocking reads and writes are not
>> necessary.
>
> No, but there should be an interface that lets you say
> "when something comes in on this fd, call this function
> for me".
>
> In other words it should be a light wrapper around
> select/poll/whatever that provides a callback interface.

A read callback, a write callback.  What about close, error, connect,
and accept callbacks?

I hate to say it (not really), but that's pretty much the handle_*()
methods of asyncore :/ .

 - Josiah

From stefan_ml at behnel.de  Thu Mar  5 21:52:02 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 05 Mar 2009 21:52:02 +0100
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <1afaf6160903051210x7dc45b07s60620261468f1c31@mail.gmail.com>
References: <49ABCFC9.1070508@gmail.com>
	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<1afaf6160903051210x7dc45b07s60620261468f1c31@mail.gmail.com>
Message-ID: <gope1i$118$1@ger.gmane.org>

Benjamin Peterson wrote:
> 2009/3/5 Guido van Rossum <guido at python.org>:
>> On Thu, Mar 5, 2009 at 2:39 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>> And, BTW, I wouldn't mind getting lxml into the stdlib either.
>> No matter how beautiful and fast lxml is, it has one downside where it
>> comes to installing it into the stdlib: it is based on large, complex
>> 3rd party libraries, libxml2 and libxslt.
> 
> And it depends on Cython, which is wonderful normally, but maybe
> difficult to deal with in language evolution since we wouldn't have
> direct control over the C sources.

I see the point, although I think that this can be dealt with by

a) using a specific, stable release version of Cython for a specific Python
release, so that this Cython version can be bug fixed if required (it's
implemented in Python, after all)

or

b) adding Cython to the stdlib and building with that

Stefan


From ajaksu at gmail.com  Thu Mar  5 21:57:24 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Thu, 5 Mar 2009 17:57:24 -0300
Subject: [Python-Dev] Tracker cleanup - Roundup hacking report :)
In-Reply-To: <0016e644cf3261684904646266b2@google.com>
References: <2d75d7660903050917n340ede0u4746bc8395e07b12@mail.gmail.com>
	<0016e644cf3261684904646266b2@google.com>
Message-ID: <2d75d7660903051257x522ad38cp202ff05b8fd7cb81@mail.gmail.com>

Jesse Noller wrote:
> Slightly off topic Daniel, but if you see any multiprocessing bugs
> lurking out there, can you make me (jnoller) the assignee?

Sure!

FWIW, I've just submitted a patch[1] that will make working with
arbitrary issue sets much saner  and should have a 'mass-add user X as
nosy to selected issues' working locally before Saturday.

When these land on bugs.python.org, this kind of task will become so
easy that my job as tracker janitorveloper might become redundant :)

Cheers,
Daniel

[1]http://psf.upfronthosting.co.za/roundup/meta/issue246

From ncoghlan at gmail.com  Thu Mar  5 22:02:02 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 06 Mar 2009 07:02:02 +1000
Subject: [Python-Dev] running the tests...
In-Reply-To: <49AFC8AA.5010103@simplistix.co.uk>
References: <49AF9CCB.2050509@simplistix.co.uk> <goo7nq$jh7$1@ger.gmane.org>
	<49AFB649.1020801@gmail.com> <49AFC8AA.5010103@simplistix.co.uk>
Message-ID: <49B03DCA.4030908@gmail.com>

Chris Withers wrote:
> Nick Coghlan wrote:
>> My personal preferences:
>>
>> Thorough: ./python -m test.regrtest -uall
>> Typical: ./python -m test.regrtest
>> Specific: ./python -m test.regrtest test_mod1 test_mod2
> 
> This looks good, I assume this would work on Windows too?

Yep - and of course, you can even leave out the ./ in that case.

(Oh, if for some reason you're working on a Python 2.4 or earlier build,
the above won't work, since -m only started supporting running modules
inside packages in 2.5)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From brett at python.org  Thu Mar  5 22:06:32 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 5 Mar 2009 13:06:32 -0800
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <gope1i$118$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com> <49AE9698.1020806@simplistix.co.uk>
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
	<gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<1afaf6160903051210x7dc45b07s60620261468f1c31@mail.gmail.com>
	<gope1i$118$1@ger.gmane.org>
Message-ID: <bbaeab100903051306va2e8293se57739c3b6f44df4@mail.gmail.com>

On Thu, Mar 5, 2009 at 12:52, Stefan Behnel <stefan_ml at behnel.de> wrote:

> Benjamin Peterson wrote:
> > 2009/3/5 Guido van Rossum <guido at python.org>:
> >> On Thu, Mar 5, 2009 at 2:39 AM, Stefan Behnel <stefan_ml at behnel.de>
> wrote:
> >>> And, BTW, I wouldn't mind getting lxml into the stdlib either.
> >> No matter how beautiful and fast lxml is, it has one downside where it
> >> comes to installing it into the stdlib: it is based on large, complex
> >> 3rd party libraries, libxml2 and libxslt.
> >
> > And it depends on Cython, which is wonderful normally, but maybe
> > difficult to deal with in language evolution since we wouldn't have
> > direct control over the C sources.
>
> I see the point, although I think that this can be dealt with by
>
> a) using a specific, stable release version of Cython for a specific Python
> release, so that this Cython version can be bug fixed if required (it's
> implemented in Python, after all)
>

So including Cython source in the stdlib and then check in the generated C
code? I don't think that adding another build dependency for the stdlib,
especially for one already with several external dependencies itself, is a
good idea.


>
> or
>
> b) adding Cython to the stdlib and building with that


That's an entirely separate discussion (for which my initial answer is to
not consider it until it has stabilized to a 1.0 release).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090305/cc8fc6ec/attachment.htm>

From janssen at parc.com  Thu Mar  5 22:09:20 2009
From: janssen at parc.com (Bill Janssen)
Date: Thu, 5 Mar 2009 13:09:20 PST
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <e6511dbf0903051249w5275518dyc1e6175984d3271b@mail.gmail.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<eae285400903051143q416b6ab0sce0f1a92a3039cad@mail.gmail.com>
	<49B03A37.4070705@canterbury.ac.nz>
	<e6511dbf0903051249w5275518dyc1e6175984d3271b@mail.gmail.com>
Message-ID: <70821.1236287360@parc.com>

Josiah Carlson <josiah.carlson at gmail.com> wrote:

> On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> > Daniel Stutzbach wrote:
> >
> >> If you have a working select(), it will tell you the sockets on which
> >> read() and write() won't block, so non-blocking reads and writes are not
> >> necessary.
> >
> > No, but there should be an interface that lets you say
> > "when something comes in on this fd, call this function
> > for me".
> >
> > In other words it should be a light wrapper around
> > select/poll/whatever that provides a callback interface.
> 
> A read callback, a write callback.  What about close, error, connect,
> and accept callbacks?
> 
> I hate to say it (not really), but that's pretty much the handle_*()
> methods of asyncore :/ .

What asyncore was missing was a timer API (a way to register functions
to be called periodically).  Then it would be pretty much like any other
event loop system.

Bill

From guido at python.org  Thu Mar  5 22:10:32 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Mar 2009 13:10:32 -0800
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
Message-ID: <ca471dc20903051310l16c89b0dvf3729f955d334462@mail.gmail.com>

On Thu, Mar 5, 2009 at 12:07 PM, Collin Winter <collinw at gmail.com> wrote:
> I'm working on some performance patches for cPickle, and one of the
> bigger wins so far has been replacing the Pickler's memo dict with a
> custom hashtable (and hence removing memo's getters and setters). In
> looking over this, Jeffrey Yasskin commented that this would break
> anyone who was accessing the memo attribute.
>
> I've found a few examples of code using the memo attribute ([1], [2],
> [3]), and there are probably more out there, but the memo attribute
> doesn't look like part of the API to me. It's only documented in
> http://docs.python.org/library/pickle.html as "you used to need this
> before Python 2.3, but don't anymore". However: I don't believe you
> should ever need this attribute.
>
> The usages of memo I've seen break down into two camps: clearing the
> memo, and wanting to explicitly populate the memo with predefined
> values. Clearing the memo is recommended as part of reusing Pickler
> objects, but I can't fathom when you would want to reuse a Pickler
> *without* clearing the memo. Reusing the Pickler without clearing the
> memo will produce pickles that are, as best I can see, invalid -- at
> least, pickletools.dis() rejects this, which is the closest thing we
> have to a validator.

I can explain this, as I invented this behavior. The use case was to
have a long-lived session between a client and a server which were
communicating repeatedly using pickles. The idea was that values that
had been transferred once wouldn't have to be sent across the wire
again -- they could just be referenced.

This was a bad idea (*), and I'd be happy to ban it -- but we'd
probably have to bump the pickle protocol version in order to maintain
backwards compatibility.

> Explicitly setting memo values has the same
> problem: an easy, very brittle way to produce invalid data.

Agreed this is just preposterous. It was never part of the plan.

> So my questions are these:
> 1) Should Pickler/Unpickler objects automatically clear their memos
> when dumping/loading?

Alas, there could be backwards compatibility issues. Bumping the
protocol could mitigate this.

> 2) Is memo an intentionally exposed, supported part of the
> Pickler/Unpickler API, despite the lack of documentation and tests?

The exposition is unintentional but for historic reasons we can't just
remove it. :-(

> Thanks,
> Collin
>
> [1] - http://google.com/codesearch/p?hl=en#Qx8E-7HUBTk/trunk/google/appengine/api/memcache/__init__.py&q=lang:py%20%5C.memo
> [2] - http://google.com/codesearch/p?hl=en#M-DDI-lCOgE/lib/python2.4/site-packages/cvs2svn_lib/primed_pickle.py&q=lang:py%20%5C.memo
> [3] - http://google.com/codesearch/p?hl=en#l_w_cA4dKMY/AtlasAnalysis/2.0.3-LST-1/PhysicsAnalysis/PyAnalysis/PyAnalysisUtils/python/root_pickle.py&q=lang:py%20pick.*%5C.memo%5Cb

__________
(*) http://code.google.com/p/googleappengine/issues/detail?id=417

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Thu Mar  5 22:17:54 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Mar 2009 10:17:54 +1300
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
Message-ID: <49B04182.7050503@canterbury.ac.nz>

Collin Winter wrote:

> Reusing the Pickler without clearing the
> memo will produce pickles that are, as best I can see, invalid

I'm not sure what you mean by "reusing the pickler" here,
and how it can produce an invalid pickle.

I think what the docs mean by it is continuing to pickle
objects to the same file, but in a logically separate
block that doesn't share any references with the previous
one, e.g.

    pickle obj1
    pickle obj2
    ---clear memo---
    pickle obj3

The whole thing is still a valid pickle containing 3 objects,
whether the memo is cleared at any point or not, and can
be unpickled using 3 corresponding unpickle calls to a
single unpickler.

> 1) Should Pickler/Unpickler objects automatically clear their memos
> when dumping/loading?

If you mean should every call to Pickler.dump() or
Unpickler.load() clear the memo first, definitely *NOT*.
It's explicitly part of the specification that you can
make multiple calls to dump() to build up a single pickle
that shares state, as long as you unpickle it using a
corresponding number of load() calls.

> 2) Is memo an intentionally exposed, supported part of the
> Pickler/Unpickler API, despite the lack of documentation and tests?

I think the 2.4 and later docs make it clear that it's
no longer considered part of the public API, if it ever
was.

If seeding the memo is considered a legitimate need, an
API could be provided for doing that.

-- 
Greg

From guido at python.org  Thu Mar  5 22:16:08 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Mar 2009 13:16:08 -0800
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <gopd12$t0k$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com> <49AE9698.1020806@simplistix.co.uk>
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
	<gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<gop8qa$csc$1@ger.gmane.org>
	<ca471dc20903051135j35ef8bdax30be5f85c09f0ed4@mail.gmail.com>
	<gopd12$t0k$1@ger.gmane.org>
Message-ID: <ca471dc20903051316u3a107616wa7adb61597ffdf1@mail.gmail.com>

Stefan,

I recommend that you give up pushing for lxml in the stdlib. There are
many complex factors to be weighed but in the balance I am not
comfortable with it, and continued argumentation is not going to
change that.

Sorry,

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Thu Mar  5 22:23:20 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Mar 2009 10:23:20 +1300
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
 of medusa
In-Reply-To: <e6511dbf0903051249w5275518dyc1e6175984d3271b@mail.gmail.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<eae285400903051143q416b6ab0sce0f1a92a3039cad@mail.gmail.com>
	<49B03A37.4070705@canterbury.ac.nz>
	<e6511dbf0903051249w5275518dyc1e6175984d3271b@mail.gmail.com>
Message-ID: <49B042C8.1070803@canterbury.ac.nz>

Josiah Carlson wrote:

> A read callback, a write callback.  What about close, error, connect,
> and accept callbacks?

Yep, all those as well.

> I hate to say it (not really), but that's pretty much the handle_*()
> methods of asyncore :/ .

Well, then, what's the problem?

Is there anything else people want that asyncore doesn't
provide, or is it just a matter of nailing down the existing
API a little?

-- 
Greg

From stefan_ml at behnel.de  Thu Mar  5 22:27:51 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 05 Mar 2009 22:27:51 +0100
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <bbaeab100903051306va2e8293se57739c3b6f44df4@mail.gmail.com>
References: <49ABCFC9.1070508@gmail.com>
	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<1afaf6160903051210x7dc45b07s60620261468f1c31@mail.gmail.com>	<gope1i$118$1@ger.gmane.org>
	<bbaeab100903051306va2e8293se57739c3b6f44df4@mail.gmail.com>
Message-ID: <gopg4n$8me$1@ger.gmane.org>

Hi,

Brett Cannon wrote:
> On Thu, Mar 5, 2009 at 12:52, Stefan Behnel wrote:
>> Benjamin Peterson wrote:
>>> it depends on Cython, which is wonderful normally, but maybe
>>> difficult to deal with in language evolution since we wouldn't have
>>> direct control over the C sources.
>> I see the point, although I think that this can be dealt with by
>>
>> a) using a specific, stable release version of Cython for a specific Python
>> release, so that this Cython version can be bug fixed if required (it's
>> implemented in Python, after all)
> 
> So including Cython source in the stdlib and then check in the generated C
> code?

Did I give the impression that a) was my preferred solution? ;)


>> b) adding Cython to the stdlib and building with that
> 
> That's an entirely separate discussion (for which my initial answer is to
> not consider it until it has stabilized to a 1.0 release).

Yes, that *is* an entirely separate discussion - for which my initial
answer is to consider it as soon as it is in a state where the compiler is
good enough to be useful and the language it compiles is stable enough to
be future proof. The language is almost Python, and the core syntax
extensions (compared to Python 2.6/3.0) haven't changed for a couple of
releases (except for the buffer syntax, which I personally don't consider
core but really nice to have).

The official goal for a 1.0 release is to compile Python programs, BTW. I
don't think the stdlib needs to wait for that.

Stefan


From greg.ewing at canterbury.ac.nz  Thu Mar  5 22:36:35 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Mar 2009 10:36:35 +1300
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <ca471dc20903051310l16c89b0dvf3729f955d334462@mail.gmail.com>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
	<ca471dc20903051310l16c89b0dvf3729f955d334462@mail.gmail.com>
Message-ID: <49B045E3.3030607@canterbury.ac.nz>

Guido van Rossum wrote:

> This was a bad idea (*), and I'd be happy to ban it -- but we'd
> probably have to bump the pickle protocol version in order to maintain
> backwards compatibility.

If you're talking about multiple calls to dump() on the same
pickler, it might be a bad idea for a network connection, but
I don't see anything wrong with using it on a file, and I find
it useful to do so sometimes. Banning it would be excessive, IMO.

> The exposition is unintentional but for historic reasons we can't just
> remove it. :-(

A compromise might be to provide a memo attribute that returns
a wrapper around the underlying cache -- maybe with only a
clear() method if that's all you want to support.

-- 
Greg

From martin at v.loewis.de  Thu Mar  5 22:45:21 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 05 Mar 2009 22:45:21 +0100
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <e45086040903050553m44f7bce7xca474d658e0c977f@mail.gmail.com>
References: <mailman.29.1235991618.1178.python-dev@python.org>	
	<p06240803c5d1f82dbc4b@10.0.1.221> <49AC5C0B.4070806@v.loewis.de>
	<e45086040903050553m44f7bce7xca474d658e0c977f@mail.gmail.com>
Message-ID: <49B047F1.9080900@v.loewis.de>

> So what is the solution?

In the specific case, I don't know. I recall that somebody offered to
pick up the change. I really didn't mean to suggest that the patch
will remain unnoticed - it was just a warning that it *might* remain
unnoticed.

The more general issue is that of patches being unreviewed for a long
time, whether they have educational background or some other background
(say, cross-compilation, HP-UX support, etc).

>From time to time, people ask what they can do push a change into Python
that they really think is important. I once offered that people who
want a patch in Python really badly should review 10 other patches in
return, up to the point where they make a recommendation about the fate
of the patches. I was then talked into accepting just 5 such patches.
I have since withdrawn this offer, because
a) I was the only one making that offer in public, and
b) I was sometimes not really able to respond in a timely manner
   when the offer was invoked, because of overload.

So, for the more general issue, I don't have a solution, either.

Regards,
Martin

From josiah.carlson at gmail.com  Thu Mar  5 22:48:02 2009
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Thu, 5 Mar 2009 13:48:02 -0800
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <70821.1236287360@parc.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<eae285400903051143q416b6ab0sce0f1a92a3039cad@mail.gmail.com>
	<49B03A37.4070705@canterbury.ac.nz>
	<e6511dbf0903051249w5275518dyc1e6175984d3271b@mail.gmail.com>
	<70821.1236287360@parc.com>
Message-ID: <e6511dbf0903051348k6a9062d5nba17d3fd2586fb06@mail.gmail.com>

On Thu, Mar 5, 2009 at 1:09 PM, Bill Janssen <janssen at parc.com> wrote:
> Josiah Carlson <josiah.carlson at gmail.com> wrote:
>
>> On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> > Daniel Stutzbach wrote:
>> >
>> >> If you have a working select(), it will tell you the sockets on which
>> >> read() and write() won't block, so non-blocking reads and writes are not
>> >> necessary.
>> >
>> > No, but there should be an interface that lets you say
>> > "when something comes in on this fd, call this function
>> > for me".
>> >
>> > In other words it should be a light wrapper around
>> > select/poll/whatever that provides a callback interface.
>>
>> A read callback, a write callback. ?What about close, error, connect,
>> and accept callbacks?
>>
>> I hate to say it (not really), but that's pretty much the handle_*()
>> methods of asyncore :/ .
>
> What asyncore was missing was a timer API (a way to register functions
> to be called periodically). ?Then it would be pretty much like any other
> event loop system.

There are two variants of patches to offer timer API functionality in
the bug tracker right now.  One from Giampaolo that uses a variant of
Twisted's scheduler, one from me that uses an updated sched.py .
Giampaolo's is more complete (it has documentation and tests), but
mine is more efficient with nontrivial task lists.

 - Josiah

From guido at python.org  Thu Mar  5 22:54:13 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 5 Mar 2009 13:54:13 -0800
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <49B045E3.3030607@canterbury.ac.nz>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
	<ca471dc20903051310l16c89b0dvf3729f955d334462@mail.gmail.com>
	<49B045E3.3030607@canterbury.ac.nz>
Message-ID: <ca471dc20903051354o25d80c2cr4b8ca95211bc7103@mail.gmail.com>

On Thu, Mar 5, 2009 at 1:36 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>
>> This was a bad idea (*), and I'd be happy to ban it -- but we'd
>> probably have to bump the pickle protocol version in order to maintain
>> backwards compatibility.
>
> If you're talking about multiple calls to dump() on the same
> pickler, it might be a bad idea for a network connection, but
> I don't see anything wrong with using it on a file, and I find
> it useful to do so sometimes. Banning it would be excessive, IMO.

I don't think I was thinking of that when I first designed pickle but
the use case makes some sense. I still wish we could ban it or somehow
make it *not* the default behavior; the bug in the App Engine bug I
referenced before was introduced by an experienced developer who
wasn't aware of this behavior and was simply trying to avoid
unnecessarily creating a new pickler for each call.

>> The exposition is unintentional but for historic reasons we can't just
>> remove it. :-(
>
> A compromise might be to provide a memo attribute that returns
> a wrapper around the underlying cache -- maybe with only a
> clear() method if that's all you want to support.

Then it'd be better to have a method clear_memo() on pickle objects.

Perhaps we should do the opposite, and have a separate API for reuse
*without* clearing the memo? <pickler>.dump_reusing_memo(<value>) and
<unpickler>.load_reusing_memo().

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From solipsis at pitrou.net  Thu Mar  5 23:43:06 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 5 Mar 2009 22:43:06 +0000 (UTC)
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<49B0389E.5070004@canterbury.ac.nz>
Message-ID: <loom.20090305T224150-476@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> 
> Neil Schemenauer wrote:
> 
> > What I would like to see is a module that provides a low-level API
> > for doing cross-platform asynchronous IO.  The two necessary parts
> > are:
> > 
> >     * a wrapper that allows non-blocking reads and writes on
> >       channels (sockets, file descriptors, serial ports, etc) 

For starters, since py3k is supposed to support non-blocking IO, why not write a
portable API to make a raw file or socket IO object non-blocking?
(I'm only suggesting it here, I don't intend to do this myself)



From glyph at divmod.com  Fri Mar  6 00:11:49 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Thu, 05 Mar 2009 23:11:49 -0000
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of	medusa
In-Reply-To: <gop99c$et3$1@ger.gmane.org>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
Message-ID: <20090305231149.12555.939089272.divmod.xquotient.5373@weber.divmod.com>


On 07:30 pm, nas at arctrix.com wrote:
>Chris McDonough <chrism at plope.com> wrote:
>>As far as I can tell, asyncore/asynchat is all "undocumented
>>internals".  Any use of asyncore in anger will use internals;
>>there never was any well-understood API to these modules.

>The implementation requires some intricate and platform specific
>code which is why it would be nice to be a standard library feature.
>
>I'm sure that Twisted has the necessary parts but the problem, IMHO,
>is that it does so much more else.

... which is exactly why I have volunteered to explain to someone how to 
separate the core event-loop bits (suitable for inclusion in the 
standard library) from the huge pile of protocol implementations which 
are not necessarily useful.

Despite the FUD to the contrary, Twisted's internal factoring is quite 
good; it's not a single, undifferentiated pile of code.  Plus, at this 
point, we're not even talking about actually putting any Twisted code 
into the standard library, just standardizing the "protocol" API, which 
basically boils down to:

  connectionMade() -> your connection has begun
  dataReceived(data) -> you got some bytes, handle them
  connectionLost(reason) -> your connection has gone away (with an object 
explaining why)

and the inverse, "transport", which is:

  write(data) -> deliver some data to the dataReceived on the other end 
of this connection (non-blocking, with buffering)
  loseConnection() -> goodbye

There are a few other minor details related to how you set these up to 
talk to each other and tell when the out-buffer is empty, but it's all 
pretty straightforward.  The main point is that you don't ever call 
recv() or send() and deal with buffering or handling weird errno values. 
For example, if your connection goes away, the notification you get is 
"your connection went away", not "oops you tried to read some bytes, but 
your connection was gone by the time you tried, even though I just told 
you it was ready for reading" or other similarly obtuse failure modes.

From josiah.carlson at gmail.com  Fri Mar  6 00:09:27 2009
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Thu, 5 Mar 2009 15:09:27 -0800
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <20090305231149.12555.939089272.divmod.xquotient.5373@weber.divmod.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<20090305231149.12555.939089272.divmod.xquotient.5373@weber.divmod.com>
Message-ID: <e6511dbf0903051509x15c43d30ve710aaaf5b9a4c6b@mail.gmail.com>

On Thu, Mar 5, 2009 at 3:11 PM,  <glyph at divmod.com> wrote:
>
> On 07:30 pm, nas at arctrix.com wrote:
>>
>> Chris McDonough <chrism at plope.com> wrote:
>>>
>>> As far as I can tell, asyncore/asynchat is all "undocumented
>>> internals". ?Any use of asyncore in anger will use internals;
>>> there never was any well-understood API to these modules.
>
>> The implementation requires some intricate and platform specific
>> code which is why it would be nice to be a standard library feature.
>>
>> I'm sure that Twisted has the necessary parts but the problem, IMHO,
>> is that it does so much more else.
>
> ... which is exactly why I have volunteered to explain to someone how to
> separate the core event-loop bits (suitable for inclusion in the standard
> library) from the huge pile of protocol implementations which are not
> necessarily useful.
>
> Despite the FUD to the contrary, Twisted's internal factoring is quite good;
> it's not a single, undifferentiated pile of code. ?Plus, at this point,
> we're not even talking about actually putting any Twisted code into the
> standard library, just standardizing the "protocol" API, which basically
> boils down to:
>
> ?connectionMade() -> your connection has begun
> ?dataReceived(data) -> you got some bytes, handle them
> ?connectionLost(reason) -> your connection has gone away (with an object
> explaining why)
>
> and the inverse, "transport", which is:
>
> ?write(data) -> deliver some data to the dataReceived on the other end of
> this connection (non-blocking, with buffering)
> ?loseConnection() -> goodbye
>
> There are a few other minor details related to how you set these up to talk
> to each other and tell when the out-buffer is empty, but it's all pretty
> straightforward. ?The main point is that you don't ever call recv() or
> send() and deal with buffering or handling weird errno values. For example,
> if your connection goes away, the notification you get is "your connection
> went away", not "oops you tried to read some bytes, but your connection was
> gone by the time you tried, even though I just told you it was ready for
> reading" or other similarly obtuse failure modes.

Those "obtuse failure modes" are handled fairly well by asynchat.  It
works pretty well, and includes sample implementations to get you 99%
of the way towards the 5-method API you describe (which is why a lot
of people use asynchat instead of asyncore).  If it weren't for the
fact that asynchat had a previously existing API (including the
.push(), collect_incoming_data(), and found_terminator()), I think it
could stand for methods to make it fit better with asyncore's
handle_*() methods.

 - Josiah

From martin at v.loewis.de  Fri Mar  6 00:18:56 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 06 Mar 2009 00:18:56 +0100
Subject: [Python-Dev] patch commit policies (was [issue4308] repr of
 httplib.IncompleteRead is stupid)
In-Reply-To: <49AF9AD3.2090405@simplistix.co.uk>
References: <49AC394E.2000903@v.loewis.de> <49AF9AD3.2090405@simplistix.co.uk>
Message-ID: <49B05DE0.4020804@v.loewis.de>

> That aside, is it actually a python-wide policy to *forbid* patching
> older releases where the patch isn't security-related?

I set this policy for the releases I manage, namely 2.4 and 2.5.

I still plan to write a PEP on security releases, and how they relate
to maintenance releases.

> I can understand the "no more releases unless there are security
> problems", but what's the harm in applying a patch to an old version
> branch on the off chance that a security release might be made some time?

Yes. *Every* change causes the risk of breaking something. In fact, for
any non-doc change, it is possible to construct a program that breaks
under the change.

The longer a release is in production use, the less breakage can be
risked. People will have worked around all regular bugs that they may
have run into. So when they ever make the experience that installing
a security fix actually breaks their working code, they will refrain
from ever installing Python patches again.

Regards,
Martin

From martin at v.loewis.de  Fri Mar  6 00:24:51 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Mar 2009 00:24:51 +0100
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <gopd12$t0k$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<gop8qa$csc$1@ger.gmane.org>	<ca471dc20903051135j35ef8bdax30be5f85c09f0ed4@mail.gmail.com>
	<gopd12$t0k$1@ger.gmane.org>
Message-ID: <49B05F43.5030706@v.loewis.de>

> I do see the point you are making here. Even if lxml gets mature and
> static, that doesn't necessarily apply to the external libraries it uses.
> However, I should note that exactly the same argument also applies to
> sqlite3 and gdbm, which, again, are in the stdlib today, with sqlite3 being
> a fairly recent addition.

Fortunately, it is possible for users to just replace the sqlite DLL in
a Python installation, with no need of recompiling anything.

Regards,
Martin

From barry at python.org  Fri Mar  6 00:28:36 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 5 Mar 2009 18:28:36 -0500
Subject: [Python-Dev] patch commit policies (was [issue4308] repr of
	httplib.IncompleteRead is stupid)
In-Reply-To: <49B05DE0.4020804@v.loewis.de>
References: <49AC394E.2000903@v.loewis.de> <49AF9AD3.2090405@simplistix.co.uk>
	<49B05DE0.4020804@v.loewis.de>
Message-ID: <1660610E-9A4C-459E-B0E7-F31150DF38A1@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 5, 2009, at 6:18 PM, Martin v. L?wis wrote:

>> That aside, is it actually a python-wide policy to *forbid* patching
>> older releases where the patch isn't security-related?
>
> I set this policy for the releases I manage, namely 2.4 and 2.5.

This is a Python-wide policy.

When Python 2.7 is released, there will be one last 2.6.x bug fix  
release, and then it will go into security-only mode.  Similarly, when  
Python 3.1 is released, there will be one last 3.0.x release and it  
too will go into security-only mode.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSbBgJHEjvBPtnXfVAQIEbQP6Ah9I4SikMQ++vlSYb4BdNgw0VXF/8PMH
aTkG0+haOBoJ+mJ9G5GbBiJVtnV0B6qRV1gSV5FSIOzbQE/t0zU27APwnIWv57l9
uZIyyjU1AA0Gz5DSOsC9xtoAqEu/iFH9aRd17tE/N2Yib5p62myqAUVHIRPBl0fD
sh+ztoL3q2c=
=vgz0
-----END PGP SIGNATURE-----

From jimjjewett at gmail.com  Fri Mar  6 03:51:38 2009
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 5 Mar 2009 21:51:38 -0500
Subject: [Python-Dev] html5lib/BeautifulSoup (was: Integrate lxml into the
	stdlib? (was: Integrate BeautifulSoup into stdlib?))
Message-ID: <fb6fbf560903051851v75d7e54cg969dd173aa1130e0@mail.gmail.com>

Stefan Behnel wrote:

> I would have a hard time feeling happy
> if a real-world HTML parser was added to the stdlib that provides a totally
> different interface than the best (and fastest) XML library that the stdlib
> currently has.

I doubt there would be any objection to someone contributing wrappers
for upgrades, but I wouldn't count on them being used.

lxml may well be the best choice for xml.

BeautifulSoup and html5lib wouldn't even exist if that actually
mattered for most of *their* use cases.  Think of them more as
pre-processors, like tidylib.  If enough web pages were even valid
HTML (let alone valid and well-formed XML), no one would have bothered
to write these libraries.

BeautifulSoup has the advantage of being long-proven in practice, for
ugly html.  (You mention an lxml feature with a similar intent, but
for lxml, it is one of several addon features; for BeautifulSoup, this
is the whole point.)

html5lib does not have as long of a history, but it does have the
advantage of being almost an endorsed standard.  Much of HTML 5 is
documenting the workarounds that browser makers already actually
employ to handle erroneous input, so that the complexities can at
least stop compounding.  html5lib is intended as a reference
implementation, and the w3c editor has used it to motivate changes in
the specification draft.  (This may make it unsuitable for inclusion
in the stdlib today, because of timing issues.)  In other words, it
isn't just the heuristics of one particular development team; it is
(modulo bugs, and after official publication) the heuristics that the
major web browser makers have agreed to treat as "correct" in the
future.

-jJ

From tseaver at palladion.com  Fri Mar  6 04:07:45 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 05 Mar 2009 22:07:45 -0500
Subject: [Python-Dev] patch commit policies (was [issue4308] repr of
 httplib.IncompleteRead is stupid)
In-Reply-To: <49AF9AD3.2090405@simplistix.co.uk>
References: <49AC394E.2000903@v.loewis.de> <49AF9AD3.2090405@simplistix.co.uk>
Message-ID: <49B09381.9070604@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Chris Withers wrote:
> Martin v. L?wis wrote:
>> Martin v. L?wis <martin at v.loewis.de> added the comment:
>>
>>> So all Chris has to do to get this applied to 2.5 is craft an exploit based
>>> on the current behavior, right? ;-)
>> Right :-) Of course, security patches should see a much more careful
>> review than regular bug fixes.
> 
> Well, it's funny you say that, since where I bumped into this, the bug 
> was effectively DOS'ing a couple of mailservers as a result of 
> mailinglogger sending out log entries of uncaught exceptions such as 
> this and so emitting 100Mb emails whenever the foreign server chose not 
> to deliver the whole chunk requested...

If it is possible for a hostile outsider to trigger the DOS by sending
mail to be processed by an application using the library, and the
application can't avoid the DOS without ditching / forking /
monkeypatching the library, then I would call the bug a "security bug",
period.

As for backward compatibility:  any application which is depending on
getting arbitrarily-long lines in its logfile is already insane, and
should be scrapped.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJsJOB+gerLs4ltQ4RAva/AKC2Ta0edNMxMLxXQM6+WsB4AKo10QCdFF58
ghfy8pT6VlrO0z0QoXnjL7o=
=9lCT
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Fri Mar  6 04:07:45 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 05 Mar 2009 22:07:45 -0500
Subject: [Python-Dev] patch commit policies (was [issue4308] repr of
 httplib.IncompleteRead is stupid)
In-Reply-To: <49AF9AD3.2090405@simplistix.co.uk>
References: <49AC394E.2000903@v.loewis.de> <49AF9AD3.2090405@simplistix.co.uk>
Message-ID: <49B09381.9070604@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Chris Withers wrote:
> Martin v. L?wis wrote:
>> Martin v. L?wis <martin at v.loewis.de> added the comment:
>>
>>> So all Chris has to do to get this applied to 2.5 is craft an exploit based
>>> on the current behavior, right? ;-)
>> Right :-) Of course, security patches should see a much more careful
>> review than regular bug fixes.
> 
> Well, it's funny you say that, since where I bumped into this, the bug 
> was effectively DOS'ing a couple of mailservers as a result of 
> mailinglogger sending out log entries of uncaught exceptions such as 
> this and so emitting 100Mb emails whenever the foreign server chose not 
> to deliver the whole chunk requested...

If it is possible for a hostile outsider to trigger the DOS by sending
mail to be processed by an application using the library, and the
application can't avoid the DOS without ditching / forking /
monkeypatching the library, then I would call the bug a "security bug",
period.

As for backward compatibility:  any application which is depending on
getting arbitrarily-long lines in its logfile is already insane, and
should be scrapped.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJsJOB+gerLs4ltQ4RAva/AKC2Ta0edNMxMLxXQM6+WsB4AKo10QCdFF58
ghfy8pT6VlrO0z0QoXnjL7o=
=9lCT
-----END PGP SIGNATURE-----


From greg.ewing at canterbury.ac.nz  Fri Mar  6 07:08:40 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Mar 2009 19:08:40 +1300
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <ca471dc20903051354o25d80c2cr4b8ca95211bc7103@mail.gmail.com>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
	<ca471dc20903051310l16c89b0dvf3729f955d334462@mail.gmail.com>
	<49B045E3.3030607@canterbury.ac.nz>
	<ca471dc20903051354o25d80c2cr4b8ca95211bc7103@mail.gmail.com>
Message-ID: <49B0BDE8.30509@canterbury.ac.nz>

Guido van Rossum wrote:

> Then it'd be better to have a method clear_memo() on pickle objects.

You should have that anyway. I was just suggesting a
way of preserving compatibility with old code without
exposing all the details of the memo.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Mar  6 07:15:25 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Mar 2009 19:15:25 +1300
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
 of medusa
In-Reply-To: <loom.20090305T224150-476@post.gmane.org>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<49B0389E.5070004@canterbury.ac.nz>
	<loom.20090305T224150-476@post.gmane.org>
Message-ID: <49B0BF7D.3040709@canterbury.ac.nz>

Antoine Pitrou wrote:

> For starters, since py3k is supposed to support non-blocking IO, why not write a
> portable API to make a raw file or socket IO object non-blocking?

I think we need to be clearer what we mean when we talk
about non-blocking in this context. Normally when you're
using select/poll you *don't* make the underlying file
descriptor non-blocking in the OS sense. The non-blockingness
comes from the fact that you're using select/poll to make
sure the fd is ready before you access it.

So I don't think it makes sense to talk about having a
non-blocking API as a separate thing from a select/poll
wrapper. The select/poll wrapper *is* the non-blocking
API.

-- 
Greg

From hrvoje.niksic at avl.com  Fri Mar  6 10:12:33 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Fri, 06 Mar 2009 10:12:33 +0100
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
 of medusa
In-Reply-To: <11950256.2690373.1236320029484.JavaMail.xicrypt@atgrzls001>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>	<49AEF247.9060800@plope.com>
	<gop99c$et3$1@ger.gmane.org>	<49B0389E.5070004@canterbury.ac.nz>	<loom.20090305T224150-476@post.gmane.org>
	<11950256.2690373.1236320029484.JavaMail.xicrypt@atgrzls001>
Message-ID: <49B0E901.9070708@avl.com>

Greg Ewing wrote:
> Antoine Pitrou wrote:
> 
>> For starters, since py3k is supposed to support non-blocking IO, why not write a
>> portable API to make a raw file or socket IO object non-blocking?
> 
> I think we need to be clearer what we mean when we talk
> about non-blocking in this context. Normally when you're
> using select/poll you *don't* make the underlying file
> descriptor non-blocking in the OS sense. The non-blockingness
> comes from the fact that you're using select/poll to make
> sure the fd is ready before you access it.
> 
> So I don't think it makes sense to talk about having a
> non-blocking API as a separate thing from a select/poll
> wrapper. The select/poll wrapper *is* the non-blocking
> API.

This is not necessarily the case.  In fact, modern sources often 
recommend using poll along with the non-blocking sockets for (among 
other things) performance reasons.  For example, when a non-blocking 
socket becomes readable, you don't read from it only once and go back to 
the event loop, you read from it in a loop until you get EAGAIN.  This 
allows for processing of fast-incoming data with fewer system calls.

Linux's select(2) man page includes a similar advice with different 
motivation:

        Under Linux, select() may report a socket file descriptor
        as "ready for reading",  while  nevertheless
        a subsequent read blocks.  This could for example
        happen when data has arrived but upon
        examination has wrong checksum and is discarded.  There may
        be other circumstances  in  which  a
        file  descriptor  is  spuriously  reported  as ready.
        Thus it may be safer to use O_NONBLOCK on
        sockets that should not block.

Even if you don't agree that using O_NONBLOCK with select/poll is the 
best approach to non-blocking, I think there is enough existing practice 
of doing this to warrant separate consideration of non-blocking sockets 
(in the OS sense) and select/poll.

From greg.ewing at canterbury.ac.nz  Fri Mar  6 11:01:35 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 06 Mar 2009 23:01:35 +1300
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
 of medusa
In-Reply-To: <49B0E901.9070708@avl.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<49B0389E.5070004@canterbury.ac.nz>
	<loom.20090305T224150-476@post.gmane.org>
	<11950256.2690373.1236320029484.JavaMail.xicrypt@atgrzls001>
	<49B0E901.9070708@avl.com>
Message-ID: <49B0F47F.7030208@canterbury.ac.nz>

Hrvoje Niksic wrote:

>        Under Linux, select() may report a socket file descriptor
>        as "ready for reading",  while  nevertheless
>        a subsequent read blocks.

Blarg. Linux is broken, then. This should not happen.

>        This could for example
>        happen when data has arrived but upon
>        examination has wrong checksum and is discarded.

That's no excuse -- the kernel should check all its
checksums *before* waking up selecting processes!

> Even if you don't agree that using O_NONBLOCK with select/poll is the 
> best approach to non-blocking, I think there is enough existing practice 
> of doing this to warrant separate consideration of non-blocking sockets 
> (in the OS sense) and select/poll.

I'm not saying there isn't merit in having support for
non-blocking file descriptors, only that it's not in
any sense a prerequisite or first step towards a
select/poll wrapper. They're orthogonal issues, even
if you might sometimes want to use them together.

-- 
Greg

From stephen at xemacs.org  Fri Mar  6 11:12:14 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 06 Mar 2009 19:12:14 +0900
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <49B047F1.9080900@v.loewis.de>
References: <mailman.29.1235991618.1178.python-dev@python.org>
	<p06240803c5d1f82dbc4b@10.0.1.221> <49AC5C0B.4070806@v.loewis.de>
	<e45086040903050553m44f7bce7xca474d658e0c977f@mail.gmail.com>
	<49B047F1.9080900@v.loewis.de>
Message-ID: <877i336jup.fsf@xemacs.org>

"Martin v. L?wis" writes:

 > >From time to time, people ask what they can do push a change into Python
 > that they really think is important. I once offered that people who
 > want a patch in Python really badly should review 10 other patches in
 > return, up to the point where they make a recommendation about the fate
 > of the patches. I was then talked into accepting just 5 such patches.
 > I have since withdrawn this offer, because

I'm really sad to hear that.  I considered that one of the really nice
features of Python as a project (even though it was of course your
individual initiative).

 > a) I was the only one making that offer in public, and

IIRC others did, but you were the only one to do so repeatedly and as
a timely response to reports that the patch queue was going untended.

 > b) I was sometimes not really able to respond in a timely manner
 >    when the offer was invoked, because of overload.

Well, that happens.  An alternative to withdrawing entirely, would be
increasing the price (eg, to ten patches as you originally suggested).
Or specifying windows in your calendar when the offer is open.  Eg,
avoid doubling up on release times when you need make time to build
installers etc. ... but of course just before release is when people
will get antsy about their "lost" patches.

I hope that somebody will pick up the slack here, because review is
really important to the workflow, and getting more people involved in
reviewing at some level is more important (because it's less
glamorous in itself) than attracting coders.

From hrvoje.niksic at avl.com  Fri Mar  6 11:19:40 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Fri, 06 Mar 2009 11:19:40 +0100
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
 of medusa
In-Reply-To: <14833741.2702472.1236333605177.JavaMail.xicrypt@atgrzls001>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>	<49AEF247.9060800@plope.com>
	<gop99c$et3$1@ger.gmane.org>	<49B0389E.5070004@canterbury.ac.nz>	<loom.20090305T224150-476@post.gmane.org>	<11950256.2690373.1236320029484.JavaMail.xicrypt@atgrzls001>	<49B0E901.9070708@avl.com>
	<14833741.2702472.1236333605177.JavaMail.xicrypt@atgrzls001>
Message-ID: <49B0F8BC.4010101@avl.com>

Greg Ewing wrote:
>> Even if you don't agree that using O_NONBLOCK with select/poll is the 
>> best approach to non-blocking, I think there is enough existing practice 
>> of doing this to warrant separate consideration of non-blocking sockets 
>> (in the OS sense) and select/poll.
> 
> I'm not saying there isn't merit in having support for
> non-blocking file descriptors, only that it's not in
> any sense a prerequisite or first step towards a
> select/poll wrapper. They're orthogonal issues, even
> if you might sometimes want to use them together.

In that case we are in agreement.  Looking back, I was somewhat confused 
by this paragraph:

     So I don't think it makes sense to talk about having a
     non-blocking API as a separate thing from a select/poll
     wrapper. The select/poll wrapper *is* the non-blocking
     API.

If they're orthogonal, then it does make sense to talk about having a 
separate non-blocking socket API and poll API, even if the latter can be 
used to implement non-blocking *functionality* (hypothetical Linux 
issues aside).

From tleeuwenburg at gmail.com  Fri Mar  6 11:38:32 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Fri, 6 Mar 2009 21:38:32 +1100
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <877i336jup.fsf@xemacs.org>
References: <mailman.29.1235991618.1178.python-dev@python.org>
	<p06240803c5d1f82dbc4b@10.0.1.221> <49AC5C0B.4070806@v.loewis.de>
	<e45086040903050553m44f7bce7xca474d658e0c977f@mail.gmail.com>
	<49B047F1.9080900@v.loewis.de> <877i336jup.fsf@xemacs.org>
Message-ID: <43c8685c0903060238j238d2177kc189aa165697eb9d@mail.gmail.com>

> Well, that happens.  An alternative to withdrawing entirely, would be
> increasing the price (eg, to ten patches as you originally suggested).
> Or specifying windows in your calendar when the offer is open.  Eg,
> avoid doubling up on release times when you need make time to build
> installers etc. ... but of course just before release is when people
> will get antsy about their "lost" patches.
>
> I hope that somebody will pick up the slack here, because review is
> really important to the workflow, and getting more people involved in
> reviewing at some level is more important (because it's less
> glamorous in itself) than attracting coders.


It's funny ... I would have thought that one of the most attractive aspects
of offering patches for inclusion was not just getting feature X into the
language, but the opportunity to have your code reviewed by the best of the
best, or similarly to review the code of others and really think about its
strengths and weaknesses. I would have said that participating in a project
at that level would basically be the best opportunity for ongoing learning
and development available.
I guess I'm saying that I'm surprised people aren't a bit more appreciative
of the opportunity to review code. I mean, I wouldn't think that Python was
"just work" for anyone who has the passion to commit back to the core
project. I don't think I would even be on this list or attempting to put
together my first (and almost inconseqentially small) patch if it weren't
for the fact that I see it as a huge opportunity. It's certainly not an
attempt to 'push' anything into the language.

Obviously that's what you found though -- people who weren't really
understanding of how the language gets put together. I can imagine having
held that view in the past myself, also. I can to some extent understand the
perspective of feeling you have some fantastic idea which you'd love to get
implemented; yet the people who can make it happen are too concerned with
their own issues to take the time to roll in your changes.

Would you object to my blogging on the topic in line with the comments that
I have just made? I almost feel silly making that kind of suggestion after
having only been here a short time -- I feel a bit boorish! -- but having
run The Python Papers and also no longer being a 'green' developer at work,
I feel as though I do have something to contribute on the topic even if it
is somewhat immaturely conceived.

Regards,
-Tennessee
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090306/8ab36c59/attachment.htm>

From glyph at divmod.com  Fri Mar  6 11:56:42 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Fri, 06 Mar 2009 10:56:42 -0000
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of	medusa
In-Reply-To: <49B0F47F.7030208@canterbury.ac.nz>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<49B0389E.5070004@canterbury.ac.nz>
	<loom.20090305T224150-476@post.gmane.org>
	<11950256.2690373.1236320029484.JavaMail.xicrypt@atgrzls001>
	<49B0E901.9070708@avl.com> <49B0F47F.7030208@canterbury.ac.nz>
Message-ID: <20090306105642.12555.469721715.divmod.xquotient.5400@weber.divmod.com>


On 10:01 am, greg.ewing at canterbury.ac.nz wrote:
>Hrvoje Niksic wrote:
>>        Under Linux, select() may report a socket file descriptor
>>        as "ready for reading",  while  nevertheless
>>        a subsequent read blocks.
>
>Blarg. Linux is broken, then. This should not happen.

You know what else is broken?  MacOS, FreeBSD, Solaris, and every 
version of Windows.  I haven't tried running Twisted on AmigaOS but I 
bet it has some problems too.

On occasion Linux has been so badly broken that Twisted has motivated a 
fix.  For example, 
http://lkml.indiana.edu/hypermail/linux/kernel/0502.3/1160.html

But even if we ignore difficulties at the OS level (which should, after 
all, be worked around rather than catered to in API design) there are 
other good reasons why the general async API should be fairly distant 
from both the select/poll wrapper and the questions of blocking vs. non- 
blocking sockets.  For another example, consider the issue of non- 
blocking SSL sockets.  Sometimes, in order to do a "read", you actually 
need to do a write and then another read.  Which means that application 
code, if it wants to be compatible with SSL, needs to deal with any 
failure that may come from a write as coming from a read, unless you 
abstract all this away somehow.

From mhagger at alum.mit.edu  Fri Mar  6 10:57:00 2009
From: mhagger at alum.mit.edu (Michael Haggerty)
Date: Fri, 06 Mar 2009 10:57:00 +0100
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
Message-ID: <49B0F36C.8070800@alum.mit.edu>

Collin Winter wrote:
> [...] I've found a few examples of code using the memo attribute ([1], [2],
> [3]) [...]

As author of [2] (current version here [4]) I can tell you my reason.
cvs2svn has to store a vast number of small objects in a database, then
read them in random order.  I spent a lot of time optimizing this part
of the code because it is crucial for the overall performance when
converting large CVS repositories.  The objects are not all of the same
class and sometimes contain other objects, so it is convenient to use
pickling instead of, say, marshaling.

It is easy to optimize the pickling of instances by giving them
__getstate__() and __setstate__() methods.  But the pickler still
records the type of each object (essentially, the name of its class) in
each record.  The space for these strings constituted a large fraction
of the database size.

So I "prime" the picklers/unpicklers by pickling then unpickling a
"primer" that contains the classes that I know will appear, and storing
the resulting memos once in the database.  Then for each record I create
a new pickler/unpickler and initialize its memo to the "primer"'s memo
before using it to read the actual object.  This removes a lot of
redundancy across database records.

I only prime my picklers/unpicklers with the classes.  But note that the
same technique could be used for any repeated subcomponents.  This would
have the added advantage that all of the unpickled instances would share
copies of the objects that appear in the primer, which could be a
semantic advantage and a significant savings in RAM in addition to the
space and processing time advantages described above.  It might even be
a useful feature to the "shelve" module.

> So my questions are these:
> 1) Should Pickler/Unpickler objects automatically clear their memos
> when dumping/loading?
> 2) Is memo an intentionally exposed, supported part of the
> Pickler/Unpickler API, despite the lack of documentation and tests?

For my application, either of the following alternatives would also suffice:

- The ability to pickle picklers and unpicklers themselves (including
their memos).  This is, of course, awkward because they are hard-wired
to files.

- Picklers and unpicklers could have get_memo() and set_memo() methods
that return an opaque (but pickleable) memo object.  In other words, I
don't need to muck around inside the memo object; I just need to be able
to save and restore it.

Please note that the memo for a pickler is not equal to the memo of the
corresponding unpickler.

A similar effect could *almost* be obtained without accessing the memos
by saving the pickled primer itself in the database.  The unpickler
would be primed by using it to load the primer before loading the record
of interest.  But AFAIK there is no way to prime new picklers, because
there is no guarantee that pickling the same primer twice will result in
the same id->object mapping in the pickler's memo.

Michael

> [2] - http://google.com/codesearch/p?hl=en#M-DDI-lCOgE/lib/python2.4/site-packages/cvs2svn_lib/primed_pickle.py&q=lang:py%20%5C.memo
[4]
http://cvs2svn.tigris.org/source/browse/cvs2svn/trunk/cvs2svn_lib/serializer.py?view=markup

From solipsis at pitrou.net  Fri Mar  6 12:45:38 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 6 Mar 2009 11:45:38 +0000 (UTC)
Subject: [Python-Dev] Pickler/Unpickler API clarification
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
	<49B0F36C.8070800@alum.mit.edu>
Message-ID: <loom.20090306T114142-800@post.gmane.org>

Michael Haggerty <mhagger <at> alum.mit.edu> writes:
> 
> It is easy to optimize the pickling of instances by giving them
> __getstate__() and __setstate__() methods.  But the pickler still
> records the type of each object (essentially, the name of its class) in
> each record.  The space for these strings constituted a large fraction
> of the database size.

If these strings are not interned, then perhaps they should be.
There is a similar optimization proposal (w/ patch) for attribute names:
http://bugs.python.org/issue5084

Regards

Antoine.



From stefan_ml at behnel.de  Fri Mar  6 13:07:15 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 06 Mar 2009 13:07:15 +0100
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <49B05F43.5030706@v.loewis.de>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<gop8qa$csc$1@ger.gmane.org>	<ca471dc20903051135j35ef8bdax30be5f85c09f0ed4@mail.gmail.com>	<gopd12$t0k$1@ger.gmane.org>
	<49B05F43.5030706@v.loewis.de>
Message-ID: <gor3lk$gao$1@ger.gmane.org>

Martin v. L?wis wrote:
>> I do see the point you are making here. Even if lxml gets mature and
>> static, that doesn't necessarily apply to the external libraries it uses.
>> However, I should note that exactly the same argument also applies to
>> sqlite3 and gdbm, which, again, are in the stdlib today, with sqlite3 being
>> a fairly recent addition.
> 
> Fortunately, it is possible for users to just replace the sqlite DLL in
> a Python installation, with no need of recompiling anything.

Interesting. I assume you are referring to Windows here, right? Does that
"just work" because the DLL is in the same directory?

That would be a nice feature for lxml, too. We could just make the libxml2
and libxslt DLLs package data under Windows in that case.

Stefan


From fuzzyman at voidspace.org.uk  Fri Mar  6 14:04:29 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Fri, 06 Mar 2009 13:04:29 +0000
Subject: [Python-Dev] Python wins Linux New Media Award for Best Open Source
 Programming Language
Message-ID: <49B11F5D.7050209@voidspace.org.uk>

Hello all,

Not sure if this is the same as the LinuxQuestions award, but it looks 
different:
(German)
http://www.linux-magazin.de/news/cebit_2009_openstreetmap_erntet_zwei_linux_new_media_awards

I particularly like this snippet from the google translation:

The prize was Martin von L?wis of the Python Foundation on behalf of the 
Python community itself.

http://translate.google.co.uk/translate?hl=en&sl=de&u=http://www.linux-magazin.de/news/cebit_2009_openstreetmap_erntet_zwei_linux_new_media_awards&ei=VByxSfSnM9nHjAfb6ojPBQ&sa=X&oi=translate&resnum=1&ct=result&prev=/search%3Fq%3Dcebit%2B2009%2B%2BLinux%2BNew%2BMedia%2BAwards%2Bpython%26hl%3Den%26client%3Dfirefox-a%26rls%3Dorg.mozilla:en-GB:official%26hs%3DP9E

All the best,

Michael Foord
--
http://www.ironpythoninaction.com/

From orsenthil at gmail.com  Fri Mar  6 14:15:57 2009
From: orsenthil at gmail.com (Senthil Kumaran)
Date: Fri, 6 Mar 2009 18:45:57 +0530
Subject: [Python-Dev] Python wins Linux New Media Award for Best Open
	Source Programming Language
In-Reply-To: <49B11F5D.7050209@voidspace.org.uk>
References: <49B11F5D.7050209@voidspace.org.uk>
Message-ID: <7c42eba10903060515y4238b5faw4f58e2c3de1158d7@mail.gmail.com>

On Fri, Mar 6, 2009 at 6:34 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:

> The prize was Martin von L?wis of the Python Foundation on behalf of the
> Python community itself.

This is a funny translation from German-to-English. :-)

But yeah, a good one and the prize was presented by Kluas Knooper of Knoppix.

Congratulations!


-- 
Senthil

From daniel at stutzbachenterprises.com  Fri Mar  6 14:53:23 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Fri, 6 Mar 2009 07:53:23 -0600
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <loom.20090306T114142-800@post.gmane.org>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
	<49B0F36C.8070800@alum.mit.edu>
	<loom.20090306T114142-800@post.gmane.org>
Message-ID: <eae285400903060553w464df0d1v6527b516527b9515@mail.gmail.com>

On Fri, Mar 6, 2009 at 5:45 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> If these strings are not interned, then perhaps they should be.
> There is a similar optimization proposal (w/ patch) for attribute names:
> http://bugs.python.org/issue5084
>

If I understand correctly, that would help with unpickling, but wouldn't
solve Michael's problem as, without memo, each pickle would still need to
store a copy.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090306/10596872/attachment-0001.htm>

From solipsis at pitrou.net  Fri Mar  6 15:05:55 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 06 Mar 2009 15:05:55 +0100
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <49B11AC9.70100@alum.mit.edu>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
	<49B0F36C.8070800@alum.mit.edu>
	<loom.20090306T114142-800@post.gmane.org>
	<49B11AC9.70100@alum.mit.edu>
Message-ID: <1236348355.6999.2.camel@fsol>

Le vendredi 06 mars 2009 ? 13:44 +0100, Michael Haggerty a ?crit :
> Antoine Pitrou wrote:
> > Michael Haggerty <mhagger <at> alum.mit.edu> writes:
> >> It is easy to optimize the pickling of instances by giving them
> >> __getstate__() and __setstate__() methods.  But the pickler still
> >> records the type of each object (essentially, the name of its class) in
> >> each record.  The space for these strings constituted a large fraction
> >> of the database size.
> > 
> > If these strings are not interned, then perhaps they should be.
> > There is a similar optimization proposal (w/ patch) for attribute names:
> > http://bugs.python.org/issue5084
> 
> If I understand correctly, this would not help:
> 
> - on writing, the strings are identical anyway, because they are read
> out of the class's __name__ and __module__ fields.  Therefore the
> Pickler's usual memoizing behavior will prevent the strings from being
> written more than once.

Then why did you say that "the space for these strings constituted a
large fraction of the database size", if they are already shared? Are
your objects so tiny that even the space taken by the pointer to the
type name grows the size of the database significantly?



From ajaksu at gmail.com  Fri Mar  6 15:25:22 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Fri, 6 Mar 2009 11:25:22 -0300
Subject: [Python-Dev] [Tracker-discuss] Adding a "Rietveld this" button?
In-Reply-To: <2d75d7660903051854i7e75ba96ib88953230de2e5dd@mail.gmail.com>
References: <2d75d7660903042012y70883ea0r3b48fba3405e1a08@mail.gmail.com>
	<2c51ecee0903050724x2cfd1a50oe8838e9280de3696@mail.gmail.com>
	<2d75d7660903050755i37f6f107hb1fd14bd533684ca@mail.gmail.com>
	<2c51ecee0903050812j5f8e21bdp9d50743b139ef8c9@mail.gmail.com>
	<49B04B42.7090800@v.loewis.de>
	<2d75d7660903051529i5300e170rc2c655fd6fdfd14a@mail.gmail.com>
	<49B067B9.7070008@v.loewis.de>
	<2d75d7660903051854i7e75ba96ib88953230de2e5dd@mail.gmail.com>
Message-ID: <2d75d7660903060625i18a48121m39a5e0f633554f0e@mail.gmail.com>

CC'ing python-dev, as more RFEs might be uncovered :)

Daniel (ajax) Diniz wrote:
> Martin v. L?wis wrote:
>> I think a patch (or full file) would be good enough. We could put it
>> into the tracker itself, and publish it prominently where people
>> upload files.
>
> I'll post it as a patch and a full file at issue 247 when I get to it.

Posted as full file to
http://psf.upfronthosting.co.za/roundup/meta/issue247 , Guido suggests
the wrapper way so I won't bother with creating patches now.

> [snip code]
>> Nice! I didn't think of something that complicated (or, rather,
>> complicated in a different way):
>>
>> upload.py --roundup 5428
>
> Just to be clear, this will work as if the user passed the following options:
>
> upload.py --message "[issue5428] Pyshell history management error"
> --cc report at bugs.python.org
>
> Right? :)

If that is right, it works :)

>> That could either fill in a description given by -d, or fetch
>> the description from roundup.
>
> Fetching a description is as easy as fetching a title[1], but I can't
> think of a fixed place to look for one (last message? last patch
> description?). Maybe we can add another option that tells the script
> where to fetch description/content from? Something like "
> [-F|--fetch_description] msg83029" ?

Works too, for files or messages :)

Examples:
http://bugs.python.org/issue400608
http://bugs.python.org/issue2771
http://codereview.appspot.com/25073
http://codereview.appspot.com/24075

Thanks again,
Daniel

From stephen at xemacs.org  Fri Mar  6 16:15:20 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 7 Mar 2009 00:15:20 +0900
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <43c8685c0903060238j238d2177kc189aa165697eb9d@mail.gmail.com>
References: <mailman.29.1235991618.1178.python-dev@python.org>
	<p06240803c5d1f82dbc4b@10.0.1.221> <49AC5C0B.4070806@v.loewis.de>
	<e45086040903050553m44f7bce7xca474d658e0c977f@mail.gmail.com>
	<49B047F1.9080900@v.loewis.de> <877i336jup.fsf@xemacs.org>
	<43c8685c0903060238j238d2177kc189aa165697eb9d@mail.gmail.com>
Message-ID: <18865.15880.40623.112881@gargle.gargle.HOWL>

Tennessee Leeuwenburg writes:

 > > I hope that somebody will pick up the slack here, because review is
 > > really important to the workflow, and getting more people involved in
 > > reviewing at some level is more important (because it's less
 > > glamorous in itself) than attracting coders.

 > I would have said that participating in a project at that level
 > would basically be the best opportunity for ongoing learning and
 > development available.

It is, and IMO Python is an excellent example of that.  Please don't
get me wrong -- the core developers do a lot of reviewing.  It's just
not as visible, or as clearly available to non-core participants, as
Martin's 1-for-5 offer was.

Many, perhaps most, contributions are one-offs by people to whom
Python is a tool, not their community.  They have little time, and as
far as they know, less expertise to participate in the review process.
Martin's offer was an open invitation, in terms that any contributor
can appreciate, even if they don't take advantage of it right away.

I admire that style.

 > Would you object to my blogging on the topic in line with the
 > comments that I have just made?

It's not my place to say yes or no, to you or on behalf of the
community.

A suggestion, though.  View the contribution visualization video based
on the commit log (the URL was posted here a while back, but I don't
seem to have it offhand), which shows what a vibrant community this is
in a very graphic way.


From mhagger at alum.mit.edu  Fri Mar  6 13:44:57 2009
From: mhagger at alum.mit.edu (Michael Haggerty)
Date: Fri, 06 Mar 2009 13:44:57 +0100
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <loom.20090306T114142-800@post.gmane.org>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>	<49B0F36C.8070800@alum.mit.edu>
	<loom.20090306T114142-800@post.gmane.org>
Message-ID: <49B11AC9.70100@alum.mit.edu>

Antoine Pitrou wrote:
> Michael Haggerty <mhagger <at> alum.mit.edu> writes:
>> It is easy to optimize the pickling of instances by giving them
>> __getstate__() and __setstate__() methods.  But the pickler still
>> records the type of each object (essentially, the name of its class) in
>> each record.  The space for these strings constituted a large fraction
>> of the database size.
> 
> If these strings are not interned, then perhaps they should be.
> There is a similar optimization proposal (w/ patch) for attribute names:
> http://bugs.python.org/issue5084

If I understand correctly, this would not help:

- on writing, the strings are identical anyway, because they are read
out of the class's __name__ and __module__ fields.  Therefore the
Pickler's usual memoizing behavior will prevent the strings from being
written more than once.

- on reading, the strings are only used to look up the class.  Therefore
they are garbage collected almost immediately.

This is a different situation that that of attribute names, which are
stored persistently as the keys in the instance's __dict__.

Michael

From cjlesh at gmail.com  Fri Mar  6 17:09:03 2009
From: cjlesh at gmail.com (charlie)
Date: Fri, 6 Mar 2009 11:09:03 -0500
Subject: [Python-Dev] Windows 2.6.1 installer - msiexec not creating
	msvcr90.dll
Message-ID: <7e7202260903060809k28af83eeke56bb2c5439b268f@mail.gmail.com>

I am trying to script a Python installation on Windows, using msiexec from
the windows cmd prompt. I do not want to register extensions.

I have tried all the combinations I can find on the following page:
http://www.python.org/download/releases/2.5/msi/

But, no matter how I run msiexec, it seems that the msvcr90.dll in not
created in the Python26 directory.

If I double click the msi installer and run through it manually, the
msvcr90.dll is created.

Is there a way to run msiexec that results in msvc90.dll (and the manifest
file) getting created?

Thanks in advance,
-cjl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090306/f753e81c/attachment.htm>

From status at bugs.python.org  Fri Mar  6 18:06:49 2009
From: status at bugs.python.org (Python tracker)
Date: Fri,  6 Mar 2009 18:06:49 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090306170649.4E02078563@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (02/27/09 - 03/06/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2364 open (+31) / 14890 closed (+17) / 17254 total (+48)

Open issues with patches:   822

Average duration of open issues: 661 days.
Median duration of open issues: 398 days.

Open Issues Breakdown
   open  2337 (+29)
pending    27 ( +2)

Issues Created Or Reopened (51)
_______________________________

subprocess leaves open fds on construction error                 03/03/09
CLOSED http://bugs.python.org/issue5179    reopened ocean-city                    
       patch                                                                   

mmap and exception type                                          02/27/09
       http://bugs.python.org/issue5384    created  ocean-city                    
                                                                               

mmap can crash after resize failure (windows)                    02/27/09
CLOSED http://bugs.python.org/issue5385    created  ocean-city                    
       patch                                                                   

mmap can crash with write_byte                                   02/27/09
CLOSED http://bugs.python.org/issue5386    created  ocean-city                    
                                                                               

mmap.move crashes by integer overflow                            02/27/09
       http://bugs.python.org/issue5387    created  ocean-city                    
                                                                               

Green-box doc glitch: winhelp version only                       02/27/09
       http://bugs.python.org/issue5388    created  tjreedy                       
                                                                               

Uninitialized variable may be used in PyUnicode_DecodeUTF7Statef 02/27/09
CLOSED http://bugs.python.org/issue5389    created  gvanrossum                    
                                                                               

Item 'Python x.x.x' in Add/Remove Programs list still lacks an i 03/05/09
       http://bugs.python.org/issue5390    reopened loewis                        
                                                                               

mmap: read_byte/write_byte and object type                       02/28/09
       http://bugs.python.org/issue5391    created  ocean-city                    
       patch                                                                   

stack overflow after hitting recursion limit twice               02/28/09
       http://bugs.python.org/issue5392    created  gagenellina                   
       patch                                                                   

cmath.cos and cmath.cosh have "nResult" typo in help             02/28/09
CLOSED http://bugs.python.org/issue5393    created  mnewman                       
                                                                               

Distutils in trunk does not work with old Python (2.3 - 2.5)     02/28/09
       http://bugs.python.org/issue5394    created  akitada                       
       patch                                                                   

array.fromfile not checking I/O errors                           02/28/09
       http://bugs.python.org/issue5395    created  aguiar                        
                                                                               

os.read not handling O_DIRECT flag                               02/28/09
       http://bugs.python.org/issue5396    created  aguiar                        
                                                                               

PEP 372:  OrderedDict                                            03/01/09
CLOSED http://bugs.python.org/issue5397    created  rhettinger                    
       patch                                                                   

strftime("%B") returns a String unusable with unicode            03/01/09
CLOSED http://bugs.python.org/issue5398    created  t.steinruecken                
                                                                               

wer                                                              03/01/09
CLOSED http://bugs.python.org/issue5399    created  ajaksu2                       
                                                                               

patches for multiprocessing module on NetBSD                     03/01/09
       http://bugs.python.org/issue5400    created  aniou                         
       patch                                                                   

mimetypes.MAGIC_FUNCTION performance problems                    03/01/09
CLOSED http://bugs.python.org/issue5401    created  aronacher                     
       patch, patch, easy, needs review                                        

MutableMapping code smell (see OrderedDict)                      03/02/09
CLOSED http://bugs.python.org/issue5402    created  jimjjewett                    
                                                                               

test_md5 segfault                                                03/02/09
CLOSED http://bugs.python.org/issue5403    created  ocean-city                    
       patch                                                                   

Cross-compiling Python                                           03/02/09
       http://bugs.python.org/issue5404    created  kumba                         
                                                                               

There is no way of determining which ABCs a class is registered  03/02/09
       http://bugs.python.org/issue5405    created  pmoore                        
                                                                               

asyncore  doc issue                                              03/02/09
       http://bugs.python.org/issue5406    created  giampaolo.rodola              
                                                                               

Broken Py3.1 release build in Visual Studio 2005                 03/02/09
       http://bugs.python.org/issue5407    created  rhettinger                    
                                                                               

test_osx_env failing                                             03/02/09
CLOSED http://bugs.python.org/issue5408    created  benjamin.peterson             
                                                                               

ConfigParser get<int/float/boolean> methods broken               03/03/09
CLOSED http://bugs.python.org/issue5409    created  Absynthe                      
                                                                               

msvcrt bytes cleanup                                             03/03/09
       http://bugs.python.org/issue5410    created  ocean-city                    
       patch                                                                   

add xz compression support to distutils                          03/03/09
       http://bugs.python.org/issue5411    created  proyvind                      
                                                                               

extend configparser to support [] syntax                         03/03/09
       http://bugs.python.org/issue5412    created  jkaufman                      
       patch                                                                   

urllib ctypes error on Mac OS X Server 10.5                      03/03/09
       http://bugs.python.org/issue5413    created  atiware at gmx.net               
                                                                               

asciibin.a2b_uu returns unexpected values on non ascii data      03/04/09
       http://bugs.python.org/issue5414    created  dmajnemer                     
                                                                               

uuid module generates incorrect values for uuid3 (and possibly u 03/04/09
       http://bugs.python.org/issue5415    created  pwr                           
                                                                               

str.replace does strange things when given a negative count      03/04/09
       http://bugs.python.org/issue5416    created  dmajnemer                     
                                                                               

Reference to missing(?) function in Extending & Embedding Docume 03/04/09
       http://bugs.python.org/issue5417    created  aydt                          
                                                                               

urllib.response.addinfourl does not support __exit__             03/05/09
       http://bugs.python.org/issue5418    created  MLModel                       
                                                                               

urllib.request.open(someURL).read() returns a  bytes object so w 03/05/09
       http://bugs.python.org/issue5419    created  MLModel                       
                                                                               

Queue deprecation warning patch                                  03/05/09
       http://bugs.python.org/issue5420    created  tleeuwenburg at gmail.com        
                                                                               

Irritating error message by socket's sendto method               03/05/09
       http://bugs.python.org/issue5421    created  helduel                       
                                                                               

load pyc file with mbcs file system in update_compiled_module    03/05/09
CLOSED http://bugs.python.org/issue5422    created  joexo                         
                                                                               

Exception raised when attempting to call set_charset on an email 03/05/09
       http://bugs.python.org/issue5423    created  cjw296                        
                                                                               

Packed IPaddr conversion tests should be extended                03/05/09
       http://bugs.python.org/issue5424    created  phihag                        
       patch                                                                   

2to3 wrong for types.StringTypes                                 03/05/09
CLOSED http://bugs.python.org/issue5425    created  hagen                         
       patch                                                                   

README slight error re OSX                                       03/05/09
CLOSED http://bugs.python.org/issue5426    created  MLModel                       
                                                                               

OSX framework make error: ld: duplicate symbol _PyExc_BlockingIO 03/05/09
CLOSED http://bugs.python.org/issue5427    created  MLModel                       
                                                                               

Pyshell history management error                                 03/05/09
       http://bugs.python.org/issue5428    created  datamoc                       
                                                                               

Core dumps on the Solaris buildbot                               03/05/09
       http://bugs.python.org/issue5429    created  pitrou                        
       patch                                                                   

imaplib: must not replace LF or CR by CRLF in literals           03/06/09
       http://bugs.python.org/issue5430    created  memeplex                      
                                                                               

cmpfunc in Python 3.0.1 windows installer                        03/06/09
CLOSED http://bugs.python.org/issue5431    created  Nigel Galloway                
                                                                               

invalid use of setlocale                                         02/27/09
CLOSED http://bugs.python.org/issue804543  reopened georg.brandl                  
       patch                                                                   

Installed but not listed *.pyo break bdist_rpm                   03/01/09
CLOSED http://bugs.python.org/issue1533164 reopened tarek                         
       patch                                                                   



Issues Now Closed (52)
______________________

[performance] Too many closed() checkings                         478 days
       http://bugs.python.org/issue1407    pitrou                        
                                                                               

run_setup can fail if the setup script uses __file__              347 days
       http://bugs.python.org/issue2385    tarek                         
       patch                                                                   

test issue                                                        305 days
       http://bugs.python.org/issue2771    ajaksu2                       
                                                                               

various doc typos                                                 240 days
       http://bugs.python.org/issue3320    jnoller                       
       patch                                                                   

possible deadlock in python IO implementation                     197 days
       http://bugs.python.org/issue3618    benjamin.peterson             
       patch, needs review                                                     

Bottom buttons of IDLE Preferences Pane on Mac not wide enough f  169 days
       http://bugs.python.org/issue3883    ronaldoussoren                
       patch                                                                   

Failure building 64-bit Python on Leopard                         134 days
       http://bugs.python.org/issue4165    ronaldoussoren                
                                                                               

BufferedWriter non-blocking overage                               120 days
       http://bugs.python.org/issue4263    benjamin.peterson             
       patch                                                                   

repr of httplib.IncompleteRead is stupid                          110 days
       http://bugs.python.org/issue4308    benjamin.peterson             
       patch                                                                   

Rewrite the IO stack in C                                          88 days
       http://bugs.python.org/issue4565    benjamin.peterson             
                                                                               

.read() and .readline() differ in failing                          83 days
       http://bugs.python.org/issue4579    benjamin.peterson             
       patch, needs review                                                     

compile() doesn't ignore the source encoding when a string is pa   82 days
       http://bugs.python.org/issue4626    haypo                         
       patch, needs review                                                     

optimize bytecode for conditional branches                         70 days
       http://bugs.python.org/issue4715    jyasskin                      
       patch                                                                   

utf-16 BOM is not skipped after seek(0)                            57 days
       http://bugs.python.org/issue4862    haypo                         
       patch                                                                   

io.TextIOWrapper calls buffer.read1()                              45 days
       http://bugs.python.org/issue4996    haypo                         
                                                                               

subprocess leaves open fds on construction error                    0 days
       http://bugs.python.org/issue5179    ocean-city                    
       patch                                                                   

OS X IDLE.app and bin/idle: missing/extra menu options             23 days
       http://bugs.python.org/issue5194    ronaldoussoren                
                                                                               

OS X installer: "Update Shell Profile" script is not updated       20 days
       http://bugs.python.org/issue5224    ronaldoussoren                
                                                                               

OS X installer: Welcome and Readme files are out-of-date           20 days
       http://bugs.python.org/issue5226    ronaldoussoren                
                                                                               

StringIO failure when reading a chunk of text without newlines     18 days
       http://bugs.python.org/issue5264    benjamin.peterson             
                                                                               

StringIO can duplicate newlines in universal newlines mode         18 days
       http://bugs.python.org/issue5265    benjamin.peterson             
                                                                               

StringIO.read(n) does not enforce requested size in newline mode   18 days
       http://bugs.python.org/issue5266    benjamin.peterson             
                                                                               

3.0.1 crashes in unicode path                                      17 days
       http://bugs.python.org/issue5273    ocean-city                    
       patch                                                                   

document expected/required behavior of 3.x io subsystem with res   14 days
       http://bugs.python.org/issue5323    benjamin.peterson             
                                                                               

array.fromfile() fails to insert values when EOFError is raised    13 days
       http://bugs.python.org/issue5334    ocean-city                    
       patch                                                                   

collections.namedtuple generates code causing PyChecker warnings   10 days
       http://bugs.python.org/issue5336    rhettinger                    
                                                                               

Unicode control characters are not allowed as identifiers           3 days
       http://bugs.python.org/issue5358    loewis                        
                                                                               

RO (shorthand for READONLY) gone, not in documentation              3 days
       http://bugs.python.org/issue5360    georg.brandl                  
                                                                               

adding --quiet to bdist_rpm                                         1 days
       http://bugs.python.org/issue5378    tarek                         
                                                                               

Allow Python keywords as keyword arguments for functions.           0 days
       http://bugs.python.org/issue5382    mrabarnett                    
                                                                               

mmap can crash after resize failure (windows)                       6 days
       http://bugs.python.org/issue5385    ocean-city                    
       patch                                                                   

mmap can crash with write_byte                                      1 days
       http://bugs.python.org/issue5386    ocean-city                    
                                                                               

Uninitialized variable may be used in PyUnicode_DecodeUTF7Statef    6 days
       http://bugs.python.org/issue5389    gvanrossum                    
                                                                               

cmath.cos and cmath.cosh have "nResult" typo in help                0 days
       http://bugs.python.org/issue5393    marketdickinson               
                                                                               

PEP 372:  OrderedDict                                               2 days
       http://bugs.python.org/issue5397    forest                        
       patch                                                                   

strftime("%B") returns a String unusable with unicode               0 days
       http://bugs.python.org/issue5398    amaury.forgeotdarc            
                                                                               

wer                                                                 0 days
       http://bugs.python.org/issue5399    ajaksu2                       
                                                                               

mimetypes.MAGIC_FUNCTION performance problems                       0 days
       http://bugs.python.org/issue5401    benjamin.peterson             
       patch, patch, easy, needs review                                        

MutableMapping code smell (see OrderedDict)                         0 days
       http://bugs.python.org/issue5402    rhettinger                    
                                                                               

test_md5 segfault                                                   1 days
       http://bugs.python.org/issue5403    ocean-city                    
       patch                                                                   

test_osx_env failing                                                2 days
       http://bugs.python.org/issue5408    benjamin.peterson             
                                                                               

ConfigParser get<int/float/boolean> methods broken                  0 days
       http://bugs.python.org/issue5409    draghuram                     
                                                                               

load pyc file with mbcs file system in update_compiled_module       0 days
       http://bugs.python.org/issue5422    ocean-city                    
                                                                               

2to3 wrong for types.StringTypes                                    1 days
       http://bugs.python.org/issue5425    hagen                         
       patch                                                                   

README slight error re OSX                                          0 days
       http://bugs.python.org/issue5426    MLModel                       
                                                                               

OSX framework make error: ld: duplicate symbol _PyExc_BlockingIO    0 days
       http://bugs.python.org/issue5427    benjamin.peterson             
                                                                               

cmpfunc in Python 3.0.1 windows installer                           0 days
       http://bugs.python.org/issue5431    amaury.forgeotdarc            
                                                                               

just testing                                                     3179 days
       http://bugs.python.org/issue400608  ajaksu2                       
       patch                                                                   

invalid use of setlocale                                            0 days
       http://bugs.python.org/issue804543  georg.brandl                  
       patch                                                                   

Add ResolveFinderAliases to macostools module                    1869 days
       http://bugs.python.org/issue881824  ronaldoussoren                
                                                                               

Installed but not listed *.pyo break bdist_rpm                      1 days
       http://bugs.python.org/issue1533164 loewis                        
       patch                                                                   

mmap.mmap can overrun buffer                                      630 days
       http://bugs.python.org/issue1733986 ocean-city                    
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 26 PEP 372:  OrderedDict                                              2 days
closed  http://bugs.python.org/issue5397   

 15 Item 'Python x.x.x' in Add/Remove Programs list still lacks an     1 days
open    http://bugs.python.org/issue5390   

 14 Installed but not listed *.pyo break bdist_rpm                     1 days
closed  http://bugs.python.org/issue1533164

 10 json needs object_pairs_hook                                       7 days
open    http://bugs.python.org/issue5381   

  9 merge json library with latest simplejson 2.0.x                  141 days
open    http://bugs.python.org/issue4136   

  9 asyncore delayed calls feature                                   445 days
open    http://bugs.python.org/issue1641   

  8 Queue deprecation warning patch                                    2 days
open    http://bugs.python.org/issue5420   

  8 test_osx_env failing                                               2 days
closed  http://bugs.python.org/issue5408   

  8 Broken Py3.1 release build in Visual Studio 2005                   4 days
pending http://bugs.python.org/issue5407   

  8 Cross-compiling Python                                             4 days
open    http://bugs.python.org/issue5404   




From tjreedy at udel.edu  Fri Mar  6 18:54:12 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 06 Mar 2009 12:54:12 -0500
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <gor3lk$gao$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<gop8qa$csc$1@ger.gmane.org>	<ca471dc20903051135j35ef8bdax30be5f85c09f0ed4@mail.gmail.com>	<gopd12$t0k$1@ger.gmane.org>	<49B05F43.5030706@v.loewis.de>
	<gor3lk$gao$1@ger.gmane.org>
Message-ID: <goro00$okr$1@ger.gmane.org>

Stefan Behnel wrote:
> Martin v. L?wis wrote:
>>> I do see the point you are making here. Even if lxml gets mature and
>>> static, that doesn't necessarily apply to the external libraries it uses.
>>> However, I should note that exactly the same argument also applies to
>>> sqlite3 and gdbm, which, again, are in the stdlib today, with sqlite3 being
>>> a fairly recent addition.
>> Fortunately, it is possible for users to just replace the sqlite DLL in
>> a Python installation, with no need of recompiling anything.
> 
> Interesting. I assume you are referring to Windows here, right? Does that
> "just work" because the DLL is in the same directory?

I have no idea, but my WinXP .../Python30/ install has

DLLs/_sqlite3.pyd 52K
DLLs/sqlite3.dll 557K
libs/_sqlite3.lib 2K

For whatever reason, most other things do not have all three files.

I do not know whether upgrades (like 3.0.0 to 3.0.1) would clobber other 
things added here.

> That would be a nice feature for lxml, too. We could just make the libxml2
> and libxslt DLLs package data under Windows in that case.


From mhagger at alum.mit.edu  Fri Mar  6 19:01:59 2009
From: mhagger at alum.mit.edu (Michael Haggerty)
Date: Fri, 06 Mar 2009 19:01:59 +0100
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <1236348355.6999.2.camel@fsol>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>	<49B0F36C.8070800@alum.mit.edu>	<loom.20090306T114142-800@post.gmane.org>	<49B11AC9.70100@alum.mit.edu>
	<1236348355.6999.2.camel@fsol>
Message-ID: <49B16517.3090207@alum.mit.edu>

Antoine Pitrou wrote:
> Le vendredi 06 mars 2009 ? 13:44 +0100, Michael Haggerty a ?crit :
>> Antoine Pitrou wrote:
>>> Michael Haggerty <mhagger <at> alum.mit.edu> writes:
>>>> It is easy to optimize the pickling of instances by giving them
>>>> __getstate__() and __setstate__() methods.  But the pickler still
>>>> records the type of each object (essentially, the name of its class) in
>>>> each record.  The space for these strings constituted a large fraction
>>>> of the database size.
>>> If these strings are not interned, then perhaps they should be.
>>> There is a similar optimization proposal (w/ patch) for attribute names:
>>> http://bugs.python.org/issue5084
>> If I understand correctly, this would not help:
>>
>> - on writing, the strings are identical anyway, because they are read
>> out of the class's __name__ and __module__ fields.  Therefore the
>> Pickler's usual memoizing behavior will prevent the strings from being
>> written more than once.
> 
> Then why did you say that "the space for these strings constituted a
> large fraction of the database size", if they are already shared? Are
> your objects so tiny that even the space taken by the pointer to the
> type name grows the size of the database significantly?

Sorry for the confusion.  I thought you were suggesting the change to
help the more typical use case, when a single Pickler is used for a lot
of data.  That use case will not be helped by interning the class
__name__ and __module__ strings, for the reasons given in my previous email.

In my case, the strings are shared via the Pickler memoizing mechanism
because I pre-populate the memo (using the API that the OP proposes to
remove), so your suggestion won't help my current code, either.  It was
before I implemented the pre-populated memoizer that "the space for
these strings constituted a large fraction of the database size".  But
your suggestion wouldn't help that case, either.

Here are the main use cases:

1. Saving and loading one large record.  A class's __name__ string is
the same string object every time it is retrieved, so it only needs to
be stored once and the Pickler memo mechanism works.  Similarly for the
class's __module__ string.

2. Saving and loading lots of records sequentially.  Provided a single
Pickler is used for all records and its memo is never cleared, this
works just as well as case 1.

3. Saving and loading lots of records in random order, as for example in
the shelve module.  It is not possible to reuse a Pickler with retained
memo, because the Unpickler might not encounter objects in the right
order.  There are two subcases:

   a. Use a clean Pickler/Unpickler object for each record.  In this
case the __name__ and __module__ of a class will appear once in each
record in which the class appears.  (This is the case regardless of
whether they are interned.)  On reading, the __name__ and __module__ are
only used to look up the class, so interning them won't help.  It is
thus impossible to avoid wasting a lot of space in the database.

   b. Use a Pickler/Unpickler with a preset memo for each record (my
unorthodox technique).  In this case the class __name__ and __module__
will be memoized in the shared memo, so in other records only their ID
needs to be stored (in fact, only the ID of the class object itself).
This allows the database to be smaller, but does not have any effect on
the RAM usage of the loaded objects.

If the OP's proposal is accepted, 3b will become impossible.  The
technique seems not to be well known, so maybe it doesn't need to be
supported.  It would mean some extra work for me on the cvs2svn project
though :-(

Michael


From guido at python.org  Fri Mar  6 19:25:44 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Mar 2009 10:25:44 -0800
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <goro00$okr$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com>
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
	<gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<gop8qa$csc$1@ger.gmane.org>
	<ca471dc20903051135j35ef8bdax30be5f85c09f0ed4@mail.gmail.com>
	<gopd12$t0k$1@ger.gmane.org> <49B05F43.5030706@v.loewis.de>
	<gor3lk$gao$1@ger.gmane.org> <goro00$okr$1@ger.gmane.org>
Message-ID: <ca471dc20903061025v3bb9e9c4n9f37dcd1a05c60dc@mail.gmail.com>

On Fri, Mar 6, 2009 at 9:54 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> Stefan Behnel wrote:
>>
>> Martin v. L?wis wrote:
>>>>
>>>> I do see the point you are making here. Even if lxml gets mature and
>>>> static, that doesn't necessarily apply to the external libraries it
>>>> uses.
>>>> However, I should note that exactly the same argument also applies to
>>>> sqlite3 and gdbm, which, again, are in the stdlib today, with sqlite3
>>>> being
>>>> a fairly recent addition.
>>>
>>> Fortunately, it is possible for users to just replace the sqlite DLL in
>>> a Python installation, with no need of recompiling anything.
>>
>> Interesting. I assume you are referring to Windows here, right? Does that
>> "just work" because the DLL is in the same directory?

No, it is expected to "just work" because sqlite3 is (presumably) very
careful about backwards compatibility, and because the Windows DLL API
(just like the shared library API in Linux and other systems) is
designed to allow substitution of newer versions. The linkage
requirements are roughly that all entry points into a DLL (or shared
library) that are referenced by the caller (in this case the wrapper
extension module) are supported in the new version, and have the same
signature and semantics.

> I have no idea, but my WinXP .../Python30/ install has
>
> DLLs/_sqlite3.pyd 52K

This is the wrapper extension module.

> DLLs/sqlite3.dll 557K

This is sqlite3 itself. I am presuming that the phrase "replace the
sqlite DLL" above refers to this one -- although the same argument
actually holds for the .pyd file, which is also a DLL (despite its
different extension).

> libs/_sqlite3.lib 2K

I think this is a summary of the entry points into one of the above
DLLs for the benefit of other code wanting to link against it, but I'm
not sure.

> For whatever reason, most other things do not have all three files.

You only see a .pyd and a .dll when there's a Python wrapper extension
*and* an underlying 3rd party library.

> I do not know whether upgrades (like 3.0.0 to 3.0.1) would clobber other
> things added here.

It would, but not in a harmful way.

>> That would be a nice feature for lxml, too. We could just make the libxml2
>> and libxslt DLLs package data under Windows in that case.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tjreedy at udel.edu  Fri Mar  6 20:08:33 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 06 Mar 2009 14:08:33 -0500
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <ca471dc20903061025v3bb9e9c4n9f37dcd1a05c60dc@mail.gmail.com>
References: <49ABCFC9.1070508@gmail.com>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<gop8qa$csc$1@ger.gmane.org>	<ca471dc20903051135j35ef8bdax30be5f85c09f0ed4@mail.gmail.com>	<gopd12$t0k$1@ger.gmane.org>
	<49B05F43.5030706@v.loewis.de>	<gor3lk$gao$1@ger.gmane.org>
	<goro00$okr$1@ger.gmane.org>
	<ca471dc20903061025v3bb9e9c4n9f37dcd1a05c60dc@mail.gmail.com>
Message-ID: <gorsbd$7oi$1@ger.gmane.org>

Guido van Rossum wrote:
> On Fri, Mar 6, 2009 at 9:54 AM, Terry Reedy <tjreedy at udel.edu> wrote:

> No, it is expected to "just work" because sqlite3 is (presumably) very
> careful about backwards compatibility, and because the Windows DLL API
> (just like the shared library API in Linux and other systems) is
> designed to allow substitution of newer versions. The linkage
> requirements are roughly that all entry points into a DLL (or shared
> library) that are referenced by the caller (in this case the wrapper
> extension module) are supported in the new version, and have the same
> signature and semantics.
> 
>> I have no idea, but my WinXP .../Python30/ install has
>>
>> DLLs/_sqlite3.pyd 52K
> 
> This is the wrapper extension module.
> 
>> DLLs/sqlite3.dll 557K
> 
> This is sqlite3 itself. I am presuming that the phrase "replace the
> sqlite DLL" above refers to this one -- although the same argument
> actually holds for the .pyd file, which is also a DLL (despite its
> different extension).
> 
>> libs/_sqlite3.lib 2K
> 
> I think this is a summary of the entry points into one of the above
> DLLs for the benefit of other code wanting to link against it, but I'm
> not sure.
> 
>> For whatever reason, most other things do not have all three files.
> 
> You only see a .pyd and a .dll when there's a Python wrapper extension
> *and* an underlying 3rd party library.

Thanks, I understand now.


>> I do not know whether upgrades (like 3.0.0 to 3.0.1) would clobber other
>> things added here.
> 
> It would, but not in a harmful way.

By 'clobber', I meant 'delete', and I do not see how that would not be 
harmful ;-).  I don't know whether the intaller creates a new directory 
(and deletes the old), clears and reuses the old, or merely replaces 
individual files.

tjr


From martin at v.loewis.de  Fri Mar  6 20:17:01 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Mar 2009 20:17:01 +0100
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <877i336jup.fsf@xemacs.org>
References: <mailman.29.1235991618.1178.python-dev@python.org>	<p06240803c5d1f82dbc4b@10.0.1.221>	<49AC5C0B.4070806@v.loewis.de>	<e45086040903050553m44f7bce7xca474d658e0c977f@mail.gmail.com>	<49B047F1.9080900@v.loewis.de>
	<877i336jup.fsf@xemacs.org>
Message-ID: <49B176AD.9030406@v.loewis.de>

> I hope that somebody will pick up the slack here, because review is
> really important to the workflow, and getting more people involved in
> reviewing at some level is more important (because it's less
> glamorous in itself) than attracting coders.

Ok, then let me phrase it this way: if somebody else makes the offer,
I'll continue to support it (so to share the load between us two).

Regards,
Martin


From martin at v.loewis.de  Fri Mar  6 20:25:05 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Mar 2009 20:25:05 +0100
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <43c8685c0903060238j238d2177kc189aa165697eb9d@mail.gmail.com>
References: <mailman.29.1235991618.1178.python-dev@python.org>	
	<p06240803c5d1f82dbc4b@10.0.1.221> <49AC5C0B.4070806@v.loewis.de>	
	<e45086040903050553m44f7bce7xca474d658e0c977f@mail.gmail.com>	
	<49B047F1.9080900@v.loewis.de> <877i336jup.fsf@xemacs.org>
	<43c8685c0903060238j238d2177kc189aa165697eb9d@mail.gmail.com>
Message-ID: <49B17891.50801@v.loewis.de>

> I guess I'm saying that I'm surprised people aren't a bit more
> appreciative of the opportunity to review code. 

Not sure what "people" you are referring to here which aren't
appreciative of the opportunity to review code. Committers?
Non-committers?

> I don't think I would even be on this list or
> attempting to put together my first (and almost inconseqentially small)
> patch if it weren't for the fact that I see it as a huge opportunity.
> It's certainly not an attempt to 'push' anything into the language.

And this attitude I like best from contributors. Many people contribute
because they want to help, and don't expect anything in return.

However, many other people contribute because it solves a problem that
they have (scratch your own itch). They keep having the problem even
after they fixed it, in a sense, because they now have to reapply the
patch over and over again - for each Python release, and possibly for
each machine they deploy to (and for some, they can't change the
installed Python). Those people are eager to see their patch integrated,
preferably into the version that is already installed on their machines
(which requires the time machine :-)

> Would you object to my blogging on the topic in line with the comments
> that I have just made? 

Go ahead! I really can't say much about blogging - I don't write blogs,
nor read them.

Regards,
Martin

From guido at python.org  Fri Mar  6 20:44:19 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Mar 2009 11:44:19 -0800
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <gorsbd$7oi$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<gop8qa$csc$1@ger.gmane.org>
	<ca471dc20903051135j35ef8bdax30be5f85c09f0ed4@mail.gmail.com>
	<gopd12$t0k$1@ger.gmane.org> <49B05F43.5030706@v.loewis.de>
	<gor3lk$gao$1@ger.gmane.org> <goro00$okr$1@ger.gmane.org>
	<ca471dc20903061025v3bb9e9c4n9f37dcd1a05c60dc@mail.gmail.com>
	<gorsbd$7oi$1@ger.gmane.org>
Message-ID: <ca471dc20903061144p4fde1557xdc7d8a2683572933@mail.gmail.com>

On Fri, Mar 6, 2009 at 11:08 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>>> I do not know whether upgrades (like 3.0.0 to 3.0.1) would clobber other
>>> things added here.
>>
>> It would, but not in a harmful way.
>
> By 'clobber', I meant 'delete', and I do not see how that would not be
> harmful ;-). ?I don't know whether the intaller creates a new directory (and
> deletes the old), clears and reuses the old, or merely replaces individual
> files.

I see. I didn't realize you were talking about adding your own files
to these directories. I have no idea; the best way to find out is to
experiment. I could see the default policy of Windows installers go
either way.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Fri Mar  6 20:57:31 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 06 Mar 2009 20:57:31 +0100
Subject: [Python-Dev] patch commit policies (was [issue4308] repr of
 httplib.IncompleteRead is stupid)
In-Reply-To: <49B09381.9070604@palladion.com>
References: <49AC394E.2000903@v.loewis.de> <49AF9AD3.2090405@simplistix.co.uk>
	<49B09381.9070604@palladion.com>
Message-ID: <49B1802B.8030100@v.loewis.de>

> If it is possible for a hostile outsider to trigger the DOS by sending
> mail to be processed by an application using the library, and the
> application can't avoid the DOS without ditching / forking /
> monkeypatching the library, then I would call the bug a "security bug",
> period.

IIUC, it would have been straight forward for the mail servers to avoid
the DOS: simply truncate log lines to 1024 bytes, or something.

> As for backward compatibility:  any application which is depending on
> getting arbitrarily-long lines in its logfile is already insane, and
> should be scrapped.

That's not the point. The point is that the very old releases don't
get sufficient review for bug fixes, because too few people care
about them. So a systematic, efficient review by a single person of the
entire release must be possible. This is only possible if the number
of changes is kept to an absolute minimum - just the patches targeted
at the audience of these releases.

Regards,
Martin

From martin at v.loewis.de  Fri Mar  6 21:02:48 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 06 Mar 2009 21:02:48 +0100
Subject: [Python-Dev] Python wins Linux New Media Award for Best Open
 Source Programming Language
In-Reply-To: <7c42eba10903060515y4238b5faw4f58e2c3de1158d7@mail.gmail.com>
References: <49B11F5D.7050209@voidspace.org.uk>
	<7c42eba10903060515y4238b5faw4f58e2c3de1158d7@mail.gmail.com>
Message-ID: <49B18168.7070703@v.loewis.de>

>> The prize was Martin von L?wis of the Python Foundation on behalf of the
>> Python community itself.
> 
> This is a funny translation from German-to-English. :-)
> 
> But yeah, a good one and the prize was presented by Klaus Knopper of Knoppix.
> 
> Congratulations!

Actually, the prize went to "Python", not to me, and not to the PSF. So
congratulations to you as well!

It was a nice ceremony; among the 200 jurors, Python was elected
"Best Open Source Programming Language" by a very clear distance
to the second place.

Regards,
Martin

From martin at v.loewis.de  Fri Mar  6 21:04:14 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Mar 2009 21:04:14 +0100
Subject: [Python-Dev] Windows 2.6.1 installer - msiexec not
	creating	msvcr90.dll
In-Reply-To: <7e7202260903060809k28af83eeke56bb2c5439b268f@mail.gmail.com>
References: <7e7202260903060809k28af83eeke56bb2c5439b268f@mail.gmail.com>
Message-ID: <49B181BE.602@v.loewis.de>

charlie wrote:
> I am trying to script a Python installation on Windows, using msiexec
> from the windows cmd prompt. I do not want to register extensions.
> 
> I have tried all the combinations I can find on the following page:
> http://www.python.org/download/releases/2.5/msi/
> 
> But, no matter how I run msiexec, it seems that the msvcr90.dll in not
> created in the Python26 directory.
> 
> If I double click the msi installer and run through it manually, the
> msvcr90.dll is created.
> 
> Is there a way to run msiexec that results in msvc90.dll (and the
> manifest file) getting created?

This question is out of scope for python-dev; use python-list at python.org
instead.

My guess is that you have installed "for all users" in the command line,
so msvcr90.dll went into system32.

Regards,
Martin

From cjlesh at gmail.com  Fri Mar  6 21:16:11 2009
From: cjlesh at gmail.com (charlie)
Date: Fri, 6 Mar 2009 15:16:11 -0500
Subject: [Python-Dev] Windows 2.6.1 installer - msiexec not creating
	msvcr90.dll
In-Reply-To: <49B181BE.602@v.loewis.de>
References: <7e7202260903060809k28af83eeke56bb2c5439b268f@mail.gmail.com>
	<49B181BE.602@v.loewis.de>
Message-ID: <7e7202260903061216m256aba64y3b2e9954a22ceaa6@mail.gmail.com>

 Thanks for the reply.

This question is out of scope for python-dev; use python-list at python.org
> instead.


The last time I tried a question about the msi installer on python-list, no
one answered, so I thought it might be more appropriate for the dev list.

I actually thought this might be a bug with the new windows installer,
because this used to work with the 2.5.x installers. Before, using the exact
same msiexec command and options, a Python25 directory would be created
containing the required (if not already present) msvcr70.dll.


> My guess is that you have installed "for all users" in the command line,
> so msvcr90.dll went into system32.


I tried passing in the option ALLUSERS=0, which should be the default, and
it still did not work. Also, the dll is not anywhere on the system, and the
Python installed with msiexec will not run without it.

Anyway, I still can't figure out any way use msiexec to install 2.6.1 and
create the needed msvcr90.dll. Maybe it's not possible?

Thanks again, I'll try this question on python-list.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090306/2a22ffea/attachment.htm>

From rdmurray at bitdance.com  Fri Mar  6 21:33:46 2009
From: rdmurray at bitdance.com (rdmurray at bitdance.com)
Date: Fri, 6 Mar 2009 15:33:46 -0500 (EST)
Subject: [Python-Dev] patch commit policies (was [issue4308] repr of
 httplib.IncompleteRead is stupid)
In-Reply-To: <49B1802B.8030100@v.loewis.de>
References: <49AC394E.2000903@v.loewis.de> <49AF9AD3.2090405@simplistix.co.uk>
	<49B09381.9070604@palladion.com> <49B1802B.8030100@v.loewis.de>
Message-ID: <Pine.LNX.4.64.0903061504200.12396@kimball.webabinitio.net>

On Fri, 6 Mar 2009 at 20:57, "Martin v. L??wis" wrote:
>> If it is possible for a hostile outsider to trigger the DOS by sending
>> mail to be processed by an application using the library, and the
>> application can't avoid the DOS without ditching / forking /
>> monkeypatching the library, then I would call the bug a "security bug",
>> period.
>
> IIUC, it would have been straight forward for the mail servers to avoid
> the DOS: simply truncate log lines to 1024 bytes, or something.

I believe that in general things that allow DOS attacks to be staged are
considered security vulnerabilities by the general security community,
albeit of lower priority than exploits.  I believe the logic is that
one would prefer the system administrator not to have to figure out what
caused the DOS and how fix it after getting hit by it and having had a
service outage as a result.

Normally the "vendor" of package with the DOS vulnerability would provide
a fix and push it out, and a conscientious sysadmin would install the
"security release" and thus be protected.  In this case the application
vendor can only fix the DOS bug by modifying the library, and that would
fix it only for that application.  The logical place to fix it is at
the source: the library in question.

But since a DOS is lower priority from a security standpoint, I can
see the argument for not burdening the release maintainer with anti-DOS
patches.

We probably should leave it to the release maintainer to decide based
on some assessment of the likely impact of not fixing it.  Which means
it might not get fixed, but that's the reality of limited development
and maintenance resources.

--RDM

From martin at v.loewis.de  Fri Mar  6 22:04:59 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Mar 2009 22:04:59 +0100
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <gor3lk$gao$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<gop8qa$csc$1@ger.gmane.org>	<ca471dc20903051135j35ef8bdax30be5f85c09f0ed4@mail.gmail.com>	<gopd12$t0k$1@ger.gmane.org>	<49B05F43.5030706@v.loewis.de>
	<gor3lk$gao$1@ger.gmane.org>
Message-ID: <49B18FFB.9060002@v.loewis.de>

> Interesting. I assume you are referring to Windows here, right? Does that
> "just work" because the DLL is in the same directory?

Correct. Also, because changes to SQLite don't change the API, just the
implementation.

Regards,
Martin

From martin at v.loewis.de  Fri Mar  6 22:10:12 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Mar 2009 22:10:12 +0100
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <ca471dc20903061025v3bb9e9c4n9f37dcd1a05c60dc@mail.gmail.com>
References: <49ABCFC9.1070508@gmail.com>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<gop8qa$csc$1@ger.gmane.org>	<ca471dc20903051135j35ef8bdax30be5f85c09f0ed4@mail.gmail.com>	<gopd12$t0k$1@ger.gmane.org>
	<49B05F43.5030706@v.loewis.de>	<gor3lk$gao$1@ger.gmane.org>
	<goro00$okr$1@ger.gmane.org>
	<ca471dc20903061025v3bb9e9c4n9f37dcd1a05c60dc@mail.gmail.com>
Message-ID: <49B19134.2070803@v.loewis.de>

>> DLLs/sqlite3.dll 557K
> 
> This is sqlite3 itself. I am presuming that the phrase "replace the
> sqlite DLL" above refers to this one

Correct.

> -- although the same argument actually holds for the .pyd file

Not quite. You can download Windows binaries for newer sqlite
versions from sqlite.org, so you don't need a compiler to update
sqlite (which you likely would if _sqlite3.pyd would need to be
replaced). So you can "bypass" Python and its release process for
updates to sqlite.

>> libs/_sqlite3.lib 2K
> 
> I think this is a summary of the entry points into one of the above
> DLLs for the benefit of other code wanting to link against it, but I'm
> not sure.

Correct. I don't know why I include them in the MSI - they are there
because they were also shipped with the Wise installer. I see no
use - nobody should be linking against an extension module.

>> I do not know whether upgrades (like 3.0.0 to 3.0.1) would clobber other
>> things added here.
> 
> It would, but not in a harmful way.

If the user had upgrade sqlite, upgrading Python would undo that,
though. So one would have to re-upgrade sqlite afterwards.

Regards,
Martin

From martin at v.loewis.de  Fri Mar  6 22:14:45 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Mar 2009 22:14:45 +0100
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <ca471dc20903061144p4fde1557xdc7d8a2683572933@mail.gmail.com>
References: <49ABCFC9.1070508@gmail.com>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<gop8qa$csc$1@ger.gmane.org>	<ca471dc20903051135j35ef8bdax30be5f85c09f0ed4@mail.gmail.com>	<gopd12$t0k$1@ger.gmane.org>
	<49B05F43.5030706@v.loewis.de>	<gor3lk$gao$1@ger.gmane.org>
	<goro00$okr$1@ger.gmane.org>	<ca471dc20903061025v3bb9e9c4n9f37dcd1a05c60dc@mail.gmail.com>	<gorsbd$7oi$1@ger.gmane.org>
	<ca471dc20903061144p4fde1557xdc7d8a2683572933@mail.gmail.com>
Message-ID: <49B19245.5090501@v.loewis.de>

> I see. I didn't realize you were talking about adding your own files
> to these directories. I have no idea; the best way to find out is to
> experiment. I could see the default policy of Windows installers go
> either way.

An upgrade installation removes all old files it installed (the old
MSI is still present to know what these files are), then installs
new files.

Microsoft intended version resources to be used in the upgrade, so
the upgrade would only have to replace the files that got a new
version (rather than having to do uninstall-then-install).
Unfortunately, that is incapable of upgrading .py files. So Microsoft
added md5 (I think) hashes that can be used to detect files that
don't need upgrade. I tested it, and it was *very* slow, so I reverted
to the current procedure.

In any case, any additional files present will remain untouched.
They will also remain on uninstallation - so uninstallation might not
be able to remove all folders that installation originally created.

Regards,
Martin

From martin at v.loewis.de  Fri Mar  6 23:06:12 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Mar 2009 23:06:12 +0100
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
 Queue.full()
In-Reply-To: <9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
Message-ID: <49B19E54.3060407@v.loewis.de>

> IIRC, that was the rationale for cmp() removal in 3.0.1.

And indeed, that removal already caused a bug report and broke
the efforts of SWIG to support Python 3.0.

I disagree that our users are served by constantly breaking the
API, and removing stuff just because we can. I can't see how
removing API can possibly serve a user.

What's wrong with empty() and full() in the first place?

Regards,
Martin

From python at rcn.com  Sat Mar  7 00:19:24 2009
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 6 Mar 2009 15:19:24 -0800
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
	<49B19E54.3060407@v.loewis.de>
Message-ID: <B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>


[Martin v. L?wis]
> I disagree that our users are served by constantly breaking the
> API, and removing stuff just because we can. I can't see how
> removing API can possibly serve a user.

Am not following you here.  My suggestion was to remove the two
methods in Py3.1 which isn't even in alpha yet.  This is for a feature
that has a simple substitute, was undocumented for Py3.0, and had
long been documented in Py2.x as being unreliable.

It's seems silly to me that an incomplete patch from a year ago
would need to wait another two years to ever see the light of day
(am presuming that 3.1 goes final this summer and that 3.2 follows
18 months later). That being said, I don't really care much.
We don't actually have to do anything.  It could stay in forever
and cause no harm.


Raymond 


From tleeuwenburg at gmail.com  Sat Mar  7 00:42:08 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Sat, 7 Mar 2009 10:42:08 +1100
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
In-Reply-To: <B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>
	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
	<49B19E54.3060407@v.loewis.de>
	<B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>
Message-ID: <43c8685c0903061542y5f292ba5g54ae6c3377dce179@mail.gmail.com>

I don't mind whether its "in" or "out", but as a language user I think it's
best to minimise undocumented interfaces.
According to that principle, if it's "in", then it should also work as
documented (and be documented), and be "supported". If it's "out" then it
should either be removed entirely or be marked "private" (i.e. leading
underscore, unless I'm mistaking my style guidelines).

Cheers,
-T

On Sat, Mar 7, 2009 at 10:19 AM, Raymond Hettinger <python at rcn.com> wrote:

>
> [Martin v. L?wis]
>
>> I disagree that our users are served by constantly breaking the
>> API, and removing stuff just because we can. I can't see how
>> removing API can possibly serve a user.
>>
>
> Am not following you here.  My suggestion was to remove the two
> methods in Py3.1 which isn't even in alpha yet.  This is for a feature
> that has a simple substitute, was undocumented for Py3.0, and had
> long been documented in Py2.x as being unreliable.
>
> It's seems silly to me that an incomplete patch from a year ago
> would need to wait another two years to ever see the light of day
> (am presuming that 3.1 goes final this summer and that 3.2 follows
> 18 months later). That being said, I don't really care much.
> We don't actually have to do anything.  It could stay in forever
> and cause no harm.
>
>
> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/tleeuwenburg%40gmail.com
>



-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090307/db8f666e/attachment-0001.htm>

From collinw at gmail.com  Sat Mar  7 00:45:09 2009
From: collinw at gmail.com (Collin Winter)
Date: Fri, 6 Mar 2009 15:45:09 -0800
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <49B16517.3090207@alum.mit.edu>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
	<49B0F36C.8070800@alum.mit.edu>
	<loom.20090306T114142-800@post.gmane.org>
	<49B11AC9.70100@alum.mit.edu> <1236348355.6999.2.camel@fsol>
	<49B16517.3090207@alum.mit.edu>
Message-ID: <43aa6ff70903061545h4ee77d06ud5922896a7b30ec9@mail.gmail.com>

On Fri, Mar 6, 2009 at 10:01 AM, Michael Haggerty <mhagger at alum.mit.edu> wrote:
> Antoine Pitrou wrote:
>> Le vendredi 06 mars 2009 ? 13:44 +0100, Michael Haggerty a ?crit :
>>> Antoine Pitrou wrote:
>>>> Michael Haggerty <mhagger <at> alum.mit.edu> writes:
>>>>> It is easy to optimize the pickling of instances by giving them
>>>>> __getstate__() and __setstate__() methods. ?But the pickler still
>>>>> records the type of each object (essentially, the name of its class) in
>>>>> each record. ?The space for these strings constituted a large fraction
>>>>> of the database size.
>>>> If these strings are not interned, then perhaps they should be.
>>>> There is a similar optimization proposal (w/ patch) for attribute names:
>>>> http://bugs.python.org/issue5084
>>> If I understand correctly, this would not help:
>>>
>>> - on writing, the strings are identical anyway, because they are read
>>> out of the class's __name__ and __module__ fields. ?Therefore the
>>> Pickler's usual memoizing behavior will prevent the strings from being
>>> written more than once.
>>
>> Then why did you say that "the space for these strings constituted a
>> large fraction of the database size", if they are already shared? Are
>> your objects so tiny that even the space taken by the pointer to the
>> type name grows the size of the database significantly?
>
> Sorry for the confusion. ?I thought you were suggesting the change to
> help the more typical use case, when a single Pickler is used for a lot
> of data. ?That use case will not be helped by interning the class
> __name__ and __module__ strings, for the reasons given in my previous email.
>
> In my case, the strings are shared via the Pickler memoizing mechanism
> because I pre-populate the memo (using the API that the OP proposes to
> remove), so your suggestion won't help my current code, either. ?It was
> before I implemented the pre-populated memoizer that "the space for
> these strings constituted a large fraction of the database size". ?But
> your suggestion wouldn't help that case, either.
>
> Here are the main use cases:
>
> 1. Saving and loading one large record. ?A class's __name__ string is
> the same string object every time it is retrieved, so it only needs to
> be stored once and the Pickler memo mechanism works. ?Similarly for the
> class's __module__ string.
>
> 2. Saving and loading lots of records sequentially. ?Provided a single
> Pickler is used for all records and its memo is never cleared, this
> works just as well as case 1.
>
> 3. Saving and loading lots of records in random order, as for example in
> the shelve module. ?It is not possible to reuse a Pickler with retained
> memo, because the Unpickler might not encounter objects in the right
> order. ?There are two subcases:
>
> ? a. Use a clean Pickler/Unpickler object for each record. ?In this
> case the __name__ and __module__ of a class will appear once in each
> record in which the class appears. ?(This is the case regardless of
> whether they are interned.) ?On reading, the __name__ and __module__ are
> only used to look up the class, so interning them won't help. ?It is
> thus impossible to avoid wasting a lot of space in the database.
>
> ? b. Use a Pickler/Unpickler with a preset memo for each record (my
> unorthodox technique). ?In this case the class __name__ and __module__
> will be memoized in the shared memo, so in other records only their ID
> needs to be stored (in fact, only the ID of the class object itself).
> This allows the database to be smaller, but does not have any effect on
> the RAM usage of the loaded objects.
>
> If the OP's proposal is accepted, 3b will become impossible. ?The
> technique seems not to be well known, so maybe it doesn't need to be
> supported. ?It would mean some extra work for me on the cvs2svn project
> though :-(

Talking it over with Guido, support for the memo attribute will have
to stay. I shall add it back to my patches.

Collin

From ncoghlan at gmail.com  Sat Mar  7 01:05:19 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 07 Mar 2009 10:05:19 +1000
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <18865.15880.40623.112881@gargle.gargle.HOWL>
References: <mailman.29.1235991618.1178.python-dev@python.org>	<p06240803c5d1f82dbc4b@10.0.1.221>
	<49AC5C0B.4070806@v.loewis.de>	<e45086040903050553m44f7bce7xca474d658e0c977f@mail.gmail.com>	<49B047F1.9080900@v.loewis.de>
	<877i336jup.fsf@xemacs.org>	<43c8685c0903060238j238d2177kc189aa165697eb9d@mail.gmail.com>
	<18865.15880.40623.112881@gargle.gargle.HOWL>
Message-ID: <49B1BA3F.6090401@gmail.com>

Stephen J. Turnbull wrote:
> A suggestion, though.  View the contribution visualization video based
> on the commit log (the URL was posted here a while back, but I don't
> seem to have it offhand), which shows what a vibrant community this is
> in a very graphic way.

There's one here:
http://www.vimeo.com/1093745

That one runs up until just after the switch to subversion (as indicated
by the big influx of "new" names at the end, which is largely an
artifact of usernames changing from shortened forms to full names).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From martin at v.loewis.de  Sat Mar  7 01:19:25 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Mar 2009 01:19:25 +0100
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
 Queue.full()
In-Reply-To: <B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
	<49B19E54.3060407@v.loewis.de>
	<B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>
Message-ID: <49B1BD8D.4040209@v.loewis.de>

>> I disagree that our users are served by constantly breaking the
>> API, and removing stuff just because we can. I can't see how
>> removing API can possibly serve a user.
> 
> Am not following you here.  My suggestion was to remove the two
> methods in Py3.1 which isn't even in alpha yet.

Your proposal was also to add a warning for 3.0.2. This is what I
primarily object to.

> This is for a feature
> that has a simple substitute, was undocumented for Py3.0, and had
> long been documented in Py2.x as being unreliable.

The latter is not true. It was not documented as unreliable. Instead,
it was correctly documented as not being able, in general, to predict
the result of a subsequent put operation. In that sense, it is as
unreliable as the qsize() method, which remains supported and
documented.

Interestingly enough, the usage of .empty() in test_multiprocessing
is entirely safe, AFAICT. So whether the API is reliable or unreliable
very much depends on the application (as is true for many
multi-threading issues).

> It's seems silly to me that an incomplete patch from a year ago
> would need to wait another two years to ever see the light of day

Right. So it might be better to revert the patch, and restore the
documentation. I still fail to see the rationale for removing these
two methods.

Regards,
Martin

From jnoller at gmail.com  Sat Mar  7 01:33:58 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 6 Mar 2009 19:33:58 -0500
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
In-Reply-To: <49B1BD8D.4040209@v.loewis.de>
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>
	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
	<49B19E54.3060407@v.loewis.de>
	<B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>
	<49B1BD8D.4040209@v.loewis.de>
Message-ID: <4222a8490903061633t3bbb7e31laa608e628848ab08@mail.gmail.com>

On Fri, Mar 6, 2009 at 7:19 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> I disagree that our users are served by constantly breaking the
>>> API, and removing stuff just because we can. I can't see how
>>> removing API can possibly serve a user.
>>
>> Am not following you here. ?My suggestion was to remove the two
>> methods in Py3.1 which isn't even in alpha yet.
>
> Your proposal was also to add a warning for 3.0.2. This is what I
> primarily object to.
>
>> This is for a feature
>> that has a simple substitute, was undocumented for Py3.0, and had
>> long been documented in Py2.x as being unreliable.
>
> The latter is not true. It was not documented as unreliable. Instead,
> it was correctly documented as not being able, in general, to predict
> the result of a subsequent put operation. In that sense, it is as
> unreliable as the qsize() method, which remains supported and
> documented.
>
> Interestingly enough, the usage of .empty() in test_multiprocessing
> is entirely safe, AFAICT. So whether the API is reliable or unreliable
> very much depends on the application (as is true for many
> multi-threading issues).
>
>> It's seems silly to me that an incomplete patch from a year ago
>> would need to wait another two years to ever see the light of day
>
> Right. So it might be better to revert the patch, and restore the
> documentation. I still fail to see the rationale for removing these
> two methods.
>
> Regards,
> Martin

I would tend to agree with Martin, while it might be nice to
straightjacket the API into completely reliable calls (really, is
there anything like that with threads?) empty and the like when used
correctly work just fine. I think anyone using Queue with threads will
generally understand that size/empty calls will only be reliable when
put calls are completed. We can continue to warn them about the issues
with using it with continual producers, but as martin points out,
qsize suffers the same issue.

-jesse

From solipsis at pitrou.net  Sat Mar  7 01:44:56 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 7 Mar 2009 00:44:56 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?Forgotten_Py3=2E0_change_to_remove_Queue?=
	=?utf-8?b?LmVtcHR5KCkgYW5kCVF1ZXVlLmZ1bGwoKQ==?=
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>
	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
	<49B19E54.3060407@v.loewis.de>
	<B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>
	<49B1BD8D.4040209@v.loewis.de>
	<4222a8490903061633t3bbb7e31laa608e628848ab08@mail.gmail.com>
Message-ID: <loom.20090307T004356-79@post.gmane.org>

Jesse Noller <jnoller <at> gmail.com> writes:
> 
> I would tend to agree with Martin,

Agreed as well.

Antoine.



From python at rcn.com  Sat Mar  7 02:17:13 2009
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 6 Mar 2009 17:17:13 -0800
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
	<49B19E54.3060407@v.loewis.de>
	<B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>
	<49B1BD8D.4040209@v.loewis.de>
Message-ID: <B1E605FA06774F04AFF8827320D73D15@RaymondLaptop1>

>> Am not following you here.  My suggestion was to remove the two
>> methods in Py3.1 which isn't even in alpha yet.
>
> Your proposal was also to add a warning for 3.0.2. This is what I
> primarily object to.

Okay, that's fine.  Seemed prudent but it isn't essential.



>> This is for a feature
>> that has a simple substitute, was undocumented for Py3.0, and had
>> long been documented in Py2.x as being unreliable.
> 
> The latter is not true. It was not documented as unreliable.

You're right.  It was the docstring that said it was unreliable.
The regular docs were more specific about its limitations.



> I still fail to see the rationale for removing these
> two methods.

I believe there was a thread (in January 2008) with a 
decision to keep qsize() but to drop empty() and full().



Raymond

From guido at python.org  Sat Mar  7 03:15:48 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Mar 2009 18:15:48 -0800
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
In-Reply-To: <B1E605FA06774F04AFF8827320D73D15@RaymondLaptop1>
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>
	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
	<49B19E54.3060407@v.loewis.de>
	<B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>
	<49B1BD8D.4040209@v.loewis.de>
	<B1E605FA06774F04AFF8827320D73D15@RaymondLaptop1>
Message-ID: <ca471dc20903061815m35c4d729u3f28568758ba5820@mail.gmail.com>

On Fri, Mar 6, 2009 at 5:17 PM, Raymond Hettinger <python at rcn.com> wrote:
> I believe there was a thread (in January 2008) with a decision to keep
> qsize() but to drop empty() and full().

Based on our experiences so far, I think that of the two options we
have, both of which are bad, it's better to keep things in 3.1 that we
were planning to remove but forgot, than to make 3.1 have a whole slew
of additional removals. We've removed cmp() in 3.0.1, and I think that
was actually the right thing to do given its prominence and the clear
decision to remove it, but for smaller stuff that didn't make the cut
I think we should favor backwards compatibility over cleanup.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Sat Mar  7 03:31:34 2009
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 6 Mar 2009 18:31:34 -0800
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>
	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
	<49B19E54.3060407@v.loewis.de>
	<B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>
	<49B1BD8D.4040209@v.loewis.de>
	<B1E605FA06774F04AFF8827320D73D15@RaymondLaptop1>
	<ca471dc20903061815m35c4d729u3f28568758ba5820@mail.gmail.com>
Message-ID: <351E8B83239D4471BDD19BCF51610A80@RaymondLaptop1>


[Guido van Rossum]
> Based on our experiences so far, I think that of the two options we
> have, both of which are bad, it's better to keep things in 3.1 that we
> were planning to remove but forgot, than to make 3.1 have a whole slew
> of additional removals. We've removed cmp() in 3.0.1, and I think that
> was actually the right thing to do given its prominence and the clear
> decision to remove it, but for smaller stuff that didn't make the cut
> I think we should favor backwards compatibility over cleanup.

To some extent we get both by leaving them in the module
but continuing to be left out of the docs.  


Raymond

From lie.1296 at gmail.com  Sat Mar  7 08:32:07 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Sat, 07 Mar 2009 18:32:07 +1100
Subject: [Python-Dev] Python wins Linux New Media Award for Best Open
 Source Programming Language
In-Reply-To: <49B18168.7070703@v.loewis.de>
References: <49B11F5D.7050209@voidspace.org.uk>	<7c42eba10903060515y4238b5faw4f58e2c3de1158d7@mail.gmail.com>
	<49B18168.7070703@v.loewis.de>
Message-ID: <got7tu$een$1@ger.gmane.org>

Martin v. L?wis wrote:
>>> The prize was Martin von L?wis of the Python Foundation on behalf of the
>>> Python community itself.
>> This is a funny translation from German-to-English. :-)
>>
>> But yeah, a good one and the prize was presented by Klaus Knopper of Knoppix.
>>
>> Congratulations!
> 
> Actually, the prize went to "Python", not to me, and not to the PSF. So
> congratulations to you as well!

The (translated) article says that YOU are the prize? WOW.

Ummm... better not to use automatic translator for anything mission 
critical.


From martin at v.loewis.de  Sat Mar  7 10:14:11 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Mar 2009 10:14:11 +0100
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
 Queue.full()
In-Reply-To: <B1E605FA06774F04AFF8827320D73D15@RaymondLaptop1>
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
	<49B19E54.3060407@v.loewis.de>
	<B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>
	<49B1BD8D.4040209@v.loewis.de>
	<B1E605FA06774F04AFF8827320D73D15@RaymondLaptop1>
Message-ID: <49B23AE3.90702@v.loewis.de>

>> I still fail to see the rationale for removing these
>> two methods.
> 
> I believe there was a thread (in January 2008) with a decision to keep
> qsize() but to drop empty() and full().

That's something different: even if there was a decision, it doesn't
mean that there was a rationale, and that the rationale was correct.
I see that you proposed to remove the methods because they are
unreliable. I still think this classification is incorrect: they are
*not* unreliable. They do what they do correctly and reliably.

Looking at the discussion (on python-3000), I don't see that an
actual decision was made. You proposed it, there was a *lot* of
objections. Guido proposed a compromise (without indicating whether
he was in favor of the removal in the first place). There was then
still opposition both to leaving qsize, and to change anything at
all - and then you removed the methods.

At this point, I do request that the patch is reverted completely
(i.e. that the documentation is restored), and that the qualification
"not reliable!" is removed from the doc strings of the methods, as
it is factually incorrect.

Regards,
Martin

From solipsis at pitrou.net  Sat Mar  7 11:14:13 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 7 Mar 2009 10:14:13 +0000 (UTC)
Subject: [Python-Dev] Regexp 2.7
Message-ID: <loom.20090307T101047-371@post.gmane.org>

Hello,

Matthew Barnett has been doing a lot of work on the regular expressions engine
(it seems he hasn't finished yet) under http://bugs.python.org/issue2636.
However, the patches are really huge and touch all of the sre internals. I
wonder what the review process can be for such patches? Is there someone
knowledgeable enough to be able to review them?

Regards

Antoine.



From martin at v.loewis.de  Sat Mar  7 12:13:57 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Mar 2009 12:13:57 +0100
Subject: [Python-Dev] Regexp 2.7
In-Reply-To: <loom.20090307T101047-371@post.gmane.org>
References: <loom.20090307T101047-371@post.gmane.org>
Message-ID: <49B256F5.6050004@v.loewis.de>

> Matthew Barnett has been doing a lot of work on the regular expressions engine
> (it seems he hasn't finished yet) under http://bugs.python.org/issue2636.
> However, the patches are really huge and touch all of the sre internals. I
> wonder what the review process can be for such patches? Is there someone
> knowledgeable enough to be able to review them?

Traditionally, Fredrik Lundh has reviewed sre patches. Not sure whether
he is interested in that anymore, though.

Regards,
Martin

From greg.ewing at canterbury.ac.nz  Sat Mar  7 12:40:46 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 08 Mar 2009 00:40:46 +1300
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <49B0F36C.8070800@alum.mit.edu>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
	<49B0F36C.8070800@alum.mit.edu>
Message-ID: <49B25D3E.1040806@canterbury.ac.nz>

Michael Haggerty wrote:

> A similar effect could *almost* be obtained without accessing the memos
> by saving the pickled primer itself in the database.  The unpickler
> would be primed by using it to load the primer before loading the record
> of interest.  But AFAIK there is no way to prime new picklers, because
> there is no guarantee that pickling the same primer twice will result in
> the same id->object mapping in the pickler's memo.

Would it help if, when creating a pickler or unpickler,
you could specify another pickler or unpickler whose
memo is used to initialise the memo of the new one?

Then you could keep the pickler that you used to pickle
the primer and "fork" new picklers off it, and similarly
with the unpicklers.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Sat Mar  7 12:48:35 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 08 Mar 2009 00:48:35 +1300
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <loom.20090306T114142-800@post.gmane.org>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
	<49B0F36C.8070800@alum.mit.edu>
	<loom.20090306T114142-800@post.gmane.org>
Message-ID: <49B25F13.4070800@canterbury.ac.nz>

Antoine Pitrou wrote:

> If these strings are not interned, then perhaps they should be.

I think this is a different problem. Even if the strings are
interned, if you start with a fresh pickler each time, you
get a copy of the strings in each pickle. What he wants is
to share strings between different pickles.

-- 
Greg

From techtonik at gmail.com  Sat Mar  7 12:54:20 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Sat, 7 Mar 2009 13:54:20 +0200
Subject: [Python-Dev] Python wins Linux New Media Award for Best Open
	Source Programming Language
In-Reply-To: <got7tu$een$1@ger.gmane.org>
References: <49B11F5D.7050209@voidspace.org.uk>
	<7c42eba10903060515y4238b5faw4f58e2c3de1158d7@mail.gmail.com>
	<49B18168.7070703@v.loewis.de> <got7tu$een$1@ger.gmane.org>
Message-ID: <d34314100903070354i6bf5512eqedc9fbe9d4de810c@mail.gmail.com>

On Sat, Mar 7, 2009 at 9:32 AM, Lie Ryan <lie.1296 at gmail.com> wrote:
>>>>
>>>> The prize was Martin von L?wis of the Python Foundation on behalf of the
>>>> Python community itself.
>>>
>>> This is a funny translation from German-to-English. :-)
>>>
>>> But yeah, a good one and the prize was presented by Klaus Knopper of
>>> Knoppix.
>>>
>>> Congratulations!
>>
>> Actually, the prize went to "Python", not to me, and not to the PSF. So
>> congratulations to you as well!
>
> The (translated) article says that YOU are the prize? WOW.
>
> Ummm... better not to use automatic translator for anything mission
> critical.
>

Congratulations! There is already a proper English version of press
release at http://www.linux-magazine.com/online/news/cebit_2009_openstreetmap_wins_two_linux_new_media_awards
so new link can probably be changed. The article still doesn't mention
competitors.

-- 
--anatoly t.

From solipsis at pitrou.net  Sat Mar  7 15:07:48 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 7 Mar 2009 14:07:48 +0000 (UTC)
Subject: [Python-Dev] 3.1 performance
Message-ID: <loom.20090307T140318-788@post.gmane.org>

Hello,

Out of curiosity, I timed running the test suite ("./python -m test.regrtest")
in non-debug mode, in both the release30-maint and py3k branches:

* release30-maint got:
302 tests OK.
[...]
165.79user 26.03system 5:01.75elapsed 63%CPU

* py3k got:
304 tests OK.
[...]
113.33user 28.93system 4:06.79elapsed 57%CPU

So, 3.1 is 30% faster in user CPU time, even though it probably has more tests
(because of io-c, ordereddict and importlib). This is on a 64-bit Linux AMD
system, and I got similar results on a 64-bit Linux Core2 system.

Regards

Antoine.



From python at rcn.com  Sat Mar  7 15:09:26 2009
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 7 Mar 2009 06:09:26 -0800
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
	Queue.full()
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
	<49B19E54.3060407@v.loewis.de>
	<B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>
	<49B1BD8D.4040209@v.loewis.de>
	<B1E605FA06774F04AFF8827320D73D15@RaymondLaptop1>
	<49B23AE3.90702@v.loewis.de>
Message-ID: <D5ECF795F36E49E6BB5637B52430838F@RaymondLaptop1>


[MvL]
> At this point, I do request that the patch is reverted completely
> (i.e. that the documentation is restored), and that the qualification
> "not reliable!" is removed from the doc strings of the methods, as
> it is factually incorrect.

I would be happy to restore the documentation.  You want the methods
back and I think that's sufficient reason to bring it back.

The "not reliable" wording in the docstrings was put there by Guido 16 years ago. 
http://svn.python.org/view/python/trunk/Lib/Queue.py?revision=3158&view=markup
Speak to him about this.  I'm done with this thread.


Raymond

From ocean-city at m2.ccsnet.ne.jp  Sat Mar  7 15:24:55 2009
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Sat, 07 Mar 2009 23:24:55 +0900
Subject: [Python-Dev] 3.1 performance
In-Reply-To: <loom.20090307T140318-788@post.gmane.org>
References: <loom.20090307T140318-788@post.gmane.org>
Message-ID: <49B283B7.8070102@m2.ccsnet.ne.jp>

Antoine Pitrou wrote:
> Hello,
> 
> Out of curiosity, I timed running the test suite ("./python -m test.regrtest")
> in non-debug mode, in both the release30-maint and py3k branches:
> 
> * release30-maint got:
> 302 tests OK.
> [...]
> 165.79user 26.03system 5:01.75elapsed 63%CPU
> 
> * py3k got:
> 304 tests OK.
> [...]
> 113.33user 28.93system 4:06.79elapsed 57%CPU
> 
> So, 3.1 is 30% faster in user CPU time, even though it probably has more tests
> (because of io-c, ordereddict and importlib). This is on a 64-bit Linux AMD
> system, and I got similar results on a 64-bit Linux Core2 system.
> 
> Regards
> 
> Antoine.

Yes, traceback in large file is also quite fast now. Good work, io-c 
guys. :-)

From python at rcn.com  Sat Mar  7 15:54:03 2009
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 7 Mar 2009 06:54:03 -0800
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty()
	andQueue.full()
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com><9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1><49B19E54.3060407@v.loewis.de><B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1><49B1BD8D.4040209@v.loewis.de><B1E605FA06774F04AFF8827320D73D15@RaymondLaptop1><49B23AE3.90702@v.loewis.de>
	<D5ECF795F36E49E6BB5637B52430838F@RaymondLaptop1>
Message-ID: <1EFA4331D3014B73BC1157E0CB785273@RaymondLaptop1>


> [MvL]
>> At this point, I do request that the patch is reverted completely
>> (i.e. that the documentation is restored), and that the qualification
>> "not reliable!" is removed from the doc strings of the methods, as
>> it is factually incorrect.
> 
> I would be happy to restore the documentation.  You want the methods
> back and I think that's sufficient reason to bring it back.
> 
> The "not reliable" wording in the docstrings was put there by Guido 16 years ago. 
> http://svn.python.org/view/python/trunk/Lib/Queue.py?revision=3158&view=markup
> Speak to him about this.  I'm done with this thread.

Also, for years, the documentation also had a "not-reliable" notice:

    Return ``True`` if the queue is empty, ``False`` otherwise. 
    Because of multithreading semantics, this is not reliable.

Skip changed it to the present wording last year:
http://svn.python.org/view/python/trunk/Doc/library/queue.rst?r1=59750&r2=59969


Raymond


From dan.mahn at digidescorp.com  Sat Mar  7 15:56:32 2009
From: dan.mahn at digidescorp.com (Dan Mahn)
Date: Sat, 07 Mar 2009 08:56:32 -0600
Subject: [Python-Dev] More on Py3K urllib -- urlencode()
In-Reply-To: <35441.1235863997@parc.com>
References: <49A99E7B.7030602@digidescorp.com> <34291.1235858913@parc.com>
	<35441.1235863997@parc.com>
Message-ID: <49B28B20.7000509@digidescorp.com>

After a harder look, I concluded there was a bit more work to be done, 
but still very basic modifications.

Attached is a version of urlencode() which seems to make the most sense 
to me.

I wonder how I could officially propose at least some of these 
modifications.

- Dan



Bill Janssen wrote:
> Bill Janssen <janssen at parc.com> wrote:
>
>   
>> Dan Mahn <dan.mahn at digidescorp.com> wrote:
>>
>>     
>>> 3) Regarding the following code fragment in urlencode():
>>>
>>>            k = quote_plus(str(k))
>>>           if isinstance(v, str):
>>>                v = quote_plus(v)
>>>                l.append(k + '=' + v)
>>>            elif isinstance(v, str):
>>>                # is there a reasonable way to convert to ASCII?
>>>                # encode generates a string, but "replace" or "ignore"
>>>                # lose information and "strict" can raise UnicodeError
>>>                v = quote_plus(v.encode("ASCII","replace"))
>>>                l.append(k + '=' + v)
>>>
>>> I don't understand how the "elif" section is invoked, as it uses the
>>> same condition as the "if" section.
>>>       
>> This looks like a 2->3 bug; clearly only the second branch should be
>> used in Py3K.  And that "replace" is also a bug; it should signal an
>> error on encoding failures.  It should probably catch UnicodeError and
>> explain the problem, which is that only Latin-1 values can be passed in
>> the query string.  So the encode() to "ASCII" is also a mistake; it
>> should be "ISO-8859-1", and the "replace" should be a "strict", I think.
>>     
>
> Sorry!  In 3.0.1, this whole thing boils down to
>
>    l.append(quote_plus(k) + '=' + quote_plus(v))
>
> Bill
>   
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: new_urlencode.py
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090307/5175bcb1/attachment.txt>

From aahz at pythoncraft.com  Sat Mar  7 16:24:24 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 7 Mar 2009 07:24:24 -0800
Subject: [Python-Dev] More on Py3K urllib -- urlencode()
In-Reply-To: <49B28B20.7000509@digidescorp.com>
References: <49A99E7B.7030602@digidescorp.com> <34291.1235858913@parc.com>
	<35441.1235863997@parc.com> <49B28B20.7000509@digidescorp.com>
Message-ID: <20090307152424.GA9464@panix.com>

On Sat, Mar 07, 2009, Dan Mahn wrote:
>
> After a harder look, I concluded there was a bit more work to be done,  
> but still very basic modifications.
>
> Attached is a version of urlencode() which seems to make the most sense  
> to me.
>
> I wonder how I could officially propose at least some of these  
> modifications.

Submit a patch to bugs.python.org
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of     
indirection."  --Butler Lampson

From guido at python.org  Sat Mar  7 16:36:41 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Mar 2009 07:36:41 -0800
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty()
	andQueue.full()
In-Reply-To: <1EFA4331D3014B73BC1157E0CB785273@RaymondLaptop1>
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>
	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
	<49B19E54.3060407@v.loewis.de>
	<B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>
	<49B1BD8D.4040209@v.loewis.de>
	<B1E605FA06774F04AFF8827320D73D15@RaymondLaptop1>
	<49B23AE3.90702@v.loewis.de>
	<D5ECF795F36E49E6BB5637B52430838F@RaymondLaptop1>
	<1EFA4331D3014B73BC1157E0CB785273@RaymondLaptop1>
Message-ID: <ca471dc20903070736q28c294ahcb7c692233f688e4@mail.gmail.com>

On Sat, Mar 7, 2009 at 6:54 AM, Raymond Hettinger <python at rcn.com> wrote:
>
>> [MvL]
>>>
>>> At this point, I do request that the patch is reverted completely
>>> (i.e. that the documentation is restored), and that the qualification
>>> "not reliable!" is removed from the doc strings of the methods, as
>>> it is factually incorrect.
>>
>> I would be happy to restore the documentation. ?You want the methods
>> back and I think that's sufficient reason to bring it back.
>>
>> The "not reliable" wording in the docstrings was put there by Guido 16
>> years ago.
>> http://svn.python.org/view/python/trunk/Lib/Queue.py?revision=3158&view=markup
>> Speak to him about this. ?I'm done with this thread.
>
> Also, for years, the documentation also had a "not-reliable" notice:
>
> ? Return ``True`` if the queue is empty, ``False`` otherwise. ? Because of
> multithreading semantics, this is not reliable.
>
> Skip changed it to the present wording last year:
> http://svn.python.org/view/python/trunk/Doc/library/queue.rst?r1=59750&r2=59969

That was a long time ago, but I think I just meant the obvious warning
about multi-threaded use: if you are using full() and empty() and
other threads may also mutate the queue, then you cannot use them to
reliably prevent you from blocking. That's why eventually the
facilities for non-blocking get() and put() were added (I think by Tim
Peters).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mhagger at alum.mit.edu  Sat Mar  7 17:04:04 2009
From: mhagger at alum.mit.edu (Michael Haggerty)
Date: Sat, 07 Mar 2009 17:04:04 +0100
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <49B25D3E.1040806@canterbury.ac.nz>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>	<49B0F36C.8070800@alum.mit.edu>
	<49B25D3E.1040806@canterbury.ac.nz>
Message-ID: <49B29AF4.5050407@alum.mit.edu>

Greg Ewing wrote:
> Michael Haggerty wrote:
>> A similar effect could *almost* be obtained without accessing the memos
>> by saving the pickled primer itself in the database.  The unpickler
>> would be primed by using it to load the primer before loading the record
>> of interest.  But AFAIK there is no way to prime new picklers, because
>> there is no guarantee that pickling the same primer twice will result in
>> the same id->object mapping in the pickler's memo.
> 
> Would it help if, when creating a pickler or unpickler,
> you could specify another pickler or unpickler whose
> memo is used to initialise the memo of the new one?
> 
> Then you could keep the pickler that you used to pickle
> the primer and "fork" new picklers off it, and similarly
> with the unpicklers.

Typically, the purpose of a database is to persist data across program
runs.  So typically, your suggestion would only help if there were a way
to persist the primed Pickler across runs.

(The primed Unpickler is not quite so important because it can be primed
by reading a pickle of the primer, which in turn can be stored somewhere
in the DB.)

In the particular case of cvs2svn, each of our databases is in fact
written in a single pass, and then in later passes only read, not
written.  So I suppose we could do entirely without pickleable Picklers,
if they were copyable within a single program run.  But that constraint
would make the feature even less general.

Michael

From guido at python.org  Sat Mar  7 17:27:20 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Mar 2009 08:27:20 -0800
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <49B29AF4.5050407@alum.mit.edu>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
	<49B0F36C.8070800@alum.mit.edu> <49B25D3E.1040806@canterbury.ac.nz>
	<49B29AF4.5050407@alum.mit.edu>
Message-ID: <ca471dc20903070827s52644dbcp6586bdcde210aae3@mail.gmail.com>

On Sat, Mar 7, 2009 at 8:04 AM, Michael Haggerty <mhagger at alum.mit.edu> wrote:
> Typically, the purpose of a database is to persist data across program
> runs. ?So typically, your suggestion would only help if there were a way
> to persist the primed Pickler across runs.

I haven't followed all this, but isn't is at least possible to
conceive of the primed pickler as being recreated from scratch from
constant data each run?

> (The primed Unpickler is not quite so important because it can be primed
> by reading a pickle of the primer, which in turn can be stored somewhere
> in the DB.)
>
> In the particular case of cvs2svn, each of our databases is in fact
> written in a single pass, and then in later passes only read, not
> written. ?So I suppose we could do entirely without pickleable Picklers,
> if they were copyable within a single program run. ?But that constraint
> would make the feature even less general.

Being copyable is mostly equivalent to being picklable, but it's
probably somewhat weaker because it's easier to define it as a pointer
copy for some types that aren't easily picklable.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rdmurray at bitdance.com  Sat Mar  7 17:37:02 2009
From: rdmurray at bitdance.com (rdmurray at bitdance.com)
Date: Sat, 7 Mar 2009 11:37:02 -0500 (EST)
Subject: [Python-Dev] Belated introduction
In-Reply-To: <49B17891.50801@v.loewis.de>
References: <mailman.29.1235991618.1178.python-dev@python.org> 
	<p06240803c5d1f82dbc4b@10.0.1.221> <49AC5C0B.4070806@v.loewis.de> 
	<e45086040903050553m44f7bce7xca474d658e0c977f@mail.gmail.com> 
	<49B047F1.9080900@v.loewis.de> <877i336jup.fsf@xemacs.org>
	<43c8685c0903060238j238d2177kc189aa165697eb9d@mail.gmail.com>
	<49B17891.50801@v.loewis.de>
Message-ID: <Pine.LNX.4.64.0903071109050.12396@kimball.webabinitio.net>

I've been watching the threads about tracker maintenance and patch
review with interest.  I'm afraid that I did not follow the list
recommendation to introduce myself when I first started posting, partly
because I initially jumped in on something that was a bit of a hot
button issue for me :)

So, a little belatedly, here is my intro.  I've been coding in and
loving Python since 1993 or thereabouts.  I am currently an independent
IT consultant, with specialization in IP networking (especially Cisco),
Lunix and Unix system administration, and systems integration.  It is in
my systems integration work that I make heavy use of Python, writing
scripts and programs to tie systems together and/or add needed
functionality.

In my previous life I was Director of Technology for an ISP, and at one
stage we were heavy into web site development.  At that time I worked
with the Zope community quite a bit, and made some contributions to the
beginnings of Zope3.  So for at least a few people this might be more of
a re-introduction.

I've decided that this year I would like to get more involved in the
Python community.  As another poster said, I'm looking forward to the
opportunity to work with and learn from some very smart people.  And
then there's the satisfaction of giving back to a community that has
given me so much over the years, by producing such a delightful language
in which to do my coding.

So, I've been reading the developer docs, setting up a bzr checkout,
learning how to compile a debug python and run tests, wandering around
in the source code, etc, etc.

My thought about how I could best contribute at the present time is to
help out with reviewing tracker issues.  I actually updated the open
tracker issue with the oldest last activity date this morning, as a sort
of trial run.  Unless someone wants to suggest a different way for me to
contribute (I'm open to any suggestions), I'll probably continue through
the list in reverse order of last update date and cherry pick things
that interest me and that I think I can make some sort of contribution to.

Don't hesitate to let me know if I miss etiquette points (gently, please
:).

Oh, and I'll be at pycon this year (my second pycon, the first was
several years back), and look forward to hanging out with a cool bunch
of people :)

--RDM

From aahz at pythoncraft.com  Sat Mar  7 17:42:34 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 7 Mar 2009 08:42:34 -0800
Subject: [Python-Dev] Belated introduction
In-Reply-To: <Pine.LNX.4.64.0903071109050.12396@kimball.webabinitio.net>
References: <mailman.29.1235991618.1178.python-dev@python.org>
	<p06240803c5d1f82dbc4b@10.0.1.221> <49AC5C0B.4070806@v.loewis.de>
	<e45086040903050553m44f7bce7xca474d658e0c977f@mail.gmail.com>
	<49B047F1.9080900@v.loewis.de> <877i336jup.fsf@xemacs.org>
	<43c8685c0903060238j238d2177kc189aa165697eb9d@mail.gmail.com>
	<49B17891.50801@v.loewis.de>
	<Pine.LNX.4.64.0903071109050.12396@kimball.webabinitio.net>
Message-ID: <20090307164234.GA1330@panix.com>

On Sat, Mar 07, 2009, rdmurray at bitdance.com wrote:
>
> So, a little belatedly, here is my intro.  [...]
>
> --RDM

Welcome!  You apparently haven't set your $NAME nor listed a name in your
.sig, so how do you prefer to be addressed?  Or do you just prefer your
initials, like RMS?  ;-)
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of     
indirection."  --Butler Lampson

From aahz at pythoncraft.com  Sat Mar  7 17:49:11 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 7 Mar 2009 08:49:11 -0800
Subject: [Python-Dev] IMAP/POP3 servers on python.org?
In-Reply-To: <86740.1236374621@parc.com>
References: <45876.1235939555@parc.com> <49AB0DAA.6010103@holdenweb.com>
	<48701.1235947609@parc.com> <49AB166F.3070602@voidspace.org.uk>
	<86740.1236374621@parc.com>
Message-ID: <20090307164911.GC1330@panix.com>

[moving discussion to python-dev from pydotorg]

On Fri, Mar 06, 2009, Bill Janssen wrote:
> Michael Foord <michael at voidspace.org.uk> wrote:
>> Bill Janssen wrote:
>>> Steve Holden <steve at holdenweb.com> wrote:
>>>>   
>>>> Seems to me it might be better to have a test start a local server then
>>>> kill it, but I am presuming you have some good reason why this is not
>>>> practical?
>>>     
>>> It seems a steep learning curve just to run the Python test suite, to
>>> have to know how to install an IMAP server on your machine.  But perhaps
>>> you're right.
>>   
>> Does it need to test against a real server - can't some of the lower
>> calls be mocked out?
> 
> Not really.  These *are* the lower-level calls.

One thing I haven't seen addressed in this discussion is why it's
undesirable to ship Twisted as part of the testing source tree.  Yes, I
realize that it could create an "attractive nuisance", but I think the
gains in simplifying testing outweigh that.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of     
indirection."  --Butler Lampson

From solipsis at pitrou.net  Sat Mar  7 18:04:47 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 7 Mar 2009 17:04:47 +0000 (UTC)
Subject: [Python-Dev] IMAP/POP3 servers on python.org?
References: <45876.1235939555@parc.com> <49AB0DAA.6010103@holdenweb.com>
	<48701.1235947609@parc.com> <49AB166F.3070602@voidspace.org.uk>
	<86740.1236374621@parc.com> <20090307164911.GC1330@panix.com>
Message-ID: <loom.20090307T165638-943@post.gmane.org>

Aahz <aahz <at> pythoncraft.com> writes:
> 
> One thing I haven't seen addressed in this discussion is why it's
> undesirable to ship Twisted as part of the testing source tree.

Because Twisted is huge and it's not reasonable to include it just for testing
purposes?
(besides, launching a twisted server implies running a reactor and I'm not sure
the reactors support being started and stopped several times yet)



From rdmurray at bitdance.com  Sat Mar  7 19:04:04 2009
From: rdmurray at bitdance.com (rdmurray at bitdance.com)
Date: Sat, 7 Mar 2009 13:04:04 -0500 (EST)
Subject: [Python-Dev] Belated introduction
In-Reply-To: <20090307164234.GA1330@panix.com>
References: <mailman.29.1235991618.1178.python-dev@python.org>
	<p06240803c5d1f82dbc4b@10.0.1.221> <49AC5C0B.4070806@v.loewis.de>
	<e45086040903050553m44f7bce7xca474d658e0c977f@mail.gmail.com>
	<49B047F1.9080900@v.loewis.de> <877i336jup.fsf@xemacs.org>
	<43c8685c0903060238j238d2177kc189aa165697eb9d@mail.gmail.com>
	<49B17891.50801@v.loewis.de>
	<Pine.LNX.4.64.0903071109050.12396@kimball.webabinitio.net>
	<20090307164234.GA1330@panix.com>
Message-ID: <Pine.LNX.4.64.0903071257410.12396@kimball.webabinitio.net>

On Sat, 7 Mar 2009 at 08:42, Aahz wrote:
> On Sat, Mar 07, 2009, rdmurray at bitdance.com wrote:
>>
>> So, a little belatedly, here is my intro.  [...]
>>
>> --RDM
>
> Welcome!  You apparently haven't set your $NAME nor listed a name in your
> .sig, so how do you prefer to be addressed?  Or do you just prefer your
> initials, like RMS?  ;-)

Thanks.

Initials is fine, but I'm more used to 'David' as form of address.  I
have been using my initials in my sig because there are just too many
Davids around.

I suppose changing my sig to my name and my website wouldn't be
be a bad thing, so...

--
R. David Murray           http://www.bitdance.com

From mhagger at alum.mit.edu  Sat Mar  7 19:22:44 2009
From: mhagger at alum.mit.edu (Michael Haggerty)
Date: Sat, 07 Mar 2009 19:22:44 +0100
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <ca471dc20903070827s52644dbcp6586bdcde210aae3@mail.gmail.com>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>	
	<49B0F36C.8070800@alum.mit.edu>
	<49B25D3E.1040806@canterbury.ac.nz>	
	<49B29AF4.5050407@alum.mit.edu>
	<ca471dc20903070827s52644dbcp6586bdcde210aae3@mail.gmail.com>
Message-ID: <49B2BB74.60001@alum.mit.edu>

Guido van Rossum wrote:
> On Sat, Mar 7, 2009 at 8:04 AM, Michael Haggerty <mhagger at alum.mit.edu> wrote:
>> Typically, the purpose of a database is to persist data across program
>> runs.  So typically, your suggestion would only help if there were a way
>> to persist the primed Pickler across runs.
> 
> I haven't followed all this, but isn't is at least possible to
> conceive of the primed pickler as being recreated from scratch from
> constant data each run?

If there were a guarantee that pickling the same data would result in
the same memo ID -> object mapping, that would also work.  But that
doesn't seem to be a realistic guarantee to make.  AFAIK the memo IDs
are integers chosen consecutively in the order that objects are pickled,
which doesn't seem so bad.  But compound objects are a problem.  For
example, when pickling a map, the map entries would have to be pickled
in an order that remains consistent across runs (and even across Python
versions).  Even worse, all user-written __getstate__() methods would
have to return exactly the same result, even across program runs.

>> (The primed Unpickler is not quite so important because it can be primed
>> by reading a pickle of the primer, which in turn can be stored somewhere
>> in the DB.)
>>
>> In the particular case of cvs2svn, each of our databases is in fact
>> written in a single pass, and then in later passes only read, not
>> written.  So I suppose we could do entirely without pickleable Picklers,
>> if they were copyable within a single program run.  But that constraint
>> would make the feature even less general.
> 
> Being copyable is mostly equivalent to being picklable, but it's
> probably somewhat weaker because it's easier to define it as a pointer
> copy for some types that aren't easily picklable.

Indeed.  And pickling the memo should not present any fundamental
problems, since by construction it can only contain pickleable objects.

Michael

From janssen at parc.com  Sat Mar  7 19:36:22 2009
From: janssen at parc.com (Bill Janssen)
Date: Sat, 7 Mar 2009 10:36:22 PST
Subject: [Python-Dev] More on Py3K urllib -- urlencode()
In-Reply-To: <20090307152424.GA9464@panix.com>
References: <49A99E7B.7030602@digidescorp.com> <34291.1235858913@parc.com>
	<35441.1235863997@parc.com> <49B28B20.7000509@digidescorp.com>
	<20090307152424.GA9464@panix.com>
Message-ID: <98678.1236450982@parc.com>

Aahz <aahz at pythoncraft.com> wrote:

> On Sat, Mar 07, 2009, Dan Mahn wrote:
> >
> > After a harder look, I concluded there was a bit more work to be done,  
> > but still very basic modifications.
> >
> > Attached is a version of urlencode() which seems to make the most sense  
> > to me.
> >
> > I wonder how I could officially propose at least some of these  
> > modifications.
> 
> Submit a patch to bugs.python.org

And it would help if it included a lot of test cases.

Bill

From benjamin at python.org  Sat Mar  7 19:53:28 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 7 Mar 2009 12:53:28 -0600
Subject: [Python-Dev] RELEASED Python 3.1 alpha 1
Message-ID: <1afaf6160903071053i559f5608nb37882f680ed41d0@mail.gmail.com>

On behalf of the Python development team and the Python community, I'm
happy to announce the first alpha release of Python 3.1.

Python 3.1 focuses on the stabilization and optimization of features and changes
Python 3.0 introduced.  The new I/O system has been rewritten in C for speed.
Other features include a ordered dictionary implementation and support for ttk
Tile in Tkinter.

Please note that these are alpha releases, and as such are not suitable for
production environments.  We continue to strive for a high degree of quality,
but there are still some known problems and the feature sets have not been
finalized.  These alphas are being released to solicit feedback and hopefully
discover bugs, as well as allowing you to determine how changes in 3.1 might
impact you.  If you find things broken or incorrect, please submit a bug report
at

     http://bugs.python.org

For more information and downloads, see the Python 3.1 website:

     http://www.python.org/download/releases/3.1/

See PEP 375 for release schedule details:

     http://www.python.org/dev/peps/pep-0361/


Enjoy,
-- Benjamin

Benjamin Peterson
benjamin at python.org
Release Manager
(on behalf of the entire python-dev team)

From grflanagan at gmail.com  Sat Mar  7 20:25:16 2009
From: grflanagan at gmail.com (Gerard Flanagan)
Date: Sat, 07 Mar 2009 19:25:16 +0000
Subject: [Python-Dev] RELEASED Python 3.1 alpha 1
In-Reply-To: <1afaf6160903071053i559f5608nb37882f680ed41d0@mail.gmail.com>
References: <1afaf6160903071053i559f5608nb37882f680ed41d0@mail.gmail.com>
Message-ID: <gouhms$lfu$1@ger.gmane.org>

Benjamin Peterson wrote:
> On behalf of the Python development team and the Python community, I'm
> happy to announce the first alpha release of Python 3.1.
> 
[...]

Looks good, thanks to all involved. Two minor things:

> 
> For more information and downloads, see the Python 3.1 website:
> 
>      http://www.python.org/download/releases/3.1/
>

On the release page, the bzip link says '3.0' not '3.1'.

> See PEP 375 for release schedule details:
> 
>      http://www.python.org/dev/peps/pep-0361/
> 

On the PEP page, the Google calendar link is borken.

Regards

G.



From aahz at pythoncraft.com  Sat Mar  7 21:10:48 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 7 Mar 2009 12:10:48 -0800
Subject: [Python-Dev] RELEASED Python 3.1 alpha 1
In-Reply-To: <1afaf6160903071053i559f5608nb37882f680ed41d0@mail.gmail.com>
References: <1afaf6160903071053i559f5608nb37882f680ed41d0@mail.gmail.com>
Message-ID: <20090307201048.GA23840@panix.com>

On Sat, Mar 07, 2009, Benjamin Peterson wrote:
>
> On behalf of the Python development team and the Python community, I'm
> happy to announce the first alpha release of Python 3.1.

Congratulations on your first baby!  Here's to hoping you release many
more of these!
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of     
indirection."  --Butler Lampson

From benjamin at python.org  Sat Mar  7 21:39:24 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 7 Mar 2009 14:39:24 -0600
Subject: [Python-Dev] RELEASED Python 3.1 alpha 1
In-Reply-To: <gouhms$lfu$1@ger.gmane.org>
References: <1afaf6160903071053i559f5608nb37882f680ed41d0@mail.gmail.com>
	<gouhms$lfu$1@ger.gmane.org>
Message-ID: <1afaf6160903071239g494d5252ocfc1aa7d4261b77@mail.gmail.com>

2009/3/7 Gerard Flanagan <grflanagan at gmail.com>:
> Benjamin Peterson wrote:
> On the release page, the bzip link says '3.0' not '3.1'.

That should be fixed now.

>
>> See PEP 375 for release schedule details:
>>
>> ? ? http://www.python.org/dev/peps/pep-0361/

That URL is actually supposed to be http://www.python.org/dev/peps/pep-0375/.



-- 
Regards,
Benjamin

From python at rcn.com  Sat Mar  7 22:21:33 2009
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 7 Mar 2009 13:21:33 -0800
Subject: [Python-Dev] RELEASED Python 3.1 alpha 1
References: <1afaf6160903071053i559f5608nb37882f680ed41d0@mail.gmail.com>
Message-ID: <51306117231E465E9FBE6EFB4D1B480A@RaymondLaptop1>


[Benjamin Peterson]
> On behalf of the Python development team and the Python community, I'm
> happy to announce the first alpha release of Python 3.1.

Thanks for the good work.


Raymond

From martin at v.loewis.de  Sat Mar  7 23:50:07 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Mar 2009 23:50:07 +0100
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty() and
 Queue.full()
In-Reply-To: <D5ECF795F36E49E6BB5637B52430838F@RaymondLaptop1>
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
	<49B19E54.3060407@v.loewis.de>
	<B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>
	<49B1BD8D.4040209@v.loewis.de>
	<B1E605FA06774F04AFF8827320D73D15@RaymondLaptop1>
	<49B23AE3.90702@v.loewis.de>
	<D5ECF795F36E49E6BB5637B52430838F@RaymondLaptop1>
Message-ID: <49B2FA1F.8090902@v.loewis.de>

> I would be happy to restore the documentation.  You want the methods
> back and I think that's sufficient reason to bring it back.

Thanks! I'll look into the docstrings.

Martin

From martin at v.loewis.de  Sat Mar  7 23:56:21 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Mar 2009 23:56:21 +0100
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty()
	andQueue.full()
In-Reply-To: <1EFA4331D3014B73BC1157E0CB785273@RaymondLaptop1>
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>	<1afaf6160903041704h45e09c3fp4c5c699adf3d950f@mail.gmail.com><9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1><49B19E54.3060407@v.loewis.de><B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1><49B1BD8D.4040209@v.loewis.de><B1E605FA06774F04AFF8827320D73D15@RaymondLaptop1><49B23AE3.90702@v.loewis.de>
	<D5ECF795F36E49E6BB5637B52430838F@RaymondLaptop1>
	<1EFA4331D3014B73BC1157E0CB785273@RaymondLaptop1>
Message-ID: <49B2FB95.60406@v.loewis.de>

> Skip changed it to the present wording last year:
> http://svn.python.org/view/python/trunk/Doc/library/queue.rst?r1=59750&r2=59969

I see. I agree that the change was for the better.

Martin

From nas at arctrix.com  Sun Mar  8 02:50:01 2009
From: nas at arctrix.com (Neil Schemenauer)
Date: Sun, 8 Mar 2009 01:50:01 +0000 (UTC)
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of	medusa
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<20090305231149.12555.939089272.divmod.xquotient.5373@weber.divmod.com>
Message-ID: <gov889$fgd$1@ger.gmane.org>

glyph at divmod.com <glyph at divmod.com> wrote:
> ... which is exactly why I have volunteered to explain to someone how to 
> separate the core event-loop bits (suitable for inclusion in the 
> standard library) from the huge pile of protocol implementations which 
> are not necessarily useful.
>
> Despite the FUD to the contrary, Twisted's internal factoring is quite 
> good; it's not a single, undifferentiated pile of code.  Plus, at this 
> point, we're not even talking about actually putting any Twisted code 
> into the standard library, just standardizing the "protocol" API, which 
> basically boils down to:
[...]

This sounds great.  I'm interested on working on this since it
scratches an itch of mine but I don't know if I will have time.  Do
you think if this part of Twisted became part of the standard
library that it would be used by Twisted or would it continue to
have its own version?

  Neil


From skip at pobox.com  Sun Mar  8 03:28:36 2009
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 7 Mar 2009 20:28:36 -0600
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
 of medusa
In-Reply-To: <gov889$fgd$1@ger.gmane.org>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<20090305231149.12555.939089272.divmod.xquotient.5373@weber.divmod.com>
	<gov889$fgd$1@ger.gmane.org>
Message-ID: <18867.11604.171244.871931@montanaro.dyndns.org>

    Glyph> ... which is exactly why I have volunteered to explain to someone
    Glyph> how to separate the core event-loop bits (suitable for inclusion
    Glyph> in the standard library) from the huge pile of protocol
    Glyph> implementations which are not necessarily useful.

    Neil> This sounds great.  I'm interested on working on this since it
    Neil> scratches an itch of mine but I don't know if I will have time.
    Neil> Do you think if this part of Twisted became part of the standard
    Neil> library that it would be used by Twisted or would it continue to
    Neil> have its own version?

Anybody interested in working on this at a PyCon Sprint?  I won't be
attending the conference proper, but plan to spend a couple days sprinting,
one on Mailman/SpamBayes integration and one on Python core stuff.  This
might fit well into my Python core "slot".  I will probably have little time
before the sprint to do much, but any brain dumps or Twisted pointers people
could give me in the interim would be appreciated.

Skip

From guido at python.org  Sun Mar  8 05:23:05 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Mar 2009 20:23:05 -0800
Subject: [Python-Dev] RELEASED Python 3.1 alpha 1
In-Reply-To: <51306117231E465E9FBE6EFB4D1B480A@RaymondLaptop1>
References: <1afaf6160903071053i559f5608nb37882f680ed41d0@mail.gmail.com>
	<51306117231E465E9FBE6EFB4D1B480A@RaymondLaptop1>
Message-ID: <ca471dc20903072023p25e9cb05id2a731e49d65f84c@mail.gmail.com>

On Sat, Mar 7, 2009 at 1:21 PM, Raymond Hettinger <python at rcn.com> wrote:
>
> [Benjamin Peterson]
>>
>> On behalf of the Python development team and the Python community, I'm
>> happy to announce the first alpha release of Python 3.1.
>
> Thanks for the good work.

Sorry to be late to the party. Indeed, thanks for all your efforts. I
really appreciate it!!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Sun Mar  8 05:55:34 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 08 Mar 2009 17:55:34 +1300
Subject: [Python-Dev] Pickler/Unpickler API clarification
In-Reply-To: <49B29AF4.5050407@alum.mit.edu>
References: <43aa6ff70903051207u54c1a9ebg2d931f6cae39c463@mail.gmail.com>
	<49B0F36C.8070800@alum.mit.edu> <49B25D3E.1040806@canterbury.ac.nz>
	<49B29AF4.5050407@alum.mit.edu>
Message-ID: <49B34FC6.8020603@canterbury.ac.nz>

Michael Haggerty wrote:

> Typically, the purpose of a database is to persist data across program
> runs.  So typically, your suggestion would only help if there were a way
> to persist the primed Pickler across runs.

I don't think you need to be able to pickle picklers.

In the case in question, the master pickler would be primed
by pickling all the shared classes, and the resulting pickle
would be stored in the database.

When unpickling, the master unpickler would be primed by
unpickling the shared pickle.

-- 
Greg

From guido at python.org  Sun Mar  8 06:21:37 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Mar 2009 21:21:37 -0800
Subject: [Python-Dev] Forgotten Py3.0 change to remove Queue.empty()
	andQueue.full()
In-Reply-To: <49B2FB95.60406@v.loewis.de>
References: <99FB9702A9114673AA7061EFC772782D@RaymondLaptop1>
	<9486400F26414A19B3FEC248B1CC69DD@RaymondLaptop1>
	<49B19E54.3060407@v.loewis.de>
	<B84EBC1F38924F258DC0841C4A7F9564@RaymondLaptop1>
	<49B1BD8D.4040209@v.loewis.de>
	<B1E605FA06774F04AFF8827320D73D15@RaymondLaptop1>
	<49B23AE3.90702@v.loewis.de>
	<D5ECF795F36E49E6BB5637B52430838F@RaymondLaptop1>
	<1EFA4331D3014B73BC1157E0CB785273@RaymondLaptop1>
	<49B2FB95.60406@v.loewis.de>
Message-ID: <ca471dc20903072121w53cb70e1k9b911e28eecf0508@mail.gmail.com>

On Sat, Mar 7, 2009 at 2:56 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Skip changed it to the present wording last year:
>> http://svn.python.org/view/python/trunk/Doc/library/queue.rst?r1=59750&r2=59969
>
> I see. I agree that the change was for the better.

Agreed too, though it would seem that *if* there is a single reader,
empty()==False would guarantee that one subsequent get() won't block,
and similar *if* there is a single writer, full()==False would
guarantee that the next put() won't block. My point being that the
lack of guarantee has to do with the necessary consequences of the
non-atomicity of making two calls, not with anything inherently buggy
in the implementation. Because Queue is designed for multi-threaded
use, it's important to emphasize the lacking guarantees; but I think
it's even more important to explain *why* these guarantees cannot
hold. After all we don't go around documenting e.g. that for a
dictionary, after "x in d" returns True, "x[d]" may still raise a
KeyError exception (in case another thread deleted it). I'm not sure
how I would modify the Queue documentation to clarify all this;
perhaps it would be helpful to add an introductory section explaining
the general issues with multiple readers and writers, and refer to
this section in the descriptions of qsize() (and empty() and full() if
they remain documented)?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From victor.stinner at haypocalc.com  Sun Mar  8 12:05:51 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 8 Mar 2009 12:05:51 +0100
Subject: [Python-Dev] 3.1 performance
In-Reply-To: <loom.20090307T140318-788@post.gmane.org>
References: <loom.20090307T140318-788@post.gmane.org>
Message-ID: <200903081205.52013.victor.stinner@haypocalc.com>

Hi,

> So, 3.1 is 30% faster in user CPU time, even though it probably has more
> tests (...)

Wow!

I just downloaded Python 2.6.1, 3.0.1 and 3.1alpha1, compiled them on 32 and 
64 bits CPU, and ran pybench 2.1(*).

Summary (minimum total) on 32 bits CPU:
 * Python 2.6.1: 8762 ms
 * Python 3.0.1: 8977 ms
 * Python 3.1a1: 9228 ms (slower than 3.0)

Summary (minimum total) on 64 bits CPU:
 * Python 2.6.1: 4219 ms
 * Python 3.0.1: 4502 ms 
 * Python 3.1a1: 4442 ms (faster than 3.0)

I also ran pybench once, is it enough? See attached files for the details.

(*) I copied pybench 2.1 from Pthon 3.1alpha1 because 2.6.1 only 
    includes pybench 2.0

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/
-------------- next part --------------
Computer:
 * Ubuntu 7.10
 * Pentium(R) 4 CPU 3.00GHz (32 bits)
 * 2 GB of RAM

#############################################################################

-------------------------------------------------------------------------------
PYBENCH 2.1
-------------------------------------------------------------------------------
* using CPython 3.0.1 (r301:69556, Mar 8 2009, 11:34:16) [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)]
* disabled garbage collection
* system check interval set to maximum: 2147483647
* using timer: time.time

Calibrating tests. Please wait... done.

Running 10 round(s) of the suite at warp factor 10:

* Round 1 done in 9.777 seconds.
* Round 2 done in 12.040 seconds.
* Round 3 done in 10.130 seconds.
* Round 4 done in 9.619 seconds.
* Round 5 done in 9.197 seconds.
* Round 6 done in 9.199 seconds.
* Round 7 done in 9.231 seconds.
* Round 8 done in 9.217 seconds.
* Round 9 done in 10.090 seconds.
* Round 10 done in 9.125 seconds.

-------------------------------------------------------------------------------
Benchmark: 2009-03-08 11:36:02
-------------------------------------------------------------------------------

    Rounds: 10
    Warp:   10
    Timer:  time.time

    Machine Details:
       Platform ID:    Linux-2.6.22-16-386-i686-with-debian-lenny-sid
       Processor:

    Python:
       Implementation: CPython
       Executable:     /home/haypo/prog/Python-3.0.1/python
       Version:        3.0.1
       Compiler:       GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)
       Bits:           32bit
       Build:          Mar  8 2009 11:34:16 (#r301:69556)
       Unicode:        UCS2


Test                             minimum  average  operation  overhead
-------------------------------------------------------------------------------
          BuiltinFunctionCalls:    149ms    154ms    0.30us    0.380ms
           BuiltinMethodLookup:    140ms    143ms    0.14us    0.469ms
                 CompareFloats:    139ms    141ms    0.12us    0.538ms
         CompareFloatsIntegers:    269ms    274ms    0.30us    0.462ms
               CompareIntegers:    172ms    176ms    0.10us    0.823ms
        CompareInternedStrings:    138ms    139ms    0.09us    2.911ms
                  CompareLongs:     97ms     98ms    0.09us    0.623ms
                CompareStrings:    120ms    125ms    0.13us    1.865ms
    ComplexPythonFunctionCalls:    162ms    165ms    0.83us    0.902ms
                 ConcatStrings:    251ms    264ms    0.53us    0.788ms
               CreateInstances:    175ms    196ms    1.75us    0.626ms
            CreateNewInstances:    132ms    134ms    1.59us    0.504ms
       CreateStringsWithConcat:    194ms    203ms    0.20us    1.310ms
                  DictCreation:    121ms    142ms    0.36us    0.526ms
             DictWithFloatKeys:    222ms    224ms    0.25us    1.001ms
           DictWithIntegerKeys:    119ms    122ms    0.10us    1.324ms
            DictWithStringKeys:    125ms    130ms    0.11us    1.347ms
                      ForLoops:    129ms    155ms    6.20us    0.072ms
                    IfThenElse:    138ms    150ms    0.11us    1.005ms
                   ListSlicing:    129ms    141ms   10.11us    0.095ms
                NestedForLoops:    142ms    160ms    0.11us    0.002ms
      NestedListComprehensions:    197ms    222ms   18.47us    0.151ms
          NormalClassAttribute:    229ms    279ms    0.23us    0.709ms
       NormalInstanceAttribute:    166ms    179ms    0.15us    0.713ms
           PythonFunctionCalls:    162ms    173ms    0.52us    0.390ms
             PythonMethodCalls:    202ms    220ms    0.98us    0.244ms
                     Recursion:    274ms    399ms    7.98us    0.656ms
                  SecondImport:    158ms    173ms    1.73us    0.307ms
           SecondPackageImport:    164ms    179ms    1.79us    0.260ms
         SecondSubmoduleImport:    216ms    233ms    2.33us    0.261ms
       SimpleComplexArithmetic:    135ms    163ms    0.19us    0.536ms
        SimpleDictManipulation:    242ms    282ms    0.24us    0.663ms
         SimpleFloatArithmetic:    148ms    162ms    0.12us    0.807ms
      SimpleIntFloatArithmetic:    196ms    205ms    0.16us    0.796ms
       SimpleIntegerArithmetic:    190ms    197ms    0.15us    0.798ms
      SimpleListComprehensions:    165ms    169ms   14.08us    0.157ms
        SimpleListManipulation:    128ms    132ms    0.11us    0.860ms
          SimpleLongArithmetic:    132ms    162ms    0.25us    0.393ms
                    SmallLists:    170ms    175ms    0.26us    0.522ms
                   SmallTuples:    179ms    191ms    0.35us    0.571ms
         SpecialClassAttribute:    407ms    440ms    0.37us    0.827ms
      SpecialInstanceAttribute:    167ms    193ms    0.16us    0.822ms
                StringMappings:    363ms    370ms    1.47us    0.638ms
              StringPredicates:    175ms    179ms    0.26us    3.582ms
                 StringSlicing:    277ms    288ms    0.52us    1.133ms
                     TryExcept:     78ms     79ms    0.04us    1.199ms
                    TryFinally:    156ms    160ms    1.00us    0.554ms
                TryRaiseExcept:     62ms     63ms    0.99us    0.542ms
                  TupleSlicing:    186ms    190ms    0.72us    0.064ms
                   WithFinally:    202ms    275ms    1.72us    0.555ms
               WithRaiseExcept:    185ms    191ms    2.39us    0.674ms
-------------------------------------------------------------------------------
Totals:                           8977ms   9763ms

#############################################################################

-------------------------------------------------------------------------------
PYBENCH 2.1
-------------------------------------------------------------------------------
* using CPython 3.1a1 (r31a1:70230, Mar 8 2009, 11:41:12) [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)]
* disabled garbage collection
* system check interval set to maximum: 2147483647
* using timer: time.time

Calibrating tests. Please wait... done.

Running 10 round(s) of the suite at warp factor 10:

* Round 1 done in 10.169 seconds.
* Round 2 done in 10.157 seconds.
* Round 3 done in 9.431 seconds.
* Round 4 done in 10.038 seconds.
* Round 5 done in 11.679 seconds.
* Round 6 done in 10.579 seconds.
* Round 7 done in 10.192 seconds.
* Round 8 done in 10.502 seconds.
* Round 9 done in 9.601 seconds.
* Round 10 done in 9.475 seconds.

-------------------------------------------------------------------------------
Benchmark: 2009-03-08 11:42:38
-------------------------------------------------------------------------------

    Rounds: 10
    Warp:   10
    Timer:  time.time

    Machine Details:
       Platform ID:    Linux-2.6.22-16-386-i686-with-debian-lenny-sid
       Processor:

    Python:
       Implementation: CPython
       Executable:     /home/haypo/prog/Python-3.1a1/python
       Version:        3.1.0
       Compiler:       GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)
       Bits:           32bit
       Build:          Mar  8 2009 11:41:12 (#r31a1:70230)
       Unicode:        UCS2


Test                             minimum  average  operation  overhead
-------------------------------------------------------------------------------
          BuiltinFunctionCalls:    165ms    173ms    0.34us    0.455ms
           BuiltinMethodLookup:    145ms    152ms    0.14us    0.547ms
                 CompareFloats:    186ms    190ms    0.16us    0.609ms
         CompareFloatsIntegers:    296ms    320ms    0.36us    0.455ms
               CompareIntegers:    175ms    176ms    0.10us    0.933ms
        CompareInternedStrings:    150ms    157ms    0.10us    2.639ms
                  CompareLongs:    110ms    124ms    0.12us    0.610ms
                CompareStrings:    126ms    139ms    0.14us    1.817ms
    ComplexPythonFunctionCalls:    162ms    189ms    0.94us    0.886ms
                 ConcatStrings:    264ms    285ms    0.57us    1.016ms
               CreateInstances:    178ms    200ms    1.78us    0.801ms
            CreateNewInstances:    133ms    142ms    1.69us    0.619ms
       CreateStringsWithConcat:    202ms    206ms    0.21us    1.769ms
                  DictCreation:    122ms    129ms    0.32us    0.685ms
             DictWithFloatKeys:    211ms    216ms    0.24us    1.379ms
           DictWithIntegerKeys:    119ms    122ms    0.10us    1.699ms
            DictWithStringKeys:    132ms    135ms    0.11us    1.775ms
                      ForLoops:    191ms    193ms    7.73us    0.081ms
                    IfThenElse:    132ms    135ms    0.10us    1.281ms
                   ListSlicing:    130ms    135ms    9.67us    0.123ms
                NestedForLoops:    182ms    203ms    0.14us    0.003ms
      NestedListComprehensions:    176ms    202ms   16.85us    0.178ms
          NormalClassAttribute:    243ms    272ms    0.23us    0.919ms
       NormalInstanceAttribute:    163ms    186ms    0.16us    0.922ms
           PythonFunctionCalls:    168ms    175ms    0.53us    0.524ms
             PythonMethodCalls:    203ms    264ms    1.17us    0.306ms
                     Recursion:    261ms    265ms    5.30us    0.883ms
                  SecondImport:    162ms    184ms    1.84us    0.361ms
           SecondPackageImport:    166ms    230ms    2.30us    0.346ms
         SecondSubmoduleImport:    215ms    252ms    2.52us    0.337ms
       SimpleComplexArithmetic:    139ms    162ms    0.18us    0.727ms
        SimpleDictManipulation:    253ms    260ms    0.22us    0.905ms
         SimpleFloatArithmetic:    155ms    168ms    0.13us    1.056ms
      SimpleIntFloatArithmetic:    189ms    294ms    0.22us    1.094ms
       SimpleIntegerArithmetic:    188ms    200ms    0.15us    1.055ms
      SimpleListComprehensions:    138ms    168ms   14.02us    0.174ms
        SimpleListManipulation:    134ms    146ms    0.12us    1.196ms
          SimpleLongArithmetic:    126ms    193ms    0.29us    0.511ms
                    SmallLists:    176ms    182ms    0.27us    0.706ms
                   SmallTuples:    182ms    194ms    0.36us    0.804ms
         SpecialClassAttribute:    403ms    419ms    0.35us    0.930ms
      SpecialInstanceAttribute:    165ms    174ms    0.14us    0.923ms
                StringMappings:    362ms    409ms    1.62us    0.771ms
              StringPredicates:    176ms    187ms    0.27us    3.718ms
                 StringSlicing:    277ms    313ms    0.56us    1.568ms
                     TryExcept:     80ms     82ms    0.04us    1.311ms
                    TryFinally:    159ms    195ms    1.22us    0.668ms
                TryRaiseExcept:     62ms     65ms    1.01us    0.732ms
                  TupleSlicing:    191ms    195ms    0.74us    0.098ms
                   WithFinally:    212ms    217ms    1.35us    0.670ms
               WithRaiseExcept:    193ms    209ms    2.61us    0.918ms
-------------------------------------------------------------------------------
Totals:                           9228ms  10182ms

#############################################################################

-------------------------------------------------------------------------------
PYBENCH 2.1
-------------------------------------------------------------------------------
* using CPython 2.6.1 (r261:67515, Mar 8 2009, 11:51:17) [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)]
* disabled garbage collection
* system check interval set to maximum: 2147483647
* using timer: time.time

Calibrating tests. Please wait... done.

Running 10 round(s) of the suite at warp factor 10:

* Round 1 done in 9.183 seconds.
* Round 2 done in 9.884 seconds.
* Round 3 done in 9.230 seconds.
* Round 4 done in 9.195 seconds.
* Round 5 done in 9.170 seconds.
* Round 6 done in 10.457 seconds.
* Round 7 done in 9.253 seconds.
* Round 8 done in 9.013 seconds.
* Round 9 done in 8.945 seconds.
* Round 10 done in 8.949 seconds.

-------------------------------------------------------------------------------
Benchmark: 2009-03-08 11:53:14
-------------------------------------------------------------------------------

    Rounds: 10
    Warp:   10
    Timer:  time.time

    Machine Details:
       Platform ID:    Linux-2.6.22-16-386-i686-with-debian-lenny-sid
       Processor:

    Python:
       Implementation: CPython
       Executable:     /home/haypo/prog/Python-2.6.1/python
       Version:        2.6.1
       Compiler:       GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)
       Bits:           32bit
       Build:          Mar  8 2009 11:51:17 (#r261:67515)
       Unicode:        UCS2


Test                             minimum  average  operation  overhead
-------------------------------------------------------------------------------
          BuiltinFunctionCalls:    153ms    159ms    0.31us    0.366ms
           BuiltinMethodLookup:    153ms    158ms    0.15us    0.427ms
                 CompareFloats:    147ms    213ms    0.18us    0.488ms
         CompareFloatsIntegers:    144ms    157ms    0.17us    0.366ms
               CompareIntegers:    111ms    121ms    0.07us    0.732ms
        CompareInternedStrings:    132ms    135ms    0.09us    1.875ms
                  CompareLongs:    204ms    214ms    0.20us    0.516ms
                CompareStrings:    136ms    145ms    0.14us    1.249ms
                CompareUnicode:    115ms    125ms    0.17us    0.938ms
    ComplexPythonFunctionCalls:    159ms    168ms    0.84us    0.614ms
                 ConcatStrings:    229ms    250ms    0.50us    0.715ms
                 ConcatUnicode:    157ms    169ms    0.56us    0.505ms
               CreateInstances:    185ms    190ms    1.70us    0.497ms
            CreateNewInstances:    138ms    141ms    1.68us    0.455ms
       CreateStringsWithConcat:    164ms    173ms    0.17us    1.226ms
       CreateUnicodeWithConcat:    129ms    135ms    0.34us    0.490ms
                  DictCreation:    119ms    122ms    0.31us    0.491ms
             DictWithFloatKeys:    220ms    225ms    0.25us    1.096ms
           DictWithIntegerKeys:    122ms    131ms    0.11us    1.229ms
            DictWithStringKeys:    123ms    128ms    0.11us    1.225ms
                      ForLoops:    128ms    134ms    5.38us    0.080ms
                    IfThenElse:    113ms    115ms    0.09us    0.920ms
                   ListSlicing:    126ms    129ms    9.19us    0.098ms
                NestedForLoops:    151ms    158ms    0.11us    0.026ms
      NestedListComprehensions:    176ms    180ms   14.97us    0.124ms
          NormalClassAttribute:    147ms    152ms    0.13us    0.741ms
       NormalInstanceAttribute:    131ms    137ms    0.11us    0.617ms
           PythonFunctionCalls:    165ms    170ms    0.52us    0.367ms
             PythonMethodCalls:    198ms    200ms    0.89us    0.190ms
                     Recursion:    235ms    240ms    4.80us    0.612ms
                  SecondImport:    148ms    151ms    1.51us    0.243ms
           SecondPackageImport:    149ms    153ms    1.53us    0.244ms
         SecondSubmoduleImport:    184ms    190ms    1.90us    0.243ms
       SimpleComplexArithmetic:    141ms    144ms    0.16us    0.488ms
        SimpleDictManipulation:    245ms    254ms    0.21us    0.615ms
         SimpleFloatArithmetic:    153ms    163ms    0.12us    0.734ms
      SimpleIntFloatArithmetic:    115ms    117ms    0.09us    0.735ms
       SimpleIntegerArithmetic:    112ms    115ms    0.09us    0.737ms
      SimpleListComprehensions:    148ms    154ms   12.85us    0.125ms
        SimpleListManipulation:    115ms    119ms    0.10us    0.797ms
          SimpleLongArithmetic:     88ms     93ms    0.14us    0.367ms
                    SmallLists:    164ms    215ms    0.32us    0.490ms
                   SmallTuples:    129ms    146ms    0.27us    0.552ms
         SpecialClassAttribute:    149ms    169ms    0.14us    0.617ms
      SpecialInstanceAttribute:    233ms    277ms    0.23us    0.618ms
                StringMappings:    138ms    156ms    0.62us    0.582ms
              StringPredicates:    157ms    180ms    0.26us    2.571ms
                 StringSlicing:     99ms    113ms    0.20us    1.033ms
                     TryExcept:     87ms     90ms    0.04us    0.923ms
                    TryFinally:    178ms    197ms    1.23us    0.492ms
                TryRaiseExcept:    123ms    126ms    1.97us    0.587ms
                  TupleSlicing:    143ms    146ms    0.56us    0.064ms
               UnicodeMappings:    147ms    151ms    4.21us    0.621ms
             UnicodePredicates:    142ms    146ms    0.27us    3.270ms
             UnicodeProperties:    137ms    142ms    0.35us    2.570ms
                UnicodeSlicing:    153ms    158ms    0.32us    0.932ms
                   WithFinally:    211ms    221ms    1.38us    0.493ms
               WithRaiseExcept:    165ms    171ms    2.13us    0.614ms
-------------------------------------------------------------------------------
Totals:                           8762ms   9328ms


#############################################################################
-------------- next part --------------
Computer:
 * Ubuntu 8.10
 * Intel(R) Core(TM)2 Quad  CPU   Q9300  @ 2.50GHz (64 bits)
 * 4 GB of RAM

#############################################################################



#############################################################################

$ ./python Tools/pybench/pybench.py
-------------------------------------------------------------------------------
PYBENCH 2.1
-------------------------------------------------------------------------------
* using CPython 3.0.1 (r301:69556, Mar 8 2009, 11:32:47) [GCC 4.3.2]
* disabled garbage collection
* system check interval set to maximum: 2147483647
* using timer: time.time

Calibrating tests. Please wait... done.

Running 10 round(s) of the suite at warp factor 10:

* Round 1 done in 4.577 seconds.
* Round 2 done in 4.670 seconds.
* Round 3 done in 4.579 seconds.
* Round 4 done in 4.560 seconds.
* Round 5 done in 4.580 seconds.
* Round 6 done in 4.563 seconds.
* Round 7 done in 4.570 seconds.
* Round 8 done in 4.579 seconds.
* Round 9 done in 4.653 seconds.
* Round 10 done in 4.585 seconds.

-------------------------------------------------------------------------------
Benchmark: 2009-03-08 11:34:11
-------------------------------------------------------------------------------

    Rounds: 10
    Warp:   10
    Timer:  time.time

    Machine Details:
       Platform ID:    Linux-2.6.27-11-generic-x86_64-with-debian-lenny-sid
       Processor:

    Python:
       Implementation: CPython
       Executable:     /home/haypo/prog/Python-3.0.1/python
       Version:        3.0.1
       Compiler:       GCC 4.3.2
       Bits:           64bit
       Build:          Mar  8 2009 11:32:47 (#r301:69556)
       Unicode:        UCS2


Test                             minimum  average  operation  overhead
-------------------------------------------------------------------------------
          BuiltinFunctionCalls:     72ms     73ms    0.14us    0.234ms
           BuiltinMethodLookup:     52ms     53ms    0.05us    0.276ms
                 CompareFloats:     86ms     87ms    0.07us    0.312ms
         CompareFloatsIntegers:    110ms    112ms    0.12us    0.234ms
               CompareIntegers:    128ms    131ms    0.07us    0.471ms
        CompareInternedStrings:    106ms    108ms    0.07us    1.189ms
                  CompareLongs:     75ms     77ms    0.07us    0.273ms
                CompareStrings:     77ms     79ms    0.08us    0.795ms
    ComplexPythonFunctionCalls:     93ms    100ms    0.50us    0.404ms
                 ConcatStrings:     90ms     91ms    0.18us    0.470ms
               CreateInstances:     84ms     85ms    0.76us    0.372ms
            CreateNewInstances:     63ms     64ms    0.76us    0.301ms
       CreateStringsWithConcat:    112ms    112ms    0.11us    0.787ms
                  DictCreation:     59ms     60ms    0.15us    0.312ms
             DictWithFloatKeys:     70ms     71ms    0.08us    0.600ms
           DictWithIntegerKeys:     69ms     69ms    0.06us    0.800ms
            DictWithStringKeys:     76ms     77ms    0.06us    0.795ms
                      ForLoops:     63ms     63ms    2.51us    0.036ms
                    IfThenElse:     81ms     81ms    0.06us    0.592ms
                   ListSlicing:     71ms     76ms    5.40us    0.054ms
                NestedForLoops:     83ms     83ms    0.06us    0.002ms
      NestedListComprehensions:    100ms    102ms    8.49us    0.078ms
          NormalClassAttribute:    145ms    147ms    0.12us    0.423ms
       NormalInstanceAttribute:     79ms     80ms    0.07us    0.422ms
           PythonFunctionCalls:     78ms     79ms    0.24us    0.234ms
             PythonMethodCalls:     94ms     95ms    0.42us    0.145ms
                     Recursion:    119ms    120ms    2.40us    0.392ms
                  SecondImport:     74ms     75ms    0.75us    0.154ms
           SecondPackageImport:     79ms     80ms    0.80us    0.155ms
         SecondSubmoduleImport:    108ms    112ms    1.12us    0.155ms
       SimpleComplexArithmetic:     59ms     61ms    0.07us    0.312ms
        SimpleDictManipulation:    124ms    128ms    0.11us    0.392ms
         SimpleFloatArithmetic:     61ms     62ms    0.05us    0.470ms
      SimpleIntFloatArithmetic:     82ms     83ms    0.06us    0.478ms
       SimpleIntegerArithmetic:     82ms     83ms    0.06us    0.477ms
      SimpleListComprehensions:     86ms     87ms    7.26us    0.078ms
        SimpleListManipulation:     66ms     68ms    0.06us    0.510ms
          SimpleLongArithmetic:     60ms     61ms    0.09us    0.238ms
                    SmallLists:     80ms     80ms    0.12us    0.313ms
                   SmallTuples:     91ms     93ms    0.17us    0.352ms
         SpecialClassAttribute:    229ms    235ms    0.20us    0.421ms
      SpecialInstanceAttribute:     79ms     80ms    0.07us    0.414ms
                StringMappings:    187ms    190ms    0.75us    0.350ms
              StringPredicates:     79ms     79ms    0.11us    1.718ms
                 StringSlicing:    122ms    127ms    0.23us    0.672ms
                     TryExcept:     61ms     61ms    0.03us    0.589ms
                    TryFinally:     60ms     62ms    0.39us    0.313ms
                TryRaiseExcept:     30ms     32ms    0.50us    0.318ms
                  TupleSlicing:     98ms    100ms    0.38us    0.039ms
                   WithFinally:     87ms     90ms    0.56us    0.313ms
               WithRaiseExcept:     86ms     88ms    1.11us    0.392ms
-------------------------------------------------------------------------------
Totals:                           4502ms   4591ms


#############################################################################

-------------------------------------------------------------------------------
PYBENCH 2.1
-------------------------------------------------------------------------------
* using CPython 3.1a1 (r31a1:70230, Mar 8 2009, 11:36:46) [GCC 4.3.2]
* disabled garbage collection
* system check interval set to maximum: 2147483647
* using timer: time.time

Calibrating tests. Please wait... done.

Running 10 round(s) of the suite at warp factor 10:

* Round 1 done in 4.531 seconds.
* Round 2 done in 4.514 seconds.
* Round 3 done in 4.534 seconds.
* Round 4 done in 4.552 seconds.
* Round 5 done in 4.536 seconds.
* Round 6 done in 4.537 seconds.
* Round 7 done in 4.507 seconds.
* Round 8 done in 4.501 seconds.
* Round 9 done in 4.508 seconds.
* Round 10 done in 4.550 seconds.

-------------------------------------------------------------------------------
Benchmark: 2009-03-08 11:39:10
-------------------------------------------------------------------------------

    Rounds: 10
    Warp:   10
    Timer:  time.time

    Machine Details:
       Platform ID:    Linux-2.6.27-11-generic-x86_64-with-debian-lenny-sid
       Processor:

    Python:
       Implementation: CPython
       Executable:     /home/haypo/prog/Python-3.1a1/python
       Version:        3.1.0
       Compiler:       GCC 4.3.2
       Bits:           64bit
       Build:          Mar  8 2009 11:36:46 (#r31a1:70230)
       Unicode:        UCS2


Test                             minimum  average  operation  overhead
-------------------------------------------------------------------------------
          BuiltinFunctionCalls:     76ms     79ms    0.16us    0.230ms
           BuiltinMethodLookup:     49ms     50ms    0.05us    0.268ms
                 CompareFloats:     93ms     95ms    0.08us    0.307ms
         CompareFloatsIntegers:    117ms    119ms    0.13us    0.227ms
               CompareIntegers:    135ms    135ms    0.08us    0.463ms
        CompareInternedStrings:    105ms    105ms    0.07us    1.168ms
                  CompareLongs:     77ms     77ms    0.07us    0.273ms
                CompareStrings:     77ms     79ms    0.08us    0.796ms
    ComplexPythonFunctionCalls:     81ms     82ms    0.41us    0.386ms
                 ConcatStrings:     84ms     93ms    0.19us    0.456ms
               CreateInstances:     77ms     80ms    0.71us    0.372ms
            CreateNewInstances:     59ms     60ms    0.72us    0.299ms
       CreateStringsWithConcat:    101ms    103ms    0.10us    0.785ms
                  DictCreation:     62ms     63ms    0.16us    0.312ms
             DictWithFloatKeys:     73ms     74ms    0.08us    0.584ms
           DictWithIntegerKeys:     72ms     73ms    0.06us    0.784ms
            DictWithStringKeys:     64ms     65ms    0.05us    0.787ms
                      ForLoops:     63ms     65ms    2.61us    0.036ms
                    IfThenElse:     85ms     86ms    0.06us    0.587ms
                   ListSlicing:     72ms     75ms    5.37us    0.053ms
                NestedForLoops:     81ms     82ms    0.05us    0.002ms
      NestedListComprehensions:     83ms     84ms    6.99us    0.077ms
          NormalClassAttribute:    137ms    138ms    0.11us    0.411ms
       NormalInstanceAttribute:     83ms     84ms    0.07us    0.418ms
           PythonFunctionCalls:     81ms     82ms    0.25us    0.231ms
             PythonMethodCalls:     94ms     95ms    0.42us    0.148ms
                     Recursion:    115ms    115ms    2.30us    0.447ms
                  SecondImport:     68ms     69ms    0.69us    0.154ms
           SecondPackageImport:     74ms     74ms    0.74us    0.154ms
         SecondSubmoduleImport:     99ms    103ms    1.03us    0.154ms
       SimpleComplexArithmetic:     63ms     64ms    0.07us    0.311ms
        SimpleDictManipulation:    119ms    121ms    0.10us    0.393ms
         SimpleFloatArithmetic:     64ms     66ms    0.05us    0.468ms
      SimpleIntFloatArithmetic:     83ms     83ms    0.06us    0.463ms
       SimpleIntegerArithmetic:     82ms     84ms    0.06us    0.514ms
      SimpleListComprehensions:     68ms     70ms    5.85us    0.077ms
        SimpleListManipulation:     67ms     67ms    0.06us    0.502ms
          SimpleLongArithmetic:     55ms     56ms    0.09us    0.230ms
                    SmallLists:     82ms     84ms    0.12us    0.308ms
                   SmallTuples:     95ms     99ms    0.18us    0.347ms
         SpecialClassAttribute:    228ms    230ms    0.19us    0.414ms
      SpecialInstanceAttribute:     81ms     84ms    0.07us    0.418ms
                StringMappings:    188ms    192ms    0.76us    0.344ms
              StringPredicates:     80ms     80ms    0.11us    1.697ms
                 StringSlicing:    132ms    137ms    0.24us    0.664ms
                     TryExcept:     63ms     63ms    0.03us    0.590ms
                    TryFinally:     58ms     59ms    0.37us    0.313ms
                TryRaiseExcept:     29ms     30ms    0.46us    0.314ms
                  TupleSlicing:    102ms    103ms    0.39us    0.040ms
                   WithFinally:     84ms     86ms    0.54us    0.315ms
               WithRaiseExcept:     82ms     83ms    1.04us    0.394ms
-------------------------------------------------------------------------------
Totals:                           4442ms   4527ms

#############################################################################

-------------------------------------------------------------------------------
PYBENCH 2.1
-------------------------------------------------------------------------------
* using CPython 2.6.1 (r261:67515, Mar 8 2009, 11:43:11) [GCC 4.3.2]
* disabled garbage collection
* system check interval set to maximum: 2147483647
* using timer: time.time

Calibrating tests. Please wait... done.

Running 10 round(s) of the suite at warp factor 10:

* Round 1 done in 4.289 seconds.
* Round 2 done in 4.308 seconds.
* Round 3 done in 4.289 seconds.
* Round 4 done in 4.309 seconds.
* Round 5 done in 4.272 seconds.
* Round 6 done in 4.353 seconds.
* Round 7 done in 4.287 seconds.
* Round 8 done in 4.333 seconds.
* Round 9 done in 4.336 seconds.
* Round 10 done in 4.291 seconds.

-------------------------------------------------------------------------------
Benchmark: 2009-03-08 11:48:24
-------------------------------------------------------------------------------

    Rounds: 10
    Warp:   10
    Timer:  time.time

    Machine Details:
       Platform ID:    Linux-2.6.27-11-generic-x86_64-with-debian-lenny-sid
       Processor:

    Python:
       Implementation: CPython
       Executable:     /home/haypo/prog/Python-2.6.1/python
       Version:        2.6.1
       Compiler:       GCC 4.3.2
       Bits:           64bit
       Build:          Mar  8 2009 11:43:11 (#r261:67515)
       Unicode:        UCS2


Test                             minimum  average  operation  overhead
-------------------------------------------------------------------------------
          BuiltinFunctionCalls:     84ms     86ms    0.17us    0.203ms
           BuiltinMethodLookup:     51ms     52ms    0.05us    0.258ms
                 CompareFloats:     67ms     67ms    0.06us    0.271ms
         CompareFloatsIntegers:     68ms     68ms    0.08us    0.204ms
               CompareIntegers:     78ms     82ms    0.05us    0.408ms
        CompareInternedStrings:     94ms     94ms    0.06us    1.297ms
                  CompareLongs:     46ms     46ms    0.04us    0.238ms
                CompareStrings:     86ms     87ms    0.09us    0.690ms
                CompareUnicode:     65ms     66ms    0.09us    0.520ms
    ComplexPythonFunctionCalls:     83ms     84ms    0.42us    0.341ms
                 ConcatStrings:     85ms     92ms    0.18us    0.390ms
                 ConcatUnicode:     49ms     54ms    0.18us    0.275ms
               CreateInstances:     80ms     81ms    0.72us    0.276ms
            CreateNewInstances:     62ms     63ms    0.75us    0.260ms
       CreateStringsWithConcat:     79ms     80ms    0.08us    0.699ms
       CreateUnicodeWithConcat:     53ms     54ms    0.14us    0.274ms
                  DictCreation:     61ms     62ms    0.16us    0.273ms
             DictWithFloatKeys:     76ms     78ms    0.09us    0.510ms
           DictWithIntegerKeys:     67ms     68ms    0.06us    0.680ms
            DictWithStringKeys:     61ms     63ms    0.05us    0.680ms
                      ForLoops:     55ms     58ms    2.30us    0.046ms
                    IfThenElse:     80ms     82ms    0.06us    0.513ms
                   ListSlicing:     67ms     69ms    4.90us    0.056ms
                NestedForLoops:     74ms     77ms    0.05us    0.012ms
      NestedListComprehensions:     95ms     97ms    8.11us    0.069ms
          NormalClassAttribute:     70ms     71ms    0.06us    0.344ms
       NormalInstanceAttribute:     67ms     68ms    0.06us    0.342ms
           PythonFunctionCalls:     81ms     81ms    0.25us    0.205ms
             PythonMethodCalls:     92ms     94ms    0.42us    0.103ms
                     Recursion:    100ms    103ms    2.07us    0.342ms
                  SecondImport:     62ms     63ms    0.63us    0.135ms
           SecondPackageImport:     66ms     68ms    0.68us    0.134ms
         SecondSubmoduleImport:     89ms     89ms    0.89us    0.135ms
       SimpleComplexArithmetic:     71ms     72ms    0.08us    0.272ms
        SimpleDictManipulation:    121ms    125ms    0.10us    0.341ms
         SimpleFloatArithmetic:     70ms     71ms    0.05us    0.408ms
      SimpleIntFloatArithmetic:     63ms     63ms    0.05us    0.408ms
       SimpleIntegerArithmetic:     59ms     59ms    0.04us    0.408ms
      SimpleListComprehensions:     82ms     85ms    7.11us    0.070ms
        SimpleListManipulation:     61ms     62ms    0.05us    0.441ms
          SimpleLongArithmetic:     33ms     33ms    0.05us    0.202ms
                    SmallLists:     66ms     68ms    0.10us    0.271ms
                   SmallTuples:     75ms     76ms    0.14us    0.307ms
         SpecialClassAttribute:     69ms     70ms    0.06us    0.343ms
      SpecialInstanceAttribute:    134ms    135ms    0.11us    0.358ms
                StringMappings:     64ms     64ms    0.26us    0.306ms
              StringPredicates:     72ms     72ms    0.10us    1.412ms
                 StringSlicing:     49ms     51ms    0.09us    0.567ms
                     TryExcept:     62ms     64ms    0.03us    0.511ms
                    TryFinally:     66ms     67ms    0.42us    0.271ms
                TryRaiseExcept:     73ms     74ms    1.16us    0.272ms
                  TupleSlicing:     77ms     80ms    0.30us    0.036ms
               UnicodeMappings:     80ms     81ms    2.24us    0.304ms
             UnicodePredicates:     68ms     69ms    0.13us    1.684ms
             UnicodeProperties:     78ms     79ms    0.20us    1.400ms
                UnicodeSlicing:     66ms     71ms    0.14us    0.510ms
                   WithFinally:     88ms     90ms    0.56us    0.272ms
               WithRaiseExcept:     76ms     79ms    0.98us    0.347ms
-------------------------------------------------------------------------------
Totals:                           4219ms   4307ms

#############################################################################


From chris at simplistix.co.uk  Sun Mar  8 12:58:32 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Sun, 08 Mar 2009 11:58:32 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
References: <49ABCFC9.1070508@gmail.com>
	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
Message-ID: <49B3B2E8.4050605@simplistix.co.uk>

Guido van Rossum wrote:
> Based on the sad example of BerkeleyDB, which was initially welcomed
> into the stdlib but more recently booted out for reasons having to do
> with the release cycle of the external dependency and other issues
> typical for large external dependencies, I think we should be very
> careful with including it in the standard library.


Yes. My experience of these kinds of libraries (bdb, lxml, etc) is that 
having them in the Python stdlib is a "bad thing".

Why? Because python (quite rightly, as I'm being convinced!) has a very 
conservative policy of changes is 3rd point releases. This, however, 
means you end up getting 'stuck" with a release of something like lxml 
that you can't upgrade to get new features because you, say, use a 
debian-packages python which only upgrades when Debian next decide to do 
  release...

In light of this, what I'd love to see (but sadly can't really help 
with, and am not optimistic about happening) is for:

- python to grow a decent, cross platform, package management system

- the standard library to actually shrink to a point where only 
libraries that are not released elsewhere are included

I'd be interested to know how many users of python also felt this way ;-)

Chris

From chris at simplistix.co.uk  Sun Mar  8 13:15:26 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Sun, 08 Mar 2009 12:15:26 +0000
Subject: [Python-Dev] patch commit policies (was [issue4308] repr of
 httplib.IncompleteRead is stupid)
In-Reply-To: <1660610E-9A4C-459E-B0E7-F31150DF38A1@python.org>
References: <49AC394E.2000903@v.loewis.de> <49AF9AD3.2090405@simplistix.co.uk>
	<49B05DE0.4020804@v.loewis.de>
	<1660610E-9A4C-459E-B0E7-F31150DF38A1@python.org>
Message-ID: <49B3B6DE.2000001@simplistix.co.uk>

Barry Warsaw wrote:
>>> That aside, is it actually a python-wide policy to *forbid* patching
>>> older releases where the patch isn't security-related?
>>
>> I set this policy for the releases I manage, namely 2.4 and 2.5.
> 
> This is a Python-wide policy.

...and, now that Martin has explained it, it makes perfect sense...

> When Python 2.7 is released, there will be one last 2.6.x bug fix 
> release, and then it will go into security-only mode.  Similarly, when 
> Python 3.1 is released, there will be one last 3.0.x release and it too 
> will go into security-only mode.

...of course, in a perfect work, it'd be nice to have 2 or 3 previous 
second point releases in bug-fix mode, rather than just one ;-)

Chris



From chris at simplistix.co.uk  Sun Mar  8 13:17:00 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Sun, 08 Mar 2009 12:17:00 +0000
Subject: [Python-Dev] patch commit policies (was [issue4308] repr of
 httplib.IncompleteRead is stupid)
In-Reply-To: <49B09381.9070604@palladion.com>
References: <49AC394E.2000903@v.loewis.de> <49AF9AD3.2090405@simplistix.co.uk>
	<49B09381.9070604@palladion.com>
Message-ID: <49B3B73C.1090000@simplistix.co.uk>

Tres Seaver wrote:
> If it is possible for a hostile outsider to trigger the DOS by sending
> mail to be processed by an application using the library, and the
> application can't avoid the DOS without ditching / forking /
> monkeypatching the library, then I would call the bug a "security bug",
> period.

I just captured the original exception and re-raised it as a subclass 
with a sane __repr__

cheers,

Chris


From solipsis at pitrou.net  Sun Mar  8 13:20:34 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 8 Mar 2009 12:20:34 +0000 (UTC)
Subject: [Python-Dev] 3.1 performance
References: <loom.20090307T140318-788@post.gmane.org>
	<200903081205.52013.victor.stinner@haypocalc.com>
Message-ID: <loom.20090308T121843-630@post.gmane.org>


Hi,

Victor Stinner <victor.stinner <at> haypocalc.com> writes:
> 
> Summary (minimum total) on 32 bits CPU:
>  * Python 2.6.1: 8762 ms
>  * Python 3.0.1: 8977 ms
>  * Python 3.1a1: 9228 ms (slower than 3.0)

Have you compiled with or without "--with-computed-gotos"?

Regards

Antoine.



From glyph at divmod.com  Sun Mar  8 14:36:06 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Sun, 08 Mar 2009 13:36:06 -0000
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of	medusa
In-Reply-To: <18867.11604.171244.871931@montanaro.dyndns.org>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<20090305231149.12555.939089272.divmod.xquotient.5373@weber.divmod.com>
	<gov889$fgd$1@ger.gmane.org>
	<18867.11604.171244.871931@montanaro.dyndns.org>
Message-ID: <20090308133606.12555.1039716756.divmod.xquotient.5515@weber.divmod.com>


On 02:28 am, skip at pobox.com wrote:
>    Glyph> ... which is exactly why I have volunteered to explain to 
>someone
>    Glyph> how to separate the core event-loop bits

>    Neil> This sounds great.

>Anybody interested in working on this at a PyCon Sprint?  I won't be
>attending the conference proper, but plan to spend a couple days 
>sprinting,
>one on Mailman/SpamBayes integration and one on Python core stuff. 
>This
>might fit well into my Python core "slot".  I will probably have little 
>time
>before the sprint to do much, but any brain dumps or Twisted pointers 
>people
>could give me in the interim would be appreciated.

I'll try to make sure we get those notes to you in advance of the 
sprints :).  I should be at the Twisted sprint the whole time - will you 
be at the physical sprint, or following along at home?

From glyph at divmod.com  Sun Mar  8 14:42:33 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Sun, 08 Mar 2009 13:42:33 -0000
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of	medusa
In-Reply-To: <gov889$fgd$1@ger.gmane.org>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<20090305231149.12555.939089272.divmod.xquotient.5373@weber.divmod.com>
	<gov889$fgd$1@ger.gmane.org>
Message-ID: <20090308134233.12555.1234628334.divmod.xquotient.5527@weber.divmod.com>


On 01:50 am, nas at arctrix.com wrote:
>glyph at divmod.com <glyph at divmod.com> wrote:

>>we're not even talking about actually putting any Twisted code
>>into the standard library, just standardizing the "protocol" API, 
>>which
>>basically boils down to:
>[...]
>
>This sounds great.  I'm interested on working on this since it
>scratches an itch of mine but I don't know if I will have time.  Do
>you think if this part of Twisted became part of the standard
>library that it would be used by Twisted or would it continue to
>have its own version?

As I said above, right now I'm just talking about the interface.  There 
are several implementations of the main loop that call those interfaces, 
as well as test implementations.

Of course I hope we can eliminate some redundancy, but that's step 2: 
before we refactor, we have to make the code that we're refactoring 
actually repetitive rather than just similar.

From skip at pobox.com  Sun Mar  8 14:46:17 2009
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 8 Mar 2009 08:46:17 -0500
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
 of medusa
In-Reply-To: <20090308133606.12555.1039716756.divmod.xquotient.5515@weber.divmod.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<20090305231149.12555.939089272.divmod.xquotient.5373@weber.divmod.com>
	<gov889$fgd$1@ger.gmane.org>
	<18867.11604.171244.871931@montanaro.dyndns.org>
	<20090308133606.12555.1039716756.divmod.xquotient.5515@weber.divmod.com>
Message-ID: <18867.52265.87793.587875@montanaro.dyndns.org>


    Glyph> I'll try to make sure we get those notes to you in advance of the
    Glyph> sprints :).  I should be at the Twisted sprint the whole time -
    Glyph> will you be at the physical sprint, or following along at home?

Thanks.  I will be at the physical sprints.  (I live in the Chicago area.)

Skip

From martin at v.loewis.de  Sun Mar  8 14:51:15 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Mar 2009 14:51:15 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49B3B2E8.4050605@simplistix.co.uk>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<49B3B2E8.4050605@simplistix.co.uk>
Message-ID: <49B3CD53.5010906@v.loewis.de>

> In light of this, what I'd love to see (but sadly can't really help
> with, and am not optimistic about happening) is for:
> 
> - python to grow a decent, cross platform, package management system
> 
> - the standard library to actually shrink to a point where only
> libraries that are not released elsewhere are included
> 
> I'd be interested to know how many users of python also felt this way ;-)

I don't like the standard library to shrink. It's good that batteries
are included.

Regards,
Martin

From fuzzyman at voidspace.org.uk  Sun Mar  8 15:02:37 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 08 Mar 2009 14:02:37 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49B3CD53.5010906@v.loewis.de>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>
	<49B3CD53.5010906@v.loewis.de>
Message-ID: <49B3CFFD.7050905@voidspace.org.uk>

Martin v. L?wis wrote:
>> In light of this, what I'd love to see (but sadly can't really help
>> with, and am not optimistic about happening) is for:
>>
>> - python to grow a decent, cross platform, package management system
>>
>> - the standard library to actually shrink to a point where only
>> libraries that are not released elsewhere are included
>>
>> I'd be interested to know how many users of python also felt this way ;-)
>>     
>
> I don't like the standard library to shrink. It's good that batteries
> are included.
>   
I have mixed feelings. It is great that the batteries are included, but 
some batteries are showing their age or not maintained (who maintains 
IDLE? - does the calendar module really warrant being in the standard 
library? - imaplib is really not useful and IMAPClient which isn't in 
the standard library is much better [1]).

If a library is well maintained then there seems to be little point in 
moving it into the standard library as it may actually be harder to 
maintain, and if a library has no active developers then do we really 
want it in the standard library...

On the other hand there are some standard tools that a significant 
portion of the community use (Python Imaging Library and the PyWin32 
extensions for example) that aren't in the standard library.

I think other developers have similar mixed feelings, or at least there 
are enough people on both sides of the fence that it is very hard to 
make changes. Perhaps this is the way it should be.

Michael

[1] http://freshfoo.com/wiki/CodeIndex

> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From lists at cheimes.de  Sun Mar  8 15:30:01 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 08 Mar 2009 15:30:01 +0100
Subject: [Python-Dev] 3.1 performance
In-Reply-To: <loom.20090308T121843-630@post.gmane.org>
References: <loom.20090307T140318-788@post.gmane.org>	<200903081205.52013.victor.stinner@haypocalc.com>
	<loom.20090308T121843-630@post.gmane.org>
Message-ID: <49B3D669.1090305@cheimes.de>

Antoine Pitrou wrote:
> Hi,
> 
> Victor Stinner <victor.stinner <at> haypocalc.com> writes:
>> Summary (minimum total) on 32 bits CPU:
>>  * Python 2.6.1: 8762 ms
>>  * Python 3.0.1: 8977 ms
>>  * Python 3.1a1: 9228 ms (slower than 3.0)
> 
> Have you compiled with or without "--with-computed-gotos"?

Why is the feature still disabled by default?

Christian

PS: Holy moly! Computed gotos totally put my Python on fire! The feature
increases the minimum run-time by approx. 25% and the average run-time
by approx. 40% on my Ubuntu 8.10 box (AMD64, Intel(R) Core(TM)2 CPU
T7600  @ 2.33GHz).
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: goto_bench.txt
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090308/3b6f20ff/attachment.txt>

From martin at v.loewis.de  Sun Mar  8 15:31:32 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Mar 2009 15:31:32 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49B3CFFD.7050905@voidspace.org.uk>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>
	<49B3CD53.5010906@v.loewis.de> <49B3CFFD.7050905@voidspace.org.uk>
Message-ID: <49B3D6C4.4010206@v.loewis.de>

> I have mixed feelings. It is great that the batteries are included, but
> some batteries are showing their age or not maintained (who maintains
> IDLE? - does the calendar module really warrant being in the standard
> library? - imaplib is really not useful and IMAPClient which isn't in
> the standard library is much better [1]).

I certainly agree that little-used modules should be removed (by means
of a proper deprecation procedure). I do think it is fairly important
that IDLE remains included (whether or not gpolo takes ownership of it).

As for imaplib: I can't comment on the library itself, since I never
use it. However, given the number of bug reports that we receive, it
seems there is a fairly significant followership of it.

> If a library is well maintained then there seems to be little point in
> moving it into the standard library as it may actually be harder to
> maintain

It depends. For quickly evolving libraries, it might be harder to
maintain them in the core, as you can't release as quickly as you
desire. In other cases, it simplifies maintenance: whenever a
systematic API change is added, all standard library modules get
typically updated by whoever makes the systematic change. That is
more productive than having each individual maintainer to figure out
what to change in response.

However, I don't think of the maintainer point of view that much:
it's rather the end users (i.e. application developers) who I worry
about: Should we remove regular expressions from Python, just because
the library doing it is unmaintained?

> On the other hand there are some standard tools that a significant
> portion of the community use (Python Imaging Library and the PyWin32
> extensions for example) that aren't in the standard library.

I continue having the same position: if the authors of those respective
libraries would like to contribute it to the core (and eventually drop
the out-of-core fork), then I would be happy to let them do that. Of
course, Guido's cautioning wrt. external libraries that those depend on
still applies: if there are strong external dependencies, the library
would have to be really important to the community to still include it.

> I think other developers have similar mixed feelings, or at least there
> are enough people on both sides of the fence that it is very hard to
> make changes. Perhaps this is the way it should be.

I think so, yes. Decisions will be made on a case-by-case basis, going
either direction one time or the other.

Regards,
Martin

From solipsis at pitrou.net  Sun Mar  8 15:43:24 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 8 Mar 2009 14:43:24 +0000 (UTC)
Subject: [Python-Dev] 3.1 performance
References: <loom.20090307T140318-788@post.gmane.org>	<200903081205.52013.victor.stinner@haypocalc.com>
	<loom.20090308T121843-630@post.gmane.org>
	<49B3D669.1090305@cheimes.de>
Message-ID: <loom.20090308T144158-282@post.gmane.org>

Christian Heimes <lists <at> cheimes.de> writes:
> 
> Why is the feature still disabled by default?

Marc-Andr? expressed concerns that it might trigger compiler issues.

> PS: Holy moly! Computed gotos totally put my Python on fire! The feature
> increases the minimum run-time by approx. 25% and the average run-time
> by approx. 40% on my Ubuntu 8.10 box (AMD64, Intel(R) Core(TM)2 CPU
> T7600  @ 2.33GHz).

You surely mean that it /decreased/ run-time by approx. 20% :-)
I don't think average numbers are significant, but they might indicate that your
system was slightly loaded when running the benchmarks.

Regards

Antoine.



From daniel at stutzbachenterprises.com  Sun Mar  8 15:46:44 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sun, 8 Mar 2009 08:46:44 -0600
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of medusa
In-Reply-To: <18867.11604.171244.871931@montanaro.dyndns.org>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<20090305231149.12555.939089272.divmod.xquotient.5373@weber.divmod.com>
	<gov889$fgd$1@ger.gmane.org>
	<18867.11604.171244.871931@montanaro.dyndns.org>
Message-ID: <eae285400903080746o2605fc21vc5383a77a399ca1b@mail.gmail.com>

On Sat, Mar 7, 2009 at 8:28 PM, <skip at pobox.com> wrote:

> Anybody interested in working on this at a PyCon Sprint?  I won't be
> attending the conference proper, but plan to spend a couple days sprinting,
>

I'll be there and interested. :)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090308/075fd691/attachment-0001.htm>

From steve at holdenweb.com  Sun Mar  8 16:08:16 2009
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 08 Mar 2009 11:08:16 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49B3CD53.5010906@v.loewis.de>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>
	<49B3CD53.5010906@v.loewis.de>
Message-ID: <gp0n10$lco$1@ger.gmane.org>

Martin v. L?wis wrote:
>> In light of this, what I'd love to see (but sadly can't really help
>> with, and am not optimistic about happening) is for:
>>
>> - python to grow a decent, cross platform, package management system
>>
>> - the standard library to actually shrink to a point where only
>> libraries that are not released elsewhere are included
>>
>> I'd be interested to know how many users of python also felt this way ;-)
> 
> I don't like the standard library to shrink. It's good that batteries
> are included.
> 
Perhaps we could encourage more "jumbo" distributions, like Enthought's
and ActiveState's. I suspect many people would rather be able to
maintain their Python functionality as a single product.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From tav at espians.com  Sun Mar  8 16:13:01 2009
From: tav at espians.com (tav)
Date: Sun, 8 Mar 2009 15:13:01 +0000
Subject: [Python-Dev] Ruby-style Blocks in Python [Pseudo-PEP]
Message-ID: <eb24b25b0903080813tff1c3c2o98c1ffbcd2d7d79d@mail.gmail.com>

Hey all,

Apologies for bringing up an old issue, but I think I've worked out a
Pythonic syntax for doing Ruby-style blocks. The short of it is:

    with employees.select do (emp):
        if emp.salary > developer.salary:
            return fireEmployee(emp)
        else:
            return extendContract(emp)

I explain in detail in this blog article:

  http://tav.espians.com/ruby-style-blocks-in-python.html

It covers everything from why these are useful to a proposal of how
the new ``do`` statement and __do__ function could work.

Let me know what you think.

My apologies if I've missed something obvious. Thanks!

-- 
love, tav

plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian

From collinw at gmail.com  Sun Mar  8 16:15:19 2009
From: collinw at gmail.com (Collin Winter)
Date: Sun, 8 Mar 2009 08:15:19 -0700
Subject: [Python-Dev] 3.1 performance
In-Reply-To: <49B3D669.1090305@cheimes.de>
References: <loom.20090307T140318-788@post.gmane.org>
	<200903081205.52013.victor.stinner@haypocalc.com>
	<loom.20090308T121843-630@post.gmane.org>
	<49B3D669.1090305@cheimes.de>
Message-ID: <43aa6ff70903080815g12d1188auc1e2198a30dc07a1@mail.gmail.com>

On Sun, Mar 8, 2009 at 7:30 AM, Christian Heimes <lists at cheimes.de> wrote:
> Antoine Pitrou wrote:
>> Hi,
>>
>> Victor Stinner <victor.stinner <at> haypocalc.com> writes:
>>> Summary (minimum total) on 32 bits CPU:
>>> ?* Python 2.6.1: 8762 ms
>>> ?* Python 3.0.1: 8977 ms
>>> ?* Python 3.1a1: 9228 ms (slower than 3.0)
>>
>> Have you compiled with or without "--with-computed-gotos"?
>
> Why is the feature still disabled by default?
>
> Christian
>
> PS: Holy moly! Computed gotos totally put my Python on fire! The feature
> increases the minimum run-time by approx. 25% and the average run-time
> by approx. 40% on my Ubuntu 8.10 box (AMD64, Intel(R) Core(TM)2 CPU
> T7600 ?@ 2.33GHz).

Note that of the benchmarks tested, PyBench benefits the most from
threaded eval loop designs. Other systems benefit less; for example,
Django template benchmarks were only sped up by 7-8% when I was
testing it.

Collin Winter

From steve at holdenweb.com  Sun Mar  8 16:18:58 2009
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 08 Mar 2009 11:18:58 -0400
Subject: [Python-Dev] RELEASED Python 3.1 alpha 1
In-Reply-To: <20090307201048.GA23840@panix.com>
References: <1afaf6160903071053i559f5608nb37882f680ed41d0@mail.gmail.com>
	<20090307201048.GA23840@panix.com>
Message-ID: <gp0nl1$m8l$1@ger.gmane.org>

Aahz wrote:
> On Sat, Mar 07, 2009, Benjamin Peterson wrote:
>> On behalf of the Python development team and the Python community, I'm
>> happy to announce the first alpha release of Python 3.1.
> 
> Congratulations on your first baby!  Here's to hoping you release many
> more of these!

Yes, well done, Benjamin. Barry Warsaw is walking with a spring in his
steps again ;-)

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From victor.stinner at haypocalc.com  Sun Mar  8 16:23:10 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sun, 8 Mar 2009 16:23:10 +0100
Subject: [Python-Dev] 3.1 performance
In-Reply-To: <loom.20090308T121843-630@post.gmane.org>
References: <loom.20090307T140318-788@post.gmane.org>
	<200903081205.52013.victor.stinner@haypocalc.com>
	<loom.20090308T121843-630@post.gmane.org>
Message-ID: <200903081623.10840.victor.stinner@haypocalc.com>

Le Sunday 08 March 2009 13:20:34 Antoine Pitrou, vous avez ?crit?:
> Hi,
>
> Victor Stinner <victor.stinner <at> haypocalc.com> writes:
> > Summary (minimum total) on 32 bits CPU:
> >  * Python 2.6.1: 8762 ms
> >  * Python 3.0.1: 8977 ms
> >  * Python 3.1a1: 9228 ms (slower than 3.0)
>
> Have you compiled with or without "--with-computed-gotos"?

I used "./configure --prefix=...", no other option. Should I enable it? For 
which version?

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From duncan.booth at suttoncourtenay.org.uk  Sun Mar  8 16:51:35 2009
From: duncan.booth at suttoncourtenay.org.uk (Duncan Booth)
Date: Sun, 8 Mar 2009 15:51:35 +0000 (UTC)
Subject: [Python-Dev] Ruby-style Blocks in Python [Pseudo-PEP]
References: <eb24b25b0903080813tff1c3c2o98c1ffbcd2d7d79d@mail.gmail.com>
Message-ID: <Xns9BC8A144EB08Eduncanrcpcouk@127.0.0.1>

tav <tav at espians.com> wrote:

> I explain in detail in this blog article:
> 
>   http://tav.espians.com/ruby-style-blocks-in-python.html
> 

"This is also possible in Python but at the needless cost of naming and 
defining a function first"

The cost of defining the function first is probably much less than the cost 
of your __do__ function. Your proposal seems to be much more limited than 
passing functions around e.g. Python allows you to pass in multiple 
functions where appropriate, or to store them for later calling. 



From solipsis at pitrou.net  Sun Mar  8 16:59:12 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 8 Mar 2009 15:59:12 +0000 (UTC)
Subject: [Python-Dev] Ruby-style Blocks in Python [Pseudo-PEP]
References: <eb24b25b0903080813tff1c3c2o98c1ffbcd2d7d79d@mail.gmail.com>
Message-ID: <loom.20090308T155837-847@post.gmane.org>

tav <tav <at> espians.com> writes:
> 
> Apologies for bringing up an old issue, but I think I've worked out a
> Pythonic syntax for doing Ruby-style blocks. The short of it is:
> [...]

This thread should probably be redirected to python-ideas.

Regards

Antoine.



From matthew at matthewwilkes.co.uk  Sun Mar  8 17:02:33 2009
From: matthew at matthewwilkes.co.uk (Matthew Wilkes)
Date: Sun, 8 Mar 2009 16:02:33 +0000
Subject: [Python-Dev] Ruby-style Blocks in Python [Pseudo-PEP]
In-Reply-To: <eb24b25b0903080813tff1c3c2o98c1ffbcd2d7d79d@mail.gmail.com>
References: <eb24b25b0903080813tff1c3c2o98c1ffbcd2d7d79d@mail.gmail.com>
Message-ID: <D65143C9-C2DC-422D-9EB4-B46FA1D8932E@matthewwilkes.co.uk>


On 8 Mar 2009, at 15:13, tav wrote:

> Apologies for bringing up an old issue, but I think I've worked out a
> Pythonic syntax for doing Ruby-style blocks. The short of it is:
>
>    with employees.select do (emp):
>        if emp.salary > developer.salary:
>            return fireEmployee(emp)
>        else:
>            return extendContract(emp)

My train of thought when seeing this:

  1. Ok, "with" it's a context manager.
  2. Huh, no it's not, "do", it's a loop.
  3. What on earth is emp?  Where's that defined?  Why the parens?
  4. Where do those returns return to?
  5. I have no idea what this does.

Ah, now, reading your Python alternative, defining a function then  
passing it is, it's more clear.

This is completely incomprehensible to me, it doesn't feel like  
python.  I'd rather define the throwaway function, anonymous callables  
that do complex things aren't my kind of thing.  Give them a sensible  
name, put them in a utils module, and import it where needed, it's  
much clearer.

Matt

From glyph at divmod.com  Sun Mar  8 17:27:04 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Sun, 08 Mar 2009 16:27:04 -0000
Subject: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version
	of	medusa
In-Reply-To: <eae285400903080746o2605fc21vc5383a77a399ca1b@mail.gmail.com>
References: <ca471dc20903041046te8fed62g66612497e949c42e@mail.gmail.com>
	<20090304185106.12853.1804820039.divmod.quotient.18062@henry.divmod.com>
	<ca471dc20903041054k1cb869d1if2e1b593b8dbe0c0@mail.gmail.com>
	<eae285400903041231u69eb40a5n2597d2d6081e395b@mail.gmail.com>
	<49AEF247.9060800@plope.com> <gop99c$et3$1@ger.gmane.org>
	<20090305231149.12555.939089272.divmod.xquotient.5373@weber.divmod.com>
	<gov889$fgd$1@ger.gmane.org>
	<18867.11604.171244.871931@montanaro.dyndns.org>
	<eae285400903080746o2605fc21vc5383a77a399ca1b@mail.gmail.com>
Message-ID: <20090308162704.12555.2054887304.divmod.xquotient.5560@weber.divmod.com>

On 02:46 pm, daniel at stutzbachenterprises.com wrote:
>On Sat, Mar 7, 2009 at 8:28 PM, <skip at pobox.com> wrote:
>>Anybody interested in working on this at a PyCon Sprint?  I won't be
>>attending the conference proper, but plan to spend a couple days 
>>sprinting,

>I'll be there and interested. :)

Great!  I plan to hold both of you to that :).

From guido at python.org  Sun Mar  8 17:30:15 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 8 Mar 2009 09:30:15 -0700
Subject: [Python-Dev] 3.1 performance
In-Reply-To: <200903081205.52013.victor.stinner@haypocalc.com>
References: <loom.20090307T140318-788@post.gmane.org>
	<200903081205.52013.victor.stinner@haypocalc.com>
Message-ID: <ca471dc20903080930n38be0bfakbdab82571883fc6d@mail.gmail.com>

On Sun, Mar 8, 2009 at 4:05 AM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> I just downloaded Python 2.6.1, 3.0.1 and 3.1alpha1, compiled them on 32 and
> 64 bits CPU, and ran pybench 2.1(*).
>
> Summary (minimum total) on 32 bits CPU:
> ?* Python 2.6.1: 8762 ms
> ?* Python 3.0.1: 8977 ms
> ?* Python 3.1a1: 9228 ms (slower than 3.0)
>
> Summary (minimum total) on 64 bits CPU:
> ?* Python 2.6.1: 4219 ms
> ?* Python 3.0.1: 4502 ms
> ?* Python 3.1a1: 4442 ms (faster than 3.0)
>
> I also ran pybench once, is it enough? See attached files for the details.
>
> (*) I copied pybench 2.1 from Pthon 3.1alpha1 because 2.6.1 only
> ? ?includes pybench 2.0

How are these numbers significant? IIUC that is not at all how pybench
is supposed to be used. Its strength is that it lets you watch the
relative performance of many individual operations. I don't think
adding up the numbers for all operations gives a very useful total,
since each individual timing loop seems to be scaled to last around
50-100 msec; this means the operation mix is probably vastly different
from that occurring in real operations.

What I'd be interested in however would be a list of which operations
got speeded up the most and which slowed down the most. That might
stir up someone's memory of a change that was made in that operations
that could explain the performance change (especially for slow-downs,
of course :-).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Mar  8 17:34:39 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 8 Mar 2009 09:34:39 -0700
Subject: [Python-Dev] Ruby-style Blocks in Python [Pseudo-PEP]
In-Reply-To: <eb24b25b0903080813tff1c3c2o98c1ffbcd2d7d79d@mail.gmail.com>
References: <eb24b25b0903080813tff1c3c2o98c1ffbcd2d7d79d@mail.gmail.com>
Message-ID: <ca471dc20903080934g6c7f35d8p1162caef6118a885@mail.gmail.com>

On Sun, Mar 8, 2009 at 8:13 AM, tav <tav at espians.com> wrote:
> Apologies for bringing up an old issue, but I think I've worked out a
> Pythonic syntax for doing Ruby-style blocks.

As a point of order, please move this discussion to python-ideas,
where it belongs roughly until the time a PEP might be ready for
approval.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Sun Mar  8 17:53:32 2009
From: barry at python.org (Barry Warsaw)
Date: Sun, 8 Mar 2009 12:53:32 -0400
Subject: [Python-Dev] RELEASED Python 3.1 alpha 1
In-Reply-To: <gp0nl1$m8l$1@ger.gmane.org>
References: <1afaf6160903071053i559f5608nb37882f680ed41d0@mail.gmail.com>
	<20090307201048.GA23840@panix.com> <gp0nl1$m8l$1@ger.gmane.org>
Message-ID: <2B25376B-BA40-4C98-B144-A1B0CA9B1551@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 8, 2009, at 11:18 AM, Steve Holden wrote:

> Aahz wrote:
>> On Sat, Mar 07, 2009, Benjamin Peterson wrote:
>>> On behalf of the Python development team and the Python community,  
>>> I'm
>>> happy to announce the first alpha release of Python 3.1.
>>
>> Congratulations on your first baby!  Here's to hoping you release  
>> many
>> more of these!
>
> Yes, well done, Benjamin. Barry Warsaw is walking with a spring in his
> steps again ;-)

I was wondering why the weather here in DC has turned so beautiful  
after last week's snow storm!

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSbP4DXEjvBPtnXfVAQJ3DgP/SI+Um8xbOP5I64dDz5y9FDj+OL/quL8U
h1pCZsaekK6NMmMZmACylsj6a8ZUg7h6Mm9RVLYgwyYje/3mEOHxNIaL4mgLBtr8
sQJS3UYyxK/uaf6UxTodWJFnFkeyx291sp3jlPppjDDyNrHK48pwEZ2+ZpCL9w9t
y3sqVsGAy40=
=h8nT
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Sun Mar  8 18:03:14 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Mar 2009 18:03:14 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <gp0n10$lco$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>
	<gp0n10$lco$1@ger.gmane.org>
Message-ID: <49B3FA52.5010508@v.loewis.de>

> Perhaps we could encourage more "jumbo" distributions, like Enthought's
> and ActiveState's. I suspect many people would rather be able to
> maintain their Python functionality as a single product.

I think the concept of "jumbo distribution" has been lost. I mentioned
it to one of the Enthought people (sorry, forgot who exactly), and he
said he had never heard the term.

Looking back, it seems that you have to be a commercial enterprise to
produce such a thing. There is the python.org distribution, where many
volunteers maintain it, and then there are the two (?) free-commercial
distributions (ActivePython, and EPD). I'm skeptical that there can be
motivation for creating another "community" jumbo distribution - why
would anybody put efforts into maintaining it? You don't get much credit
for release engineering - except from fellow release engineers.

In addition, you have the Linux distributions, which you can also count
as jumbo Python distributions (and also jumbo Perl, jumbo Java, jumbo
LISP - at least for Debian :-). Again, many of these are commercially
based, although there still seems to be space for multiple community
distributions (Debian, Gentoo).

This is precisely the reason why I want Python to continue including its
batteries. If we give that up, it will not come back, and users get
frustrated that they have to collect stuff from many places, and that
the stuff doesn't fit together completely.

What that means for BeautifulSoup, I don't know. First, its authors
would have to offer contributing it, and then experts should also
agree that this would be a useful inclusion. Some apparently say that
html5lib would be a better choice. If that's the thing that is currently
on release 0.11, then I think we should take no action at this point -
I don't want to include anything that has version 0.11.

Regards,
Martin

From henning.vonbargen at arcor.de  Sun Mar  8 18:53:04 2009
From: henning.vonbargen at arcor.de (Henning von Bargen)
Date: Sun, 8 Mar 2009 18:53:04 +0100
Subject: [Python-Dev] Ruby-style Blocks in Python
References: <mailman.5518.1236528158.11745.python-dev@python.org>
Message-ID: <F4B779A8C0E94DC8B02F3738CD88304C@max>

I totally agree with Matthew.
I don't understand the code at first sight.
The "with ... do ..." syntax breaks the
"Python is executable pseudo-code" pattern.

And according the example given at Tav's web site:

with employees.select do (emp):
    if emp.salary > developer.salary:
        return fireEmployee(emp)
    else:
        return extendContract(emp)

It is neither shorter nor clearer than the same thing with existing Python 
syntax:

for emp in employees.select:
    # or should it be "select()"?
    if emp.salary > developer.salary:
        fireEmployee(emp)
    else:
        extendContract(emp)

BTW it seems to me that Python development might go the wrong direction.

Using Python for the development of commercial programs for years,
I never felt the need to use anything more complicated than generators or 
decorators.

Security, performance, and the "batteries included" are by far more 
important
than esotheric features which Joe Average doesn't understand anyway
(and therefore hopefully won't use it).

If there's anything where Python should be improved, it's static code 
analysis
(like pylint) in the standard lib and optional type declarations, which 
could be
used to assist pylint and friends as well as JIT compilers, Cython,....
and not even more Meta-thingies.

Sorry, I just had to say this once.

Henning


From janssen at parc.com  Sun Mar  8 19:37:22 2009
From: janssen at parc.com (Bill Janssen)
Date: Sun, 8 Mar 2009 11:37:22 PDT
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49B3D6C4.4010206@v.loewis.de>
References: <49ABCFC9.1070508@gmail.com> <49AE9698.1020806@simplistix.co.uk>
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
	<gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<49B3B2E8.4050605@simplistix.co.uk> <49B3CD53.5010906@v.loewis.de>
	<49B3CFFD.7050905@voidspace.org.uk> <49B3D6C4.4010206@v.loewis.de>
Message-ID: <13446.1236537442@parc.com>

Martin v. L?wis <martin at v.loewis.de> wrote:

> > If a library is well maintained then there seems to be little point in
> > moving it into the standard library as it may actually be harder to
> > maintain
> 
> It depends. For quickly evolving libraries, it might be harder to
> maintain them in the core, as you can't release as quickly as you
> desire. In other cases, it simplifies maintenance: whenever a
> systematic API change is added, all standard library modules get
> typically updated by whoever makes the systematic change. That is
> more productive than having each individual maintainer to figure out
> what to change in response.

This is a complicated issue.  But two sub-threads seem to be about (1)
modules dependent (or wrapping) a large, complicated third-party library
under active development, and (2) hard-to-routinely-test modules, like
imaplib.

Bill

From janssen at parc.com  Sun Mar  8 19:39:28 2009
From: janssen at parc.com (Bill Janssen)
Date: Sun, 8 Mar 2009 11:39:28 PDT
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49B3FA52.5010508@v.loewis.de>
References: <49ABCFC9.1070508@gmail.com> <49AE9698.1020806@simplistix.co.uk>
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
	<gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<49B3B2E8.4050605@simplistix.co.uk>
	<49B3CD53.5010906@v.loewis.de> <gp0n10$lco$1@ger.gmane.org>
	<49B3FA52.5010508@v.loewis.de>
Message-ID: <13497.1236537568@parc.com>

Martin v. L?wis <martin at v.loewis.de> wrote:

> then there are the two (?) free-commercial
> distributions (ActivePython, and EPD).

Apple's system Python seems to qualify; it comes with a huge additional
library, including numpy and Twisted.

Bill

From jimjjewett at gmail.com  Sun Mar  8 20:51:51 2009
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 8 Mar 2009 15:51:51 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
Message-ID: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>

Michael Foord wrote:
> Chris Withers wrote:
>> ... love to see ... but ... not optimistic

>> - python to grow a decent, cross platform, package management system

As stated, this may be impossible, because of the difference in what a
package should mean on Windows vs Unix.

If you just mean a way to add python packages from pypi as with
EasyInstall, then maybe.

>> - the standard library to actually shrink to a point where only
>> libraries that are not released elsewhere are included

In some environments, each new component must be approved.  Once
python is approved, the standard library is OK, but adding 7 packages
from pypi requires 7 more sets of approvals.

On the other hand, if there were a way to say "The PSF explicitly
endorses Packages X, Y, and Z as worthy of the stdlib; they are
distributed separately for administrative reasons", then the initial
request could be for "Python plus officially endorsed addons"

That said, it may make sense to just give greater prominence to
existing repackagers, such as ActiveState or Enthought.

> If a library is well maintained then there seems to be little point in
> moving it into the standard library

The official endorsement is in many cases more important than shared
distribution.

-jJ

From fabiofz at gmail.com  Sun Mar  8 21:07:46 2009
From: fabiofz at gmail.com (Fabio Zadrozny)
Date: Sun, 8 Mar 2009 17:07:46 -0300
Subject: [Python-Dev] Python 3.0 grammar ambiguous?
Message-ID: <cfb578b20903081307l738c07d5g2888f0d13f9e6309@mail.gmail.com>

Hi All,

I'm trying to parse Python 3.0 following the Python 3.0 grammar from:
http://svn.python.org/projects/python/branches/py3k/Grammar/Grammar

Now, when getting to the arglist, it seems that the grammar is
ambiguous, and I was wondering how does Python disambiguate that (I'll
not put the whole grammar here, just the part that appears to be
ambiguous):

arglist: (argument ',')*
                        (argument [',']
                         |'*' test (',' argument)* [',' '**' test]
                         |'**' test
                         )

argument: test [comp_for]
test: or_test
or_test: and_test
and_test: not_test
not_test: 'not' not_test | comparison
comparison: star_expr
star_expr: ['*'] expr


So, with that construct, having call(arglist) in a format:
call(*test), the grammar would find it to be consumed in the argument
construction (because of the start_expr) and not in the arglist in the
'*' test (because the construct is ambiguous and the argument
construct comes first), so, I was wondering how does Python
disambiguate that... anyone has any pointers on it? It appears that
happened while adding PEP 3132.

Am I missing something here?

Thanks,

Fabio

From martin at v.loewis.de  Sun Mar  8 21:20:34 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Mar 2009 21:20:34 +0100
Subject: [Python-Dev] Python 3.0 grammar ambiguous?
In-Reply-To: <cfb578b20903081307l738c07d5g2888f0d13f9e6309@mail.gmail.com>
References: <cfb578b20903081307l738c07d5g2888f0d13f9e6309@mail.gmail.com>
Message-ID: <49B42892.20903@v.loewis.de>

Fabio Zadrozny wrote:
> I was wondering how does Python
> disambiguate that... anyone has any pointers on it?

That is easy to answer:

py> parser.expr("f(*x)").totuple()
(258, (326, (303, (304, (305, (306, (307, (309, (310, (311, (312, (313,
(314, (315, (316, (317, (1, 'f')), (321, (7, '('), (329, (16, '*'),
(303, (304, (305, (306, (307, (309, (310, (311, (312, (313, (314, (315,
(316, (317, (1, 'x')))))))))))))))), (8, ')')))))))))))))))), (4, ''),
(0, ''))
py> symbol.arglist
329

So much for the "how", I don't know why it makes this choice; notice
that this is the better choice, though:

py> f((*x))
  File "<stdin>", line 1
SyntaxError: can use starred expression only as assignment target

Regards,
Martin

From ncoghlan at gmail.com  Sun Mar  8 21:37:22 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 09 Mar 2009 06:37:22 +1000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
Message-ID: <49B42C82.3060903@gmail.com>

Jim Jewett wrote:
> That said, it may make sense to just give greater prominence to
> existing repackagers, such as ActiveState or Enthought.

Perhaps we should think about linking to the relevant
ActiveState/Enthought distributions from the pydotorg download pages?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From fabiofz at gmail.com  Sun Mar  8 21:38:49 2009
From: fabiofz at gmail.com (Fabio Zadrozny)
Date: Sun, 8 Mar 2009 17:38:49 -0300
Subject: [Python-Dev] Python 3.0 grammar ambiguous?
In-Reply-To: <49B42892.20903@v.loewis.de>
References: <cfb578b20903081307l738c07d5g2888f0d13f9e6309@mail.gmail.com>
	<49B42892.20903@v.loewis.de>
Message-ID: <cfb578b20903081338i19680dc1v36206bb9d6bc1412@mail.gmail.com>

>> I was wondering how does Python
>> disambiguate that... anyone has any pointers on it?
>
> That is easy to answer:
>
> py> parser.expr("f(*x)").totuple()
> (258, (326, (303, (304, (305, (306, (307, (309, (310, (311, (312, (313,
> (314, (315, (316, (317, (1, 'f')), (321, (7, '('), (329, (16, '*'),
> (303, (304, (305, (306, (307, (309, (310, (311, (312, (313, (314, (315,
> (316, (317, (1, 'x')))))))))))))))), (8, ')')))))))))))))))), (4, ''),
> (0, ''))
> py> symbol.arglist
> 329
>
> So much for the "how", I don't know why it makes this choice; notice
> that this is the better choice, though:
>
> py> f((*x))
> ?File "<stdin>", line 1
> SyntaxError: can use starred expression only as assignment target


Yeap, very strange that it works... I can't get it to work in JavaCC.

I'm considering setting arglist as
((argument() [','])+  ['**' test])
| '**' test

Because it is able to handle the constructs removing the ambiguity,
and make things right semantically later on, but I don't like the idea
of being so different from the official grammar (although I'm running
out of choices).

Thanks,

Fabio

From ncoghlan at gmail.com  Sun Mar  8 22:18:08 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 09 Mar 2009 07:18:08 +1000
Subject: [Python-Dev] Python 3.0 grammar ambiguous?
In-Reply-To: <cfb578b20903081338i19680dc1v36206bb9d6bc1412@mail.gmail.com>
References: <cfb578b20903081307l738c07d5g2888f0d13f9e6309@mail.gmail.com>	<49B42892.20903@v.loewis.de>
	<cfb578b20903081338i19680dc1v36206bb9d6bc1412@mail.gmail.com>
Message-ID: <49B43610.2040107@gmail.com>

Fabio Zadrozny wrote:
> Because it is able to handle the constructs removing the ambiguity,
> and make things right semantically later on, but I don't like the idea
> of being so different from the official grammar (although I'm running
> out of choices).

I'm not sure it is the same (or at least related) problem, but a parsing
ambiguity was also reported for the lambda varargslist in 2.x:
http://bugs.python.org/issue2009

I think I'd be happier with fiddling the Grammar to eliminate these
problems if Guido was to chime in as the author of pgen though... (since
pgen appears to be able to make sense of the grammar as it currently
stands, while other parsers sometimes have trouble)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From guido at python.org  Sun Mar  8 23:23:25 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 8 Mar 2009 15:23:25 -0700
Subject: [Python-Dev] Python 3.0 grammar ambiguous?
In-Reply-To: <49B43610.2040107@gmail.com>
References: <cfb578b20903081307l738c07d5g2888f0d13f9e6309@mail.gmail.com>
	<49B42892.20903@v.loewis.de>
	<cfb578b20903081338i19680dc1v36206bb9d6bc1412@mail.gmail.com>
	<49B43610.2040107@gmail.com>
Message-ID: <ca471dc20903081523t5cb7abf0tc943843c1060e49f@mail.gmail.com>

On Sun, Mar 8, 2009 at 2:18 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Fabio Zadrozny wrote:
>> Because it is able to handle the constructs removing the ambiguity,
>> and make things right semantically later on, but I don't like the idea
>> of being so different from the official grammar (although I'm running
>> out of choices).
>
> I'm not sure it is the same (or at least related) problem, but a parsing
> ambiguity was also reported for the lambda varargslist in 2.x:
> http://bugs.python.org/issue2009
>
> I think I'd be happier with fiddling the Grammar to eliminate these
> problems if Guido was to chime in as the author of pgen though... (since
> pgen appears to be able to make sense of the grammar as it currently
> stands, while other parsers sometimes have trouble)

To be honest I wasn't aware of this ambiguity. It seems that whoever
wrote the patch for argument unpacking (a, b, *c = ...) got "lucky"
with an ambiguous grammar. This surprises me, because IIRC usually
pgen doesn't like ambiguities. Other parser generators usually have
some way to deal with ambiguous grammars, but they also usually have
features that make it unnecessary to use the exact same grammar as
pgen -- for example, most parser generators are able to backtrack or
look ahead more than one token, so that they can distinguish between
"a = b" and "a" once the '=' token is seen, rather than having to
commit to parse an expression first.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Sun Mar  8 23:28:18 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Mar 2009 23:28:18 +0100
Subject: [Python-Dev] Python 3.0 grammar ambiguous?
In-Reply-To: <ca471dc20903081523t5cb7abf0tc943843c1060e49f@mail.gmail.com>
References: <cfb578b20903081307l738c07d5g2888f0d13f9e6309@mail.gmail.com>	
	<49B42892.20903@v.loewis.de>	
	<cfb578b20903081338i19680dc1v36206bb9d6bc1412@mail.gmail.com>	
	<49B43610.2040107@gmail.com>
	<ca471dc20903081523t5cb7abf0tc943843c1060e49f@mail.gmail.com>
Message-ID: <49B44682.1030603@v.loewis.de>

> To be honest I wasn't aware of this ambiguity. It seems that whoever
> wrote the patch for argument unpacking (a, b, *c = ...) got "lucky"
> with an ambiguous grammar. This surprises me, because IIRC usually
> pgen doesn't like ambiguities. 

So how does it resolve the ambiguity?

Regards,
Martin

From guido at python.org  Sun Mar  8 23:45:05 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 8 Mar 2009 15:45:05 -0700
Subject: [Python-Dev] Python 3.0 grammar ambiguous?
In-Reply-To: <49B44682.1030603@v.loewis.de>
References: <cfb578b20903081307l738c07d5g2888f0d13f9e6309@mail.gmail.com>
	<49B42892.20903@v.loewis.de>
	<cfb578b20903081338i19680dc1v36206bb9d6bc1412@mail.gmail.com>
	<49B43610.2040107@gmail.com>
	<ca471dc20903081523t5cb7abf0tc943843c1060e49f@mail.gmail.com>
	<49B44682.1030603@v.loewis.de>
Message-ID: <ca471dc20903081545pd0f9d0dx7dbac844f4b7a819@mail.gmail.com>

On Sun, Mar 8, 2009 at 3:28 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> To be honest I wasn't aware of this ambiguity. It seems that whoever
>> wrote the patch for argument unpacking (a, b, *c = ...) got "lucky"
>> with an ambiguous grammar. This surprises me, because IIRC usually
>> pgen doesn't like ambiguities.
>
> So how does it resolve the ambiguity?

I have no idea. :-(

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fabiofz at gmail.com  Sun Mar  8 23:58:06 2009
From: fabiofz at gmail.com (Fabio Zadrozny)
Date: Sun, 8 Mar 2009 19:58:06 -0300
Subject: [Python-Dev] Python 3.0 grammar ambiguous?
In-Reply-To: <ca471dc20903081523t5cb7abf0tc943843c1060e49f@mail.gmail.com>
References: <cfb578b20903081307l738c07d5g2888f0d13f9e6309@mail.gmail.com>
	<49B42892.20903@v.loewis.de>
	<cfb578b20903081338i19680dc1v36206bb9d6bc1412@mail.gmail.com>
	<49B43610.2040107@gmail.com>
	<ca471dc20903081523t5cb7abf0tc943843c1060e49f@mail.gmail.com>
Message-ID: <cfb578b20903081558l4ef9f89dw8ecf4903269e2753@mail.gmail.com>

> To be honest I wasn't aware of this ambiguity. It seems that whoever
> wrote the patch for argument unpacking (a, b, *c = ...) got "lucky"
> with an ambiguous grammar. This surprises me, because IIRC usually
> pgen doesn't like ambiguities. Other parser generators usually have
> some way to deal with ambiguous grammars, but they also usually have
> features that make it unnecessary to use the exact same grammar as
> pgen -- for example, most parser generators are able to backtrack or
> look ahead more than one token, so that they can distinguish between
> "a = b" and "a" once the '=' token is seen, rather than having to
> commit to parse an expression first.

JavaCC can actually do that, but in the current construct, the
ambiguity is not dependent on a lookahead, because both '*' test and
star_expr will match it equally well -- because they're actually the
same thing grammar-wise (so, there's no way to disambiguate without a
semantic handling later on)

After taking a 2nd look, I think that probably the best solution would
be creating a new testlist to be used from the expr_stmt -- something
like testlist_start_expr.

E.g.:

testlist: test (',' test)* [',']
testlist_star_expr:  [test | star_expr] (',' [test | star_expr])* [',']

And also, star_expr could probably be just
'*' NAME
(to make it faster to match -- or could it match something else?)

Note that I still haven't tested this -- I'll have time to do it
tomorrow on my JavaCC grammar (so, I can give you a better feedback if
this actually works).

Regards,

Fabio

From rdmurray at bitdance.com  Mon Mar  9 02:09:24 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 8 Mar 2009 21:09:24 -0400 (EDT)
Subject: [Python-Dev] BZR mirror and pushing to Launchpad
Message-ID: <Pine.LNX.4.64.0903082028560.2596@kimball.webabinitio.net>

I just posted a (tiny) patch to the tracker, and for the
exercise of it I thought I would push the branch out to Launchpad
as suggested in the wiki (http://wiki.python.org/moin/Bazaar).
It looks like it is uploading every file in the branch instead
of the delta from the trunk.  Did I do something wrong in my local
bzr setup, or is this the expected behavior?

-- R. David Murray           http://www.bitdance.com

From rlight2 at gmail.com  Mon Mar  9 02:28:00 2009
From: rlight2 at gmail.com (Ross Light)
Date: Sun, 8 Mar 2009 18:28:00 -0700
Subject: [Python-Dev] BZR mirror and pushing to Launchpad
In-Reply-To: <Pine.LNX.4.64.0903082028560.2596@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0903082028560.2596@kimball.webabinitio.net>
Message-ID: <4553f0903081828u4ee200far9d97a524cf150638@mail.gmail.com>

Yes, this is the expected behavior.  Bazaar will upload all of the
revisions since it is not stacking off of another branch.  You could
try using the Launchpad or Python.org mirrors as a stacking branch, as
described here:

http://doc.bazaar-vcs.org/bzr.dev/en/user-guide/index.html#using-stacked-branches

Cheers,
Ross Light

On Sun, Mar 8, 2009 at 6:09 PM, R. David Murray <rdmurray at bitdance.com> wrote:
>
> I just posted a (tiny) patch to the tracker, and for the
> exercise of it I thought I would push the branch out to Launchpad
> as suggested in the wiki (http://wiki.python.org/moin/Bazaar).
> It looks like it is uploading every file in the branch instead
> of the delta from the trunk. ?Did I do something wrong in my local
> bzr setup, or is this the expected behavior?
>
> -- R. David Murray ? ? ? ? ? http://www.bitdance.com
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/rlight2%40gmail.com

From andrew-pythondev at puzzling.org  Mon Mar  9 03:37:54 2009
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Mon, 9 Mar 2009 12:37:54 +1000
Subject: [Python-Dev] BZR mirror and pushing to Launchpad
In-Reply-To: <Pine.LNX.4.64.0903082028560.2596@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0903082028560.2596@kimball.webabinitio.net>
Message-ID: <20090309023754.GC6920@steerpike.home.puzzling.org>

R. David Murray wrote:
> I just posted a (tiny) patch to the tracker, and for the
> exercise of it I thought I would push the branch out to Launchpad
> as suggested in the wiki (http://wiki.python.org/moin/Bazaar).
> It looks like it is uploading every file in the branch instead
> of the delta from the trunk.  Did I do something wrong in my local
> bzr setup, or is this the expected behavior?

It should Just Work, i.e. pushing that branch to Launchpad should stack
automatically.  I just spoke to one of the code.launchpad.net developers and
everything seems to be working normally on his end.  Can you provide a bit
more information about exactly what you did, so we can try to reproduce the
problem?

Also, as this may be more of an issue with Launchpad than the Python bzr
import, it might be better to continue this conversation on launchpad-users
(subscribe at https://launchpad.net/~launchpad-users) rather than
python-dev.  I guess it depends on whether or not python-dev would consider
this noise or not...

-Andrew.


From andrew-pythondev at puzzling.org  Mon Mar  9 03:40:59 2009
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Mon, 9 Mar 2009 12:40:59 +1000
Subject: [Python-Dev] BZR mirror and pushing to Launchpad
In-Reply-To: <4553f0903081828u4ee200far9d97a524cf150638@mail.gmail.com>
References: <Pine.LNX.4.64.0903082028560.2596@kimball.webabinitio.net>
	<4553f0903081828u4ee200far9d97a524cf150638@mail.gmail.com>
Message-ID: <20090309024059.GD6920@steerpike.home.puzzling.org>

Ross Light wrote:
> Yes, this is the expected behavior.  Bazaar will upload all of the
> revisions since it is not stacking off of another branch.  You could
> try using the Launchpad or Python.org mirrors as a stacking branch, as
> described here:
> 
> http://doc.bazaar-vcs.org/bzr.dev/en/user-guide/index.html#using-stacked-branches

Actually, the Launchpad code server is configured to make bzr automatically
stack on the appropriate branch.  The docs you point to there are about manually
specifying the branch to stack on.

-Andrew.


From MLMLists at Comcast.net  Mon Mar  9 04:04:13 2009
From: MLMLists at Comcast.net (Mitchell L Model)
Date: Sun, 8 Mar 2009 23:04:13 -0400
Subject: [Python-Dev] IDLE maintenance
In-Reply-To: <mailman.5571.1236547101.11745.python-dev@python.org>
References: <mailman.5571.1236547101.11745.python-dev@python.org>
Message-ID: <p06240800c5da353385b2@[10.0.1.221]>

I've been trying to decipher the various mentions of IDLE maintenance 
on this list to determine the likely future (not necessarily feature 
addition, just keeping it working with each new version of Python 
and, I suppose, Tk) , who is currently responsible for it, and who 
will be responsible for it. Presumably these questions have official 
answers somewhere, but I wouldn't know where to find them.

I have several strong reasons to want IDLE to survive, including 
using it in teaching and writing, where it becomes absurd to provide 
instructions full of variations based on different IDE's. IDLE is 
sufficient for introductory courses and books, and useful for many 
purposes even beyond that. The community should want to minimize the 
entry friction new users experience, and new users in these times 
will not necessarily have any experience with command-line shells or 
editors like vi and emacs. IDLE is at just the right level for 
someone new to Python to just jump in and start exploring or even for 
someone with reasonable Python experience investigating a new area of 
the library.

As I said, I don't know the plans or people surrounding IDLE are. If 
it needs a new maintainer I hereby volunteer. Discussions about my 
qualifications should probably be conducted through email to me 
personally, rather than through this mailing list, using the address 
I use when submitting issues to the tracker: mlm at acm.org.
-- 
		Mitchell L Model, Ph.D.
         mlm at acm.org		MLM Consulting
         (508) 358-8055 (v)		42 Oxbow Road
         (508) 641-7772 (m)		Wayland, MA 01778

From bugbee at mac.com  Mon Mar  9 05:10:02 2009
From: bugbee at mac.com (Larry Bugbee)
Date: Sun, 08 Mar 2009 21:10:02 -0700
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <mailman.5502.1236523607.11745.python-dev@python.org>
References: <mailman.5502.1236523607.11745.python-dev@python.org>
Message-ID: <6CD71A59-E79A-495E-BDB6-24A1CDAAD7D5@mac.com>

>>> In light of this, what I'd love to see (but sadly can't really help
>>> with, and am not optimistic about happening) is for:
>>>
>>> - python to grow a decent, cross platform, package management system
>>>
>>> - the standard library to actually shrink to a point where only
>>> libraries that are not released elsewhere are included
>>>
>>> I'd be interested to know how many users of python also felt this  
>>> way ;-)
>>>
>>
>> I don't like the standard library to shrink. It's good that batteries
>> are included.
>>
> I have mixed feelings. It is great that the batteries are included,  
> but
> some batteries are showing their age or not maintained (who maintains
> IDLE? - does the calendar module really warrant being in the standard
> library? - imaplib is really not useful and IMAPClient which isn't in
> the standard library is much better [1]).
>
> If a library is well maintained then there seems to be little point in
> moving it into the standard library as it may actually be harder to
> maintain, and if a library has no active developers then do we really
> want it in the standard library...
>
> On the other hand there are some standard tools that a significant
> portion of the community use (Python Imaging Library and the PyWin32
> extensions for example) that aren't in the standard library.
>
> I think other developers have similar mixed feelings, or at least  
> there
> are enough people on both sides of the fence that it is very hard to
> make changes. Perhaps this is the way it should be.

Batteries Included should not be interpreted as Soup and Nuts or the  
Kitchen Sink.  There are a lot of good, no, outstanding libraries out  
there that are excellent candidates for the standard library.  IMO,  
the standard library should include those things that would be truly  
helpful getting small projects started.  Beyond that the needs of the  
project will likely be specialized to the point that one should be  
using libraries beyond the standard libraries.  Heck, there are things  
I use frequently that I'd like to be "standard", but could be an  
imposition on the rest of the Python community.  The last thing I want  
to see is a library so large that we spend time figuring out how to  
cull unwanted items.

Beautiful Soup and html5lib are good, but specialized libraries, and  
not, IMO, of *general* interest.  The same goes for lxml, which by the  
way, I like and have used.  ...but it doesn't have to be in the  
standard library to be useful.

I'd like to suggest that any new candidate for the standard library be  
discussed and then set aside for a cooling off period of ONE YEAR.  If  
then folks can all agree the library is not only Goodness, but of  
general interest, especially for bootstrapping small projects, then  
take a vote, or the BDFL can decide.

A key criteria should be, "Will the new library help small projects  
get started by providing basic capabilities without introducing a  
steep learning curve?"

Larry





From rlight2 at gmail.com  Mon Mar  9 05:20:44 2009
From: rlight2 at gmail.com (Ross Light)
Date: Sun, 8 Mar 2009 21:20:44 -0700
Subject: [Python-Dev] BZR mirror and pushing to Launchpad
In-Reply-To: <4553f0903082119h4678c91dg43eaee3c2814be4e@mail.gmail.com>
References: <Pine.LNX.4.64.0903082028560.2596@kimball.webabinitio.net>
	<4553f0903081828u4ee200far9d97a524cf150638@mail.gmail.com>
	<20090309024059.GD6920@steerpike.home.puzzling.org>
	<4553f0903082119h4678c91dg43eaee3c2814be4e@mail.gmail.com>
Message-ID: <4553f0903082120i413f6bc7jc6be7f17cea956a@mail.gmail.com>

On Sun, Mar 8, 2009 at 7:40 PM, Andrew Bennetts
<andrew-pythondev at puzzling.org> wrote:
> Actually, the Launchpad code server is configured to make bzr automatically
> stack on the appropriate branch. ?The docs you point to there are about manually
> specifying the branch to stack on.
>
> -Andrew.

However, my understanding is that Launchpad will not automatically
stack if you are using the Python.org branch; you must use the
Launchpad mirror. ?If you want to stack off the Python.org branch,
then I think you need to manually stack.

Cheers,
Ross Light

(sorry for possible dupe posting, I thought my mail client was sending
to python-dev, but it didn't seem to)

From andrew-pythondev at puzzling.org  Mon Mar  9 06:26:49 2009
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Mon, 9 Mar 2009 15:26:49 +1000
Subject: [Python-Dev] BZR mirror and pushing to Launchpad
In-Reply-To: <4553f0903082120i413f6bc7jc6be7f17cea956a@mail.gmail.com>
References: <Pine.LNX.4.64.0903082028560.2596@kimball.webabinitio.net>
	<4553f0903081828u4ee200far9d97a524cf150638@mail.gmail.com>
	<20090309024059.GD6920@steerpike.home.puzzling.org>
	<4553f0903082119h4678c91dg43eaee3c2814be4e@mail.gmail.com>
	<4553f0903082120i413f6bc7jc6be7f17cea956a@mail.gmail.com>
Message-ID: <20090309052649.GF6920@steerpike.home.puzzling.org>

Ross Light wrote:
[...]
> However, my understanding is that Launchpad will not automatically
> stack if you are using the Python.org branch; you must use the
> Launchpad mirror. ?If you want to stack off the Python.org branch,
> then I think you need to manually stack.

It is true that if the source branch that you push to Launchpad is not in a
stacking-capable format, then the automatic stacking will not happen.  But
that shouldn't be the case with the Python.org branch;
<http://code.python.org/python/trunk/> uses the latest format which is
stacking-capable.

On the other hand, it does appear that
<https://code.launchpad.net/~rdmurray/python/bug5450> is a branch that is
not stacking-capable.  I'm not sure how this could happen by following the
instructions on the the wiki page.

-Andrew.


From python at rcn.com  Mon Mar  9 06:35:23 2009
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 8 Mar 2009 22:35:23 -0700
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de><gp0n10$lco$1@ger.gmane.org>
	<49B3FA52.5010508@v.loewis.de>
Message-ID: <CF383A595CEF4327BAC9F10A9D26F153@RaymondLaptop1>

> This is precisely the reason why I want Python to continue including its
> batteries. If we give that up, it will not come back, and users get
> frustrated that they have to collect stuff from many places, and that
> the stuff doesn't fit together completely.

I concur.


Raymond
 

From lie.1296 at gmail.com  Mon Mar  9 07:41:07 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Mon, 09 Mar 2009 17:41:07 +1100
Subject: [Python-Dev] PEP 239 - Rational
Message-ID: <gp2dma$p1f$1@ger.gmane.org>

PEP 239 says that Rational type was Rejected, but some time ago this 
decision is reverted, and now python 3.0 and python 2.6 includes a 
fractions.Fraction type. Shouldn't this PEP be updated? (At least to 
include a note of its obsoleted status or to point to the reversion)


From martin at v.loewis.de  Mon Mar  9 08:04:50 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Mar 2009 08:04:50 +0100
Subject: [Python-Dev] IDLE maintenance
In-Reply-To: <p06240800c5da353385b2@[10.0.1.221]>
References: <mailman.5571.1236547101.11745.python-dev@python.org>
	<p06240800c5da353385b2@[10.0.1.221]>
Message-ID: <49B4BF92.40504@v.loewis.de>

> As I said, I don't know the plans or people surrounding IDLE are. If it
> needs a new maintainer I hereby volunteer. 

Can you please start by looking into the issues reported for the IDLE
component? (there are currently 71 of them)

For those with a patch in particular (17 currently), make a
recommendation whether to accept or reject the patch (perhaps giving
the submitter a chance to revise the patch if you have complaints).
Post a list of patches that you have triaged to this list.

For the bug reports, review whether the bug is reproducible, and
if so, propose a patch. If not, put an analysis into the report, and
post a list of patches that you propose to close here.

For feature requests, analyze whether the feature is desirable and
sound. If it is, propose a patch. If not, post a list of requests
to close here (and an analysis into the report).

Regards,
Martin


From martin at v.loewis.de  Mon Mar  9 08:06:44 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Mar 2009 08:06:44 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <6CD71A59-E79A-495E-BDB6-24A1CDAAD7D5@mac.com>
References: <mailman.5502.1236523607.11745.python-dev@python.org>
	<6CD71A59-E79A-495E-BDB6-24A1CDAAD7D5@mac.com>
Message-ID: <49B4C004.1000907@v.loewis.de>

> I'd like to suggest that any new candidate for the standard library be
> discussed and then set aside for a cooling off period of ONE YEAR.  If
> then folks can all agree the library is not only Goodness, but of
> general interest, especially for bootstrapping small projects, then take
> a vote, or the BDFL can decide.
> 
> A key criteria should be, "Will the new library help small projects get
> started by providing basic capabilities without introducing a steep
> learning curve?"

These are all thoughts that I can sympathize with.

Regards,
Martin

From python at rcn.com  Mon Mar  9 08:52:53 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 9 Mar 2009 00:52:53 -0700
Subject: [Python-Dev] RELEASED Python 3.1 alpha 1
References: <1afaf6160903071053i559f5608nb37882f680ed41d0@mail.gmail.com>
Message-ID: <4CBF8A04D1B447B2BE313F4D3BD29869@RaymondLaptop1>


> On behalf of the Python development team and the Python community, I'm
> happy to announce the first alpha release of Python 3.1.

Are there any plans for a Windows installer?


Raymond

From martin at v.loewis.de  Mon Mar  9 09:25:45 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Mar 2009 09:25:45 +0100
Subject: [Python-Dev] RELEASED Python 3.1 alpha 1
In-Reply-To: <4CBF8A04D1B447B2BE313F4D3BD29869@RaymondLaptop1>
References: <1afaf6160903071053i559f5608nb37882f680ed41d0@mail.gmail.com>
	<4CBF8A04D1B447B2BE313F4D3BD29869@RaymondLaptop1>
Message-ID: <49B4D289.7090606@v.loewis.de>

>> On behalf of the Python development team and the Python community, I'm
>> happy to announce the first alpha release of Python 3.1.
> 
> Are there any plans for a Windows installer?

Yes. However, I cannot produce them on weekends.

Regards,
Martin

From stefan_ml at behnel.de  Mon Mar  9 10:09:18 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 09 Mar 2009 10:09:18 +0100
Subject: [Python-Dev] RELEASED Python 3.1 alpha 1
In-Reply-To: <49B4D289.7090606@v.loewis.de>
References: <1afaf6160903071053i559f5608nb37882f680ed41d0@mail.gmail.com>	<4CBF8A04D1B447B2BE313F4D3BD29869@RaymondLaptop1>
	<49B4D289.7090606@v.loewis.de>
Message-ID: <gp2mbu$duq$1@ger.gmane.org>

Martin v. L?wis wrote:
>>> On behalf of the Python development team and the Python community, I'm
>>> happy to announce the first alpha release of Python 3.1.
>> Are there any plans for a Windows installer?
> 
> Yes. However, I cannot produce them on weekends.

Sounds like a bug in the MS installer. Dates are just sooo hard to handle
these days, especially since 7 was nominated prime.

Stefan ;o)


From barry at python.org  Mon Mar  9 13:15:46 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 9 Mar 2009 08:15:46 -0400
Subject: [Python-Dev] BZR mirror and pushing to Launchpad
In-Reply-To: <20090309052649.GF6920@steerpike.home.puzzling.org>
References: <Pine.LNX.4.64.0903082028560.2596@kimball.webabinitio.net>
	<4553f0903081828u4ee200far9d97a524cf150638@mail.gmail.com>
	<20090309024059.GD6920@steerpike.home.puzzling.org>
	<4553f0903082119h4678c91dg43eaee3c2814be4e@mail.gmail.com>
	<4553f0903082120i413f6bc7jc6be7f17cea956a@mail.gmail.com>
	<20090309052649.GF6920@steerpike.home.puzzling.org>
Message-ID: <540B9FE9-424F-4946-A450-E5416469C6D2@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 9, 2009, at 1:26 AM, Andrew Bennetts wrote:

> On the other hand, it does appear that
> <https://code.launchpad.net/~rdmurray/python/bug5450> is a branch  
> that is
> not stacking-capable.  I'm not sure how this could happen by  
> following the
> instructions on the the wiki page.

Perhaps RDM branched from code.python.org before I upgraded the  
repository format to 1.9-rich-root?

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSbUIcnEjvBPtnXfVAQI47AP/QdS/7qyrGXseNsDYLIl/aRC4DV3HOBxm
G4721SchQAw8CF6zJw1gU/Guw4b+S96eYD3MHBM2B6qgSXZ3SMXGoWvVl2xwKbFd
s29f9ALZZaXuj3YU5YY3ildCQmKx0Kxaqbp9Hu1UdaAAZrq25rI2vMbtdTphtG2j
i8vQZZifegY=
=1HK/
-----END PGP SIGNATURE-----

From rdmurray at bitdance.com  Mon Mar  9 14:02:27 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 9 Mar 2009 09:02:27 -0400 (EDT)
Subject: [Python-Dev] BZR mirror and pushing to Launchpad
In-Reply-To: <540B9FE9-424F-4946-A450-E5416469C6D2@python.org>
References: <Pine.LNX.4.64.0903082028560.2596@kimball.webabinitio.net>
	<4553f0903081828u4ee200far9d97a524cf150638@mail.gmail.com>
	<20090309024059.GD6920@steerpike.home.puzzling.org>
	<4553f0903082119h4678c91dg43eaee3c2814be4e@mail.gmail.com>
	<4553f0903082120i413f6bc7jc6be7f17cea956a@mail.gmail.com>
	<20090309052649.GF6920@steerpike.home.puzzling.org>
	<540B9FE9-424F-4946-A450-E5416469C6D2@python.org>
Message-ID: <Pine.LNX.4.64.0903090859120.2596@kimball.webabinitio.net>

On Mon, 9 Mar 2009 at 08:15, Barry Warsaw wrote:
> On Mar 9, 2009, at 1:26 AM, Andrew Bennetts wrote:
>> On the other hand, it does appear that
>> <https://code.launchpad.net/~rdmurray/python/bug5450> is a branch that is
>> not stacking-capable.  I'm not sure how this could happen by following the
>> instructions on the the wiki page.
>
> Perhaps RDM branched from code.python.org before I upgraded the repository 
> format to 1.9-rich-root?

I grabbed the tarball and made my initial branch on 2/20, from python.org.
Then I branched from that to make the branch I pushed to Launchpad.
Could the second branch be the problem?  I just did 'bzr branch trunk
trunk-myfix'.

Is there a way I can check if my branch is stacking capable? (I'm very
new to bzr.)

--
R. David Murray           http://www.bitdance.com

From barry at python.org  Mon Mar  9 14:30:41 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 9 Mar 2009 09:30:41 -0400
Subject: [Python-Dev] BZR mirror and pushing to Launchpad
In-Reply-To: <Pine.LNX.4.64.0903090859120.2596@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0903082028560.2596@kimball.webabinitio.net>
	<4553f0903081828u4ee200far9d97a524cf150638@mail.gmail.com>
	<20090309024059.GD6920@steerpike.home.puzzling.org>
	<4553f0903082119h4678c91dg43eaee3c2814be4e@mail.gmail.com>
	<4553f0903082120i413f6bc7jc6be7f17cea956a@mail.gmail.com>
	<20090309052649.GF6920@steerpike.home.puzzling.org>
	<540B9FE9-424F-4946-A450-E5416469C6D2@python.org>
	<Pine.LNX.4.64.0903090859120.2596@kimball.webabinitio.net>
Message-ID: <04A5EE8A-91F6-4FF1-ADDB-41E927F4FC2A@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 9, 2009, at 9:02 AM, R. David Murray wrote:

> On Mon, 9 Mar 2009 at 08:15, Barry Warsaw wrote:
>> On Mar 9, 2009, at 1:26 AM, Andrew Bennetts wrote:
>>> On the other hand, it does appear that
>>> <https://code.launchpad.net/~rdmurray/python/bug5450> is a branch  
>>> that is
>>> not stacking-capable.  I'm not sure how this could happen by  
>>> following the
>>> instructions on the the wiki page.
>>
>> Perhaps RDM branched from code.python.org before I upgraded the  
>> repository format to 1.9-rich-root?
>
> I grabbed the tarball and made my initial branch on 2/20, from  
> python.org.
> Then I branched from that to make the branch I pushed to Launchpad.
> Could the second branch be the problem?  I just did 'bzr branch trunk
> trunk-myfix'.

I upgraded the repository on 2/25, so it's possible your older tarball  
is the problem.  try running 'bzr upgrade --1.9.rich-root' on it and  
see if that allows you to stack on Launchpad.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSbUaAnEjvBPtnXfVAQLnggP/bF/P4N5WqwW4ozHOrbL1hJ9mc1nDdb9s
4z8V75qIiHJTYRQ3A9koTQJzEzCrsrqwhW04zs+hJikhHp1ZqjEVYdEvI4ibDikr
yigsbiI/XWHt1KaKBiRGXDal47NXyvy7VDkc5QMoPAyx7ed7lvbqslHIkJJqJR1N
6ZmnZ9AZnvw=
=+6oN
-----END PGP SIGNATURE-----

From rdmurray at bitdance.com  Mon Mar  9 16:23:43 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 9 Mar 2009 11:23:43 -0400 (EDT)
Subject: [Python-Dev] BZR mirror and pushing to Launchpad
In-Reply-To: <6E5DC51B-91CB-48F6-8809-E5D991157E22@python.org>
References: <Pine.LNX.4.64.0903082028560.2596@kimball.webabinitio.net>
	<4553f0903081828u4ee200far9d97a524cf150638@mail.gmail.com>
	<20090309024059.GD6920@steerpike.home.puzzling.org>
	<4553f0903082119h4678c91dg43eaee3c2814be4e@mail.gmail.com>
	<4553f0903082120i413f6bc7jc6be7f17cea956a@mail.gmail.com>
	<20090309052649.GF6920@steerpike.home.puzzling.org>
	<540B9FE9-424F-4946-A450-E5416469C6D2@python.org>
	<Pine.LNX.4.64.0903090859120.2596@kimball.webabinitio.net>
	<04A5EE8A-91F6-4FF1-ADDB-41E927F4FC2A@python.org>
	<Pine.LNX.4.64.0903091045180.2596@kimball.webabinitio.net>
	<6E5DC51B-91CB-48F6-8809-E5D991157E22@python.org>
Message-ID: <Pine.LNX.4.64.0903091120040.2596@kimball.webabinitio.net>

On Mon, 9 Mar 2009 at 10:49, Barry Warsaw wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> On Mar 9, 2009, at 10:48 AM, R. David Murray wrote:
>> On Mon, 9 Mar 2009 at 09:30, Barry Warsaw wrote:
>> > On Mar 9, 2009, at 9:02 AM, R. David Murray wrote:
>> > > On Mon, 9 Mar 2009 at 08:15, Barry Warsaw wrote:
>> > > >  Perhaps RDM branched from code.python.org before I upgraded the > 
>> > > >  repository format to 1.9-rich-root?
>> > > I grabbed the tarball and made my initial branch on 2/20, from 
>> > > python.org.
>> > > Then I branched from that to make the branch I pushed to Launchpad.
>> > > Could the second branch be the problem?  I just did 'bzr branch trunk
>> > > trunk-myfix'.
>> > 
>> > I upgraded the repository on 2/25, so it's possible your older tarball is 
>> > the problem.  try running 'bzr upgrade --1.9.rich-root' on it and see if 
>> > that allows you to stack on Launchpad.
>> 
>> Hmm.  My client is 1.12, and it says:
>>
>>     rdmurray at maestro:~/src/python/bzr>bzr upgrade --1.9.rich-root
>>     bzr: ERROR: no such option: --1.9.rich-root
>> 
>> Do I need to downgrade to 1.9?
>
> No, sorry, that was a typo on my part.
>
> Try --1.9-rich-root
>
> Also, 'bzr help upgrade'

Ach.  I looked at an example of running the command I found on the web
and didn't see the typo.  I'd probably have spotted it if I'd checked
the help :(.

Upgrade running now, I'll report back after I try the push again.

--
R. David Murray           http://www.bitdance.com

From rdmurray at bitdance.com  Mon Mar  9 16:47:37 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 9 Mar 2009 11:47:37 -0400 (EDT)
Subject: [Python-Dev] BZR mirror and pushing to Launchpad
In-Reply-To: <Pine.LNX.4.64.0903091120040.2596@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0903082028560.2596@kimball.webabinitio.net>
	<4553f0903081828u4ee200far9d97a524cf150638@mail.gmail.com>
	<20090309024059.GD6920@steerpike.home.puzzling.org>
	<4553f0903082119h4678c91dg43eaee3c2814be4e@mail.gmail.com>
	<4553f0903082120i413f6bc7jc6be7f17cea956a@mail.gmail.com>
	<20090309052649.GF6920@steerpike.home.puzzling.org>
	<540B9FE9-424F-4946-A450-E5416469C6D2@python.org>
	<Pine.LNX.4.64.0903090859120.2596@kimball.webabinitio.net>
	<04A5EE8A-91F6-4FF1-ADDB-41E927F4FC2A@python.org>
	<Pine.LNX.4.64.0903091045180.2596@kimball.webabinitio.net>
	<6E5DC51B-91CB-48F6-8809-E5D991157E22@python.org>
	<Pine.LNX.4.64.0903091120040.2596@kimball.webabinitio.net>
Message-ID: <Pine.LNX.4.64.0903091135160.2596@kimball.webabinitio.net>

On Mon, 9 Mar 2009 at 11:23, R. David Murray wrote:
> On Mon, 9 Mar 2009 at 10:49, Barry Warsaw wrote:
>>  -----BEGIN PGP SIGNED MESSAGE-----
>>  On Mar 9, 2009, at 10:48 AM, R. David Murray wrote:
>> >  On Mon, 9 Mar 2009 at 09:30, Barry Warsaw wrote:
>> > >  On Mar 9, 2009, at 9:02 AM, R. David Murray wrote:
>> > > >  On Mon, 9 Mar 2009 at 08:15, Barry Warsaw wrote:
>> > > > >   Perhaps RDM branched from code.python.org before I upgraded the
>> > > > >   repository format to 1.9-rich-root?
>> > > >  I grabbed the tarball and made my initial branch on 2/20, from
>> > > >  python.org.  Then I branched from that to make the branch I
>> > > >  pushed to Launchpad.  Could the second branch be the problem?
>> > > >  I just did 'bzr branch trunk trunk-myfix'.
>> > > 
>> > >  I upgraded the repository on 2/25, so it's possible your older tarball 
>> > >  is the problem.  try running 'bzr upgrade --1.9.rich-root' on it and 
>> > >  see if that allows you to stack on Launchpad.
>> > 
>> >  Hmm.  My client is 1.12, and it says:
>> > 
>> >      rdmurray at maestro:~/src/python/bzr>bzr upgrade --1.9.rich-root
>> >      bzr: ERROR: no such option: --1.9.rich-root
>> > 
>> >  Do I need to downgrade to 1.9?
>>
>>  No, sorry, that was a typo on my part.
>>
>>  Try --1.9-rich-root
>>
>>  Also, 'bzr help upgrade'
>
> Ach.  I looked at an example of running the command I found on the web
> and didn't see the typo.  I'd probably have spotted it if I'd checked
> the help :(.
>
> Upgrade running now, I'll report back after I try the push again.

OK, that fixed it.  New push is properly stacked.

Thanks everyone, especially Barry.

--
R. David Murray           http://www.bitdance.com

From guido at python.org  Mon Mar  9 18:13:46 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 9 Mar 2009 10:13:46 -0700
Subject: [Python-Dev] PEP 239 - Rational
In-Reply-To: <gp2dma$p1f$1@ger.gmane.org>
References: <gp2dma$p1f$1@ger.gmane.org>
Message-ID: <ca471dc20903091013i781c68edia62d232ec8aad9a9@mail.gmail.com>

On Sun, Mar 8, 2009 at 11:41 PM, Lie Ryan <lie.1296 at gmail.com> wrote:
> PEP 239 says that Rational type was Rejected, but some time ago this
> decision is reverted, and now python 3.0 and python 2.6 includes a
> fractions.Fraction type. Shouldn't this PEP be updated? (At least to include
> a note of its obsoleted status or to point to the reversion)

Good idea. I've added a reference to PEP 3141 mentioning the Rational
ABC and the fractions module.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fabiofz at gmail.com  Mon Mar  9 20:06:02 2009
From: fabiofz at gmail.com (Fabio Zadrozny)
Date: Mon, 9 Mar 2009 16:06:02 -0300
Subject: [Python-Dev] Python 3.0 grammar ambiguous?
In-Reply-To: <cfb578b20903081558l4ef9f89dw8ecf4903269e2753@mail.gmail.com>
References: <cfb578b20903081307l738c07d5g2888f0d13f9e6309@mail.gmail.com>
	<49B42892.20903@v.loewis.de>
	<cfb578b20903081338i19680dc1v36206bb9d6bc1412@mail.gmail.com>
	<49B43610.2040107@gmail.com>
	<ca471dc20903081523t5cb7abf0tc943843c1060e49f@mail.gmail.com>
	<cfb578b20903081558l4ef9f89dw8ecf4903269e2753@mail.gmail.com>
Message-ID: <cfb578b20903091206g422af48eoc47ececb4546e24@mail.gmail.com>

>> To be honest I wasn't aware of this ambiguity. It seems that whoever
>> wrote the patch for argument unpacking (a, b, *c = ...) got "lucky"
>> with an ambiguous grammar. This surprises me, because IIRC usually
>> pgen doesn't like ambiguities. Other parser generators usually have
>> some way to deal with ambiguous grammars, but they also usually have
>> features that make it unnecessary to use the exact same grammar as
>> pgen -- for example, most parser generators are able to backtrack or
>> look ahead more than one token, so that they can distinguish between
>> "a = b" and "a" once the '=' token is seen, rather than having to
>> commit to parse an expression first.
>
> JavaCC can actually do that, but in the current construct, the
> ambiguity is not dependent on a lookahead, because both '*' test and
> star_expr will match it equally well -- because they're actually the
> same thing grammar-wise (so, there's no way to disambiguate without a
> semantic handling later on)
>
> After taking a 2nd look, I think that probably the best solution would
> be creating a new testlist to be used from the expr_stmt -- something
> like testlist_start_expr.
>
> E.g.:
>
> testlist: test (',' test)* [',']
> testlist_star_expr: ?[test | star_expr] (',' [test | star_expr])* [',']
>
> And also, star_expr could probably be just
> '*' NAME
> (to make it faster to match -- or could it match something else?)
>
> Note that I still haven't tested this -- I'll have time to do it
> tomorrow on my JavaCC grammar (so, I can give you a better feedback if
> this actually works).
>

Ok, I've created a bug for that at: http://bugs.python.org/issue5460
and commented on a solution (and just to note, star_expr could match
b, *a.a = [1,2,3], so, it cannot be changed for NAME)

The solution is working for me, and it should be straightforward to
apply it to Python.

Regards,

Fabio

From tjreedy at udel.edu  Mon Mar  9 20:22:38 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 09 Mar 2009 15:22:38 -0400
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
Message-ID: <gp3q9m$glr$1@ger.gmane.org>

Benjamin Peterson wrote:
>
>> You might also want to collect a list of serious changes that you want
>> in this release; I know I/O in C is on the list (and without it I
>> wouldn't consider it worth releasing) but there may be others. The
>> developers of such features ought to be on board with delivering their
>> code before the first beta.
> 
> I've started a list on the release PEP [1].
> 
> [1] http://www.python.org/dev/peps/pep-0375/

Please add "auto-numbered replacement fields in str.format() strings"
<http://bugs.python.org/issue5237>

Guido wrote today "Please go ahead and finish this. I'm glad this is 
going in!"

Eric Smith says he should have a final patch ready by the end of PyCon 
or so.


From phd at phd.pp.ru  Mon Mar  9 21:08:44 2009
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Mon, 9 Mar 2009 23:08:44 +0300
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <687FDB81-61F5-419B-A7F5-3F5EC8CDE68E@python.org>
References: <49ABCFC9.1070508@gmail.com> <49AE9698.1020806@simplistix.co.uk>
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
	<gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<687FDB81-61F5-419B-A7F5-3F5EC8CDE68E@python.org>
Message-ID: <20090309200844.GB9075@phd.pp.ru>

On Thu, Mar 05, 2009 at 01:30:25PM -0500, Barry Warsaw wrote:
> Ubuntu (and probably Debian): apt-get install python-lxml

   Tested in Debian: yes, the incantation works.

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From benjamin at python.org  Mon Mar  9 21:21:16 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 9 Mar 2009 15:21:16 -0500
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <gp3q9m$glr$1@ger.gmane.org>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<gp3q9m$glr$1@ger.gmane.org>
Message-ID: <1afaf6160903091321gc76ce14r4f925440b697d116@mail.gmail.com>

2009/3/9 Terry Reedy <tjreedy at udel.edu>:
> Benjamin Peterson wrote:
>>
>>> You might also want to collect a list of serious changes that you want
>>> in this release; I know I/O in C is on the list (and without it I
>>> wouldn't consider it worth releasing) but there may be others. The
>>> developers of such features ought to be on board with delivering their
>>> code before the first beta.
>>
>> I've started a list on the release PEP [1].
>>
>> [1] http://www.python.org/dev/peps/pep-0375/
>
> Please add "auto-numbered replacement fields in str.format() strings"
> <http://bugs.python.org/issue5237>

Done.



-- 
Regards,
Benjamin

From facundobatista at gmail.com  Mon Mar  9 22:08:48 2009
From: facundobatista at gmail.com (Facundo Batista)
Date: Mon, 9 Mar 2009 19:08:48 -0200
Subject: [Python-Dev] Regexp 2.7
In-Reply-To: <loom.20090307T101047-371@post.gmane.org>
References: <loom.20090307T101047-371@post.gmane.org>
Message-ID: <e04bdf310903091408g5db702b5l922a3907d68bb563@mail.gmail.com>

2009/3/7 Antoine Pitrou <solipsis at pitrou.net>:

> Matthew Barnett has been doing a lot of work on the regular expressions engine
> (it seems he hasn't finished yet) under http://bugs.python.org/issue2636.
> However, the patches are really huge and touch all of the sre internals. I
> wonder what the review process can be for such patches? Is there someone
> knowledgeable enough to be able to review them?

All test cases run ok? How well covered is that library?

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From solipsis at pitrou.net  Tue Mar 10 00:07:12 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 9 Mar 2009 23:07:12 +0000 (UTC)
Subject: [Python-Dev] Regexp 2.7
References: <loom.20090307T101047-371@post.gmane.org>
	<e04bdf310903091408g5db702b5l922a3907d68bb563@mail.gmail.com>
Message-ID: <loom.20090309T230621-950@post.gmane.org>

Facundo Batista <facundobatista <at> gmail.com> writes:
> 
> > Matthew Barnett has been doing a lot of work on the regular expressions
engine
> > (it seems he hasn't finished yet) under http://bugs.python.org/issue2636.
> > However, the patches are really huge and touch all of the sre internals. I
> > wonder what the review process can be for such patches? Is there someone
> > knowledgeable enough to be able to review them?
> 
> All test cases run ok? How well covered is that library?

I don't know, I haven't even tried it.

Regards

Antoine.



From tleeuwenburg at gmail.com  Tue Mar 10 01:01:28 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Tue, 10 Mar 2009 11:01:28 +1100
Subject: [Python-Dev] reviewing patches
Message-ID: <43c8685c0903091701m6413ded5qa15f1ba4d3c8a0bb@mail.gmail.com>

Hi all,

I have seen it said that one very useful activity is reviewing patches. Of
the issues in the tracker, it is not immediately clear to me what is
required of such a review. Many of these patches appear to be bundled in
with feature requests, leaving the question of whether the review it judging
the quality of the code or the merits of the feature request. I do realise
that these issues have probably been previously discussed on this list.
Let's take as an example the following issue:

  http://bugs.python.org/issue1818

This has obviously been around for a while, but not been implemented for
some reason. There are no links in any of the comments to any email threads
regarding its merits, however I recognise the name of the submitter. This
makes me think the patch is probably implementing desirable functionality.
However, it has no priority set, which makes me think that the community
hasn't yet given it any kind of 'status', insofar as a priority can stand in
for desirability.

It seems to make sense that code quality reviews should be separated from
feature request reviews (quality code evaluation vs desirability of function
evaluation) but I don't see how this occurs. I feel a lot more qualified to
evaluate code quality than desirability of function.

Questions like this make it difficult for someone in my position, who is
happy to tackle 'whatever needs to be done', to begin the task of patch
reviews. While I'm not sure that a formal or semi-formal approval process
would make anything better, I think it would be good if there were some kind
of 'executive review' process by which an issue could be marked as being a
good thing or not.

Regards,
-Tennessee

-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090310/c42407f1/attachment-0001.htm>

From skip at pobox.com  Tue Mar 10 01:03:06 2009
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 9 Mar 2009 19:03:06 -0500
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <gp3q9m$glr$1@ger.gmane.org>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<gp3q9m$glr$1@ger.gmane.org>
Message-ID: <18869.44602.432059.2350@montanaro.dyndns.org>


    >>> You might also want to collect a list of serious changes that you
    >>> want in this release;

http://bugs.python.org/issue4847

Not yet fixed.  Needs:

    * Decision about the correct fix (I think it will involve an API
      change).

    * Test case and a patch.

    * Probably small documentation changes as well.  

I'm wiped out this evening.  I'll try to look into it, but I suspect it
might require a bit more time than I have in the next day or two.

Skip

From python at rcn.com  Tue Mar 10 01:21:23 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 9 Mar 2009 17:21:23 -0700
Subject: [Python-Dev] draft 3.1 release schedule
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com><ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com><1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com><gp3q9m$glr$1@ger.gmane.org>
	<18869.44602.432059.2350@montanaro.dyndns.org>
Message-ID: <21E2F630429946A9A80448A23E1FAA8D@RaymondLaptop1>

>    >>> You might also want to collect a list of serious changes that you
>    >>> want in this release;

Bob Ippolito has a good sized patch to update the json module
and improve its performance.

http://bugs.python.org/issue4136



Raymond

From benjamin at python.org  Tue Mar 10 01:25:46 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 9 Mar 2009 19:25:46 -0500
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <21E2F630429946A9A80448A23E1FAA8D@RaymondLaptop1>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<gp3q9m$glr$1@ger.gmane.org>
	<18869.44602.432059.2350@montanaro.dyndns.org>
	<21E2F630429946A9A80448A23E1FAA8D@RaymondLaptop1>
Message-ID: <1afaf6160903091725n4cf47084t3177e8bd8f33f834@mail.gmail.com>

2009/3/9 Raymond Hettinger <python at rcn.com>:
>> ? >>> You might also want to collect a list of serious changes that you
>> ? >>> want in this release;
>
> Bob Ippolito has a good sized patch to update the json module
> and improve its performance.
>
> http://bugs.python.org/issue4136

...and it's already on the PEP. :)


-- 
Regards,
Benjamin

From benjamin at python.org  Tue Mar 10 01:28:57 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 9 Mar 2009 19:28:57 -0500
Subject: [Python-Dev] draft 3.1 release schedule
In-Reply-To: <18869.44602.432059.2350@montanaro.dyndns.org>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<gp3q9m$glr$1@ger.gmane.org>
	<18869.44602.432059.2350@montanaro.dyndns.org>
Message-ID: <1afaf6160903091728i456b3a77x1328c8a912594bb5@mail.gmail.com>

2009/3/9  <skip at pobox.com>:
>
> ? ?>>> You might also want to collect a list of serious changes that you
> ? ?>>> want in this release;
>
> http://bugs.python.org/issue4847
>
> Not yet fixed. ?Needs:
>
> ? ?* Decision about the correct fix (I think it will involve an API
> ? ? ?change).
>
> ? ?* Test case and a patch.
>
> ? ?* Probably small documentation changes as well.
>
> I'm wiped out this evening. ?I'll try to look into it, but I suspect it
> might require a bit more time than I have in the next day or two.

That seems important, but quite "serious" enough to warrant inclusion
in the PEP. (and not a feature) If you'd like it to block the release,
you can set the priority to "release blocker."



-- 
Regards,
Benjamin

From martin at v.loewis.de  Tue Mar 10 01:40:34 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 10 Mar 2009 01:40:34 +0100
Subject: [Python-Dev] reviewing patches
In-Reply-To: <43c8685c0903091701m6413ded5qa15f1ba4d3c8a0bb@mail.gmail.com>
References: <43c8685c0903091701m6413ded5qa15f1ba4d3c8a0bb@mail.gmail.com>
Message-ID: <49B5B702.5050709@v.loewis.de>

> I have seen it said that one very useful activity is reviewing patches.
> Of the issues in the tracker, it is not immediately clear to me what is
> required of such a review. Many of these patches appear to be bundled in
> with feature requests, leaving the question of whether the review it
> judging the quality of the code or the merits of the feature request.

That's correct.

> I
> do realise that these issues have probably been previously discussed on
> this list. Let's take as an example the following issue:
> 
>   http://bugs.python.org/issue1818

This is somewhat of a special case, as the original submission is from
a committer (Raymond). If he would feel like it, he could commit it
at any time. That he didn't is probably because he is uncertain himself.

> This has obviously been around for a while, but not been implemented for
> some reason. There are no links in any of the comments to any email
> threads regarding its merits

This likely means there hasn't been any email communication. This is
common; often discussion is carried out entirely in the tracker (and
if you aren't on the nosy list of the issue, you will miss the
discussion completely)

> This makes me think the patch is probably implementing
> desirable functionality. However, it has no priority set, which makes me
> think that the community hasn't yet given it any kind of 'status',
> insofar as a priority can stand in for desirability.

No. It means that the classification fields are often completely
ignored, both by submitters and reviewers.

> It seems to make sense that code quality reviews should be separated
> from feature request reviews (quality code evaluation vs desirability of
> function evaluation) but I don't see how this occurs.

Most likely, the functionality itself is uncontested. This is typically
because
a) some reviewers agreed that the functionality is desirable, and
b) some reviewers didn't consider the possibility of questioning the
   functionality, and
c) some possible opponents are unaware of the discussion in the first
   place.

> I feel a lot more
> qualified to evaluate code quality than desirability of function.

I think it is fairly straight-forward to ask that you can access the
fields of a CSV list by field name, rather than by index; I would take
that as granted.

Notice that much of the discussion is about fine points of the API,
which is an indication that the actual design of the functionality is
tricky, not just the implementation. It would be helpful to be a heavy
CSV user to understand all aspects, and to evaluate usefulness of a
specific API.

> Questions like this make it difficult for someone in my position, who is
> happy to tackle 'whatever needs to be done', to begin the task of patch
> reviews.

I recommend that you look at patches with no long discussions - things
that did not get any attention at all so far. Issues that have already
long discussions typically are inherently difficult, and its not always
clear that another reviewer will be able to progress the issue - unless
he happens to be an export on the matter, such as Alexander for knots
on ox carts.

> While I'm not sure that a formal or semi-formal approval
> process would make anything better, I think it would be good if there
> were some kind of 'executive review' process by which an issue could be
> marked as being a good thing or not.

If you think creating more keywords would help, please feel free to
propose some. If it is more status values, or more whatever - likewise.
All of these are cheap to create.

Regards,
Martin

From python at rcn.com  Tue Mar 10 02:05:25 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 9 Mar 2009 18:05:25 -0700
Subject: [Python-Dev] draft 3.1 release schedule
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<gp3q9m$glr$1@ger.gmane.org>
	<18869.44602.432059.2350@montanaro.dyndns.org>
	<21E2F630429946A9A80448A23E1FAA8D@RaymondLaptop1>
	<1afaf6160903091725n4cf47084t3177e8bd8f33f834@mail.gmail.com>
Message-ID: <A9C5B7EB749841AE80F09D9CD12814A0@RaymondLaptop1>

>>> You might also want to collect a list of serious changes that you
>>> want in this release;

I'm making minor updates to the decimal module to match the 1.68 version of the spec.


Raymond

From python at rcn.com  Tue Mar 10 02:39:48 2009
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 9 Mar 2009 18:39:48 -0700
Subject: [Python-Dev] draft 3.1 release schedule
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com><ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com><1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com><gp3q9m$glr$1@ger.gmane.org><18869.44602.432059.2350@montanaro.dyndns.org><21E2F630429946A9A80448A23E1FAA8D@RaymondLaptop1><1afaf6160903091725n4cf47084t3177e8bd8f33f834@mail.gmail.com>
	<A9C5B7EB749841AE80F09D9CD12814A0@RaymondLaptop1>
Message-ID: <D5486FC0172A4E6BA6C124CCF8E946C5@RaymondLaptop1>

 
> I'm making minor updates to the decimal module to match the 1.68 version of the spec.

Looks like most was already done.  Just needs some doc fixes.


Raymond

From dan.mahn at digidescorp.com  Tue Mar 10 04:18:22 2009
From: dan.mahn at digidescorp.com (Dan Mahn)
Date: Mon, 09 Mar 2009 22:18:22 -0500
Subject: [Python-Dev] More on Py3K urllib -- urlencode()
In-Reply-To: <98678.1236450982@parc.com>
References: <49A99E7B.7030602@digidescorp.com> <34291.1235858913@parc.com>
	<35441.1235863997@parc.com> <49B28B20.7000509@digidescorp.com>
	<20090307152424.GA9464@panix.com> <98678.1236450982@parc.com>
Message-ID: <49B5DBFE.3060505@digidescorp.com>

Yes, that was a good idea.  I found some problems, and attached a new 
version.  It looks more complicated than I wanted, but it is a very 
regular repetition, so I hope it is generally readable.

I used "doctest" to include the test scenarios.  I was not familiar with 
it before, but it seems to work quite well.  The main snag I hit was 
that I had to jazz around with the escape sequences (backslashes) in 
order to get the doc string to go in properly.  That is, the lines in 
the string are not the lines I typed at the command prompt, as Python is 
interpreting the escapes in the strings when the file is imported.

In an effort to make fewer tests, the lines of the test strings grew 
pretty long.  I'm not sure if I should try to cut the lengths down or not.

Any suggestions would be welcome before I try to submit this as a patch.

- Dan


Bill Janssen wrote:
> Aahz <aahz at pythoncraft.com> wrote:
>
>   
>> On Sat, Mar 07, 2009, Dan Mahn wrote:
>>     
>>> After a harder look, I concluded there was a bit more work to be done,  
>>> but still very basic modifications.
>>>
>>> Attached is a version of urlencode() which seems to make the most sense  
>>> to me.
>>>
>>> I wonder how I could officially propose at least some of these  
>>> modifications.
>>>       
>> Submit a patch to bugs.python.org
>>     
>
> And it would help if it included a lot of test cases.
>
> Bill
>   
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: new_urlencode.py
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090309/7e746b0b/attachment.txt>

From tleeuwenburg at gmail.com  Tue Mar 10 04:25:30 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Tue, 10 Mar 2009 14:25:30 +1100
Subject: [Python-Dev] Addition of further status options to tracker
Message-ID: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>

Hi all,

I am beginning reviewing some more issues in the tracker. I think it would
be useful to have the following status options (new status options marked
with a '+'):
  Open: Means that the issue has been created and not further reviewed
  + Request Approved: Means that the issue is marked as worth further
development by the community
  + Specification Approved: Means that the functionality to be developed is
written down in some form, and agreed at least in general terms (preferably
in fairly specific terms)
  + Under Development: Means that an implementation is currently under
development
  Pending Feedback: Means that work is suspended pending feedback
  + Under Final Review: Means that a patch has been submitted and may be a
flag that core developers can usefully review the work done without having
to revisit the whole process
  Closed: Means that the issue is either suspended indefinitely or has been
resolved (see resolution value)

My reasoning for this is as follows:
  Example one: I am currently reviewing issue 2706 which covers
datetime.timedelta operators. I have a reasonable amount of familiarity with
time programming and think I have some valuable input on the functionality
aspects, however I'm not a super-great C programmer. I'd like to help with
coming up with the final feature specification, but then leave it to others
to consider the merits of the C code patch. Being able to help get this
marked as "request approved", then "specification approved" might help speed
up a lot of the process of developing the patch and getting it accepted. It
also saves code developers from having to develop an implementation while a
functionality debate is still ongoing...

  I think that having these additional steps will help to avoid erroneous
development of non-features, and also break up the review process into more
manageable chunks of work. Of course I realise not everything needs such a
delineated process, but by the same token it might assist in keeping some of
the less visible/popular changes moving through the process...

Comments welcome! Crocker's rules, but please keep it constructive...

-T


-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090310/4a9f196d/attachment.htm>

From brett at python.org  Tue Mar 10 04:39:51 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 9 Mar 2009 20:39:51 -0700
Subject: [Python-Dev] Addition of further status options to tracker
In-Reply-To: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>
References: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>
Message-ID: <bbaeab100903092039m7dcd73a4j93bb9f3255bf963e@mail.gmail.com>

On Mon, Mar 9, 2009 at 20:25, Tennessee Leeuwenburg
<tleeuwenburg at gmail.com>wrote:

> Hi all,
>
> I am beginning reviewing some more issues in the tracker. I think it would
> be useful to have the following status options (new status options marked
> with a '+'):
>   Open: Means that the issue has been created and not further reviewed
>   + Request Approved: Means that the issue is marked as worth further
> development by the community
>   + Specification Approved: Means that the functionality to be developed is
> written down in some form, and agreed at least in general terms (preferably
> in fairly specific terms)
>   + Under Development: Means that an implementation is currently under
> development
>   Pending Feedback: Means that work is suspended pending feedback
>   + Under Final Review: Means that a patch has been submitted and may be a
> flag that core developers can usefully review the work done without having
> to revisit the whole process
>   Closed: Means that the issue is either suspended indefinitely or has been
> resolved (see resolution value)


 I assume you want all of this for the Status field, correct?

As for the options, some of these overlap with the Stage field. For
instance, if something has been set to any stage other than
"accepted/rejected" it means it needs to be looked into, so that duplicates
your "Request Approved" status. Similar thing with the review stages and
"Under Final Review".

But a general "under development" status would probably be worth adding.
That way if an issue is "open" it needs attention, "Under development" means
someone is working on a solution, "pending" means someone is blocking the
issue for more information, and "closed" means closed.

One thing to watch out for, Tennessee, is getting too specific. Like your
"Request Approved" and "Specification Approved" just seems too heavy-handed
to my tastes. Personally, I prefer to make sure the issue workflow to be
somewhat simple so that people don't end up arguing over stuff like whether
something has been specified well enough to have it set to "Spec Approved"
even if someone else disagrees (which you know would happen).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090309/f9cceccf/attachment-0001.htm>

From tjreedy at udel.edu  Tue Mar 10 04:55:57 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 09 Mar 2009 23:55:57 -0400
Subject: [Python-Dev] reviewing patches
In-Reply-To: <49B5B702.5050709@v.loewis.de>
References: <43c8685c0903091701m6413ded5qa15f1ba4d3c8a0bb@mail.gmail.com>
	<49B5B702.5050709@v.loewis.de>
Message-ID: <gp4oce$461$1@ger.gmane.org>

Martin v. L?wis wrote:
>> I have seen it said that one very useful activity is reviewing patches.
>> Of the issues in the tracker, it is not immediately clear to me what is
>> required of such a review. Many of these patches appear to be bundled in
>> with feature requests, leaving the question of whether the review it
>> judging the quality of the code or the merits of the feature request.

Both may be needed, depending on the issue.  But see below.
> 
> That's correct.
> 
>> I
>> do realise that these issues have probably been previously discussed on
>> this list. Let's take as an example the following issue:
>>
>>   http://bugs.python.org/issue1818

Briefly, as I read it: Raymond proposed that his named tuples be used 
with Barry and Skip's csv module.  He got a short reply from Skip but 
went on to other things.  A year later, two new people entered and 
energized the discussion, Barry said 'useful', Skip commented on the 
details.  The second half of the discussion has been thrashing out and 
getting consesus on the devilish details.  What is devilish is that the 
merits of feature details sometimes interacts with code quality.  This 
is pretty typical.  It looks to me like this will be in 3.1.

 > often discussion is carried out entirely in the tracker (and
 > if you aren't on the nosy list of the issue, you will miss the
 > discussion completely)

Every Friday a list of opened and closed issues is posted.  Even if you 
have nothing to say, you can add yourself to the nosy list.

 >> I feel a lot more
 >> qualified to evaluate code quality than desirability of function.

Good, in the sense that I believe the former is rarer and more needed.

There are 5 broad areas covered in the tracker:
1. C code, interpreter core
2. C code, extension modules
3. Python code, modules
4. Tests of the above
5. Documentation of above

Where to start? What interests you most?

Or look at the thread "draft 3.1 release schedule"
and posts giving features aimed at 3.1.  For instance,
http://bugs.python.org/issue5237
has a new C patch that could use at least a preliminary eyeball, pending 
the more polished patch.  Help adding the test patches would be welcome 
by me and, I am sure, by Eric.

Terry Jan Reedy



From tleeuwenburg at gmail.com  Tue Mar 10 05:00:47 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Tue, 10 Mar 2009 15:00:47 +1100
Subject: [Python-Dev] Addition of further status options to tracker
In-Reply-To: <bbaeab100903092039m7dcd73a4j93bb9f3255bf963e@mail.gmail.com>
References: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>
	<bbaeab100903092039m7dcd73a4j93bb9f3255bf963e@mail.gmail.com>
Message-ID: <43c8685c0903092100i3457c5b0wbdaae31272a47895@mail.gmail.com>

On Tue, Mar 10, 2009 at 2:39 PM, Brett Cannon <brett at python.org> wrote:

>
>
> On Mon, Mar 9, 2009 at 20:25, Tennessee Leeuwenburg <
> tleeuwenburg at gmail.com> wrote:
>
>> Hi all,
>>
>> I am beginning reviewing some more issues in the tracker. I think it would
>> be useful to have the following status options (new status options marked
>> with a '+'):
>>   Open: Means that the issue has been created and not further reviewed
>>   + Request Approved: Means that the issue is marked as worth further
>> development by the community
>>   + Specification Approved: Means that the functionality to be developed
>> is written down in some form, and agreed at least in general terms
>> (preferably in fairly specific terms)
>>   + Under Development: Means that an implementation is currently under
>> development
>>   Pending Feedback: Means that work is suspended pending feedback
>>   + Under Final Review: Means that a patch has been submitted and may be a
>> flag that core developers can usefully review the work done without having
>> to revisit the whole process
>>   Closed: Means that the issue is either suspended indefinitely or has
>> been resolved (see resolution value)
>
>
>  I assume you want all of this for the Status field, correct?
>
> As for the options, some of these overlap with the Stage field. For
> instance, if something has been set to any stage other than
> "accepted/rejected" it means it needs to be looked into, so that duplicates
> your "Request Approved" status. Similar thing with the review stages and
> "Under Final Review".
>
> But a general "under development" status would probably be worth adding.
> That way if an issue is "open" it needs attention, "Under development" means
> someone is working on a solution, "pending" means someone is blocking the
> issue for more information, and "closed" means closed.
>
> One thing to watch out for, Tennessee, is getting too specific. Like your
> "Request Approved" and "Specification Approved" just seems too heavy-handed
> to my tastes. Personally, I prefer to make sure the issue workflow to be
> somewhat simple so that people don't end up arguing over stuff like whether
> something has been specified well enough to have it set to "Spec Approved"
> even if someone else disagrees (which you know would happen).
>
> -Brett
>

Hi Brett,

Some great points. The "stage" settings do overlap a lot of what I had
suggested. What I'm trying to reach is a way to flag quite clearly whether a
tracker issue is:
  * In need of a first response
  * Being taken care of by someone
  * Needs the attention of core developers, just about ready

At the moment, the open/closed settings are a bit useless -- issues are
"Open" until they are finished, at which point they are "Closed". I
personally suspect that "Pending feedback" is little-used. At the moment,
many "Open" issues are in various states of maturity or disrepair, so the
only really useful information is whether an issue is "Closed".

Examining the stage options more closely, they are very code-oriented.
Really they relate to implementation, not other parts of the process. It's
difficult for me as someone who wants to help out with the Issue resolution
process as much as coding to take an atomic action to help advance the issue
through a process.

What do you think about the following "Status" flags:

  * Open: Means newly-created, needs early attention
  * + Request for Comment: Means that a discussion regarding functionality
is requested
  * + Under Development: Means that a caring developer is taking care of it
  * Pending Feedback: blocking on new information
  * Closed: Means closed

Once the issue is marked as under development, more information is then
provided by the "Stage" flag.

"Request for Comment" means that anyone who is stuck on trying to get their
ideas sorted or a loose specification agreed on can call for feedback
through this flag. There's still some risk of arguments blocking the
progression to "Under development", but the issue owner can at least pass to
"Under development" at their own will without requiring universal agreement.

Core developers can then search on 'commit review' or 'patch review' to see
what could use their attention, while people who are happy to contribute to
the debate can search on "Open" and "Request for Comment" issues. Obviously
the RFC flag is redundant when an issue is first raised, since the submitter
always has the option of simply emailing this list. However, for older,
potentially stale issues, it is a useful indicator that the issue may be
either closed out or could use refreshing.

I don't mind what approach is taken -- I'm happy to work within the current
infrastructure if someone can suggest a good way. I really just want to be
able to start distinguishing between issues that are essentially new and
under debate versus issues that most people agree are a "Good Thing To Do",
and then helping those issues advance to the point of implementation by
discussing and, if necessary, formalising or recording requirements against
them.

Regards,
-Tennessee



-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090310/5df1ef57/attachment.htm>

From ajaksu at gmail.com  Tue Mar 10 05:05:58 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 10 Mar 2009 01:05:58 -0300
Subject: [Python-Dev] Addition of further status options to tracker
In-Reply-To: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>
References: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>
Message-ID: <2d75d7660903092105n17bc5a5fke2c373be0e00705b@mail.gmail.com>

Hi Tennessee,
I plan to take a look at all open issues before PyCon, do you want to
join forces? :)

Tennessee Leeuwenburg wrote:
> I am beginning reviewing some more issues in the tracker. I think it would
> be useful to have the following status options (new status options marked
> with a '+'):

Great! Thanks a lot :)

I think your ideas are good, but believe you should take a better look
at the Stages, some messages from this list and tracker-discuss from
February, and Brett's document linked from there.

> ? + Request Approved: Means that the issue is marked as worth further
> development by the community

Not sure it's a 'status', but I like the idea of disambiguating
between 'send us tests/patches so we can think whether the bug/feature
is valid' and 'the bug/feature is valid, send us tests/patches'.
However, it might be better to merge this with 'Specification
Approved', as the important bit is 'contributions on this are
welcome'.

> ? + Specification Approved: Means that the functionality to be developed is
> written down in some form, and agreed at least in general terms (preferably
> in fairly specific terms)

See above. +1 on a way to make this clear, -0 on it being separate
from the above, -1 on being a status.

> ? + Under Development: Means that an implementation is currently under
> development

This one I like a lot, but think it should be a keyword, like
'claimed'. I think it should be set when someone says "I'll work on
this one". But if you mean 'should be under development', -1 :)

> ? + Under Final Review: Means that a patch has been submitted and may be a
> flag that core developers can usefully review the work done without having
> to revisit the whole process

-1, as it is rather redundant with Stage.

[...]

> My reasoning for this is as follows:
> ? Example one: I am currently reviewing issue 2706 which covers
> datetime.timedelta operators. I have a reasonable amount of familiarity with
> time programming and think I have some valuable input on the functionality
> aspects, however I'm not a super-great C programmer. I'd like to help with
> coming up with the final feature specification, but then leave it to others
> to consider the merits of the C code patch. Being able to help get this
> marked as "request approved", then "specification approved" might help speed
> up a lot of the process of developing the patch and getting it accepted. It
> also saves code developers from having to develop an implementation while a
> functionality debate is still ongoing...

You can provide unit tests (and perhaps an equivalent Python
implementation),  they are a great way to specify behavior. Also, they
are required for a healthy commit and make the corner cases easier to
spot.

> ? I think that having these additional steps will help to avoid erroneous
> development of non-features, and also break up the review process into more
> manageable chunks of work.

I think core developers shouldn't waste time setting a hundred little
fields on each issue, but giving them practical ways to query (and
denote) these extended properties seems worthwhile.

> Of course I realise not everything needs such a
> delineated process, but by the same token it might assist in keeping some of
> the less visible/popular changes moving through the process...

Well, there's the signal-to-noise ratio to consider too. Like with the
nosy_count noise issue, things might get in the way of developers work
(I'll work on this soon, promise :D).

I'll be doing some janitorial tasks this week and next, triaging
issues and populating their fields. If you want to discuss specific
changes or suggest different methods/goals/rules for this work, I'd be
very grateful. If you want to join the tracker janitors club, remember
to bring a shrubbery :)

Regards,
 Daniel

From tjreedy at udel.edu  Tue Mar 10 05:07:37 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 10 Mar 2009 00:07:37 -0400
Subject: [Python-Dev] Addition of further status options to tracker
In-Reply-To: <bbaeab100903092039m7dcd73a4j93bb9f3255bf963e@mail.gmail.com>
References: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>
	<bbaeab100903092039m7dcd73a4j93bb9f3255bf963e@mail.gmail.com>
Message-ID: <gp4p2e$5fb$1@ger.gmane.org>

Brett Cannon wrote:
> 
> 
> On Mon, Mar 9, 2009 at 20:25, Tennessee Leeuwenburg 
> <tleeuwenburg at gmail.com <mailto:tleeuwenburg at gmail.com>> wrote:
> 
>     Hi all,
> 
>     I am beginning reviewing some more issues in the tracker. I think it
>     would be useful to have the following status options (new status
>     options marked with a '+'):
>       Open: Means that the issue has been created and not further reviewed
>       + Request Approved: Means that the issue is marked as worth
>     further development by the community
>       + Specification Approved: Means that the functionality to be
>     developed is written down in some form, and agreed at least in
>     general terms (preferably in fairly specific terms)
>       + Under Development: Means that an implementation is currently
>     under development
>       Pending Feedback: Means that work is suspended pending feedback
>       + Under Final Review: Means that a patch has been submitted and
>     may be a flag that core developers can usefully review the work done
>     without having to revisit the whole process
>       Closed: Means that the issue is either suspended indefinitely or
>     has been resolved (see resolution value)
> 
> 
>  I assume you want all of this for the Status field, correct?
> 
> As for the options, some of these overlap with the Stage field. For 
> instance, if something has been set to any stage other than 
> "accepted/rejected" it means it needs to be looked into, so that 
> duplicates your "Request Approved" status. Similar thing with the review 
> stages and "Under Final Review".
> 
> But a general "under development" status would probably be worth adding. 
> That way if an issue is "open" it needs attention, "Under development" 
> means someone is working on a solution, "pending" means someone is 
> blocking the issue for more information, and "closed" means closed.

I like this.  Open would mean that triage is still needed: reject and 
close (or provisionally reject 'pending'), fix and close, or move to 
'under developemnt.

> One thing to watch out for, Tennessee, is getting too specific. Like 
> your "Request Approved" and "Specification Approved" just seems too 
> heavy-handed to my tastes. Personally, I prefer to make sure the issue 
> workflow to be somewhat simple so that people don't end up arguing over 
> stuff like whether something has been specified well enough to have it 
> set to "Spec Approved" even if someone else disagrees (which you know 
> would happen).

The other problem with too many specifics is non-use.  As it is, an 
issue is sometimes closed with no resolution marked, so one has to 
scroll down, possibly a long way, to see whether it was accepted or 
rejected.  (Is it possible to require a resolution when closing?)

Terry


From ajaksu at gmail.com  Tue Mar 10 05:20:25 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 10 Mar 2009 01:20:25 -0300
Subject: [Python-Dev] Addition of further status options to tracker
In-Reply-To: <gp4p2e$5fb$1@ger.gmane.org>
References: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>
	<bbaeab100903092039m7dcd73a4j93bb9f3255bf963e@mail.gmail.com>
	<gp4p2e$5fb$1@ger.gmane.org>
Message-ID: <2d75d7660903092120u15fb24a9qa0ca0641d072a197@mail.gmail.com>

Terry Reedy  wrote:
> The other problem with too many specifics is non-use. ?As it is, an issue is
> sometimes closed with no resolution marked, so one has to scroll down,
> possibly a long way, to see whether it was accepted or rejected. ?(Is it
> possible to require a resolution when closing?)

Yes, it is possible. If you think it should be implemented, file a RFE
in the meta tracker.

However, it'll also be much easier to mass-edit issues in the near
future, so if this kind of (careful) cleanup can be done by tracker
janitors later, it might be best to leave core developers free of
these requirements...

Soon we'll also have the option of marking an issue as Pending and
having it automatically closed some time later.

Daniel

From aahz at pythoncraft.com  Tue Mar 10 05:22:04 2009
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 9 Mar 2009 21:22:04 -0700
Subject: [Python-Dev] More on Py3K urllib -- urlencode()
In-Reply-To: <49B5DBFE.3060505@digidescorp.com>
References: <49A99E7B.7030602@digidescorp.com> <34291.1235858913@parc.com>
	<35441.1235863997@parc.com> <49B28B20.7000509@digidescorp.com>
	<20090307152424.GA9464@panix.com> <98678.1236450982@parc.com>
	<49B5DBFE.3060505@digidescorp.com>
Message-ID: <20090310042204.GA29477@panix.com>

On Mon, Mar 09, 2009, Dan Mahn wrote:
>
> Any suggestions would be welcome before I try to submit this as a patch.

Just go ahead and submit it now; it's easier to review patches when
they're in the system, and it also makes sure that it won't get lost.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of     
indirection."  --Butler Lampson

From brett at python.org  Tue Mar 10 05:23:54 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 9 Mar 2009 21:23:54 -0700
Subject: [Python-Dev] Addition of further status options to tracker
In-Reply-To: <43c8685c0903092100i3457c5b0wbdaae31272a47895@mail.gmail.com>
References: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>
	<bbaeab100903092039m7dcd73a4j93bb9f3255bf963e@mail.gmail.com>
	<43c8685c0903092100i3457c5b0wbdaae31272a47895@mail.gmail.com>
Message-ID: <bbaeab100903092123v5aede825wcb80e6f80e63edb7@mail.gmail.com>

On Mon, Mar 9, 2009 at 21:00, Tennessee Leeuwenburg
<tleeuwenburg at gmail.com>wrote:

>
>
> On Tue, Mar 10, 2009 at 2:39 PM, Brett Cannon <brett at python.org> wrote:
>
>>
>>
>> On Mon, Mar 9, 2009 at 20:25, Tennessee Leeuwenburg <
>> tleeuwenburg at gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I am beginning reviewing some more issues in the tracker. I think it
>>> would be useful to have the following status options (new status options
>>> marked with a '+'):
>>>   Open: Means that the issue has been created and not further reviewed
>>>   + Request Approved: Means that the issue is marked as worth further
>>> development by the community
>>>   + Specification Approved: Means that the functionality to be developed
>>> is written down in some form, and agreed at least in general terms
>>> (preferably in fairly specific terms)
>>>   + Under Development: Means that an implementation is currently under
>>> development
>>>   Pending Feedback: Means that work is suspended pending feedback
>>>   + Under Final Review: Means that a patch has been submitted and may be
>>> a flag that core developers can usefully review the work done without having
>>> to revisit the whole process
>>>   Closed: Means that the issue is either suspended indefinitely or has
>>> been resolved (see resolution value)
>>
>>
>>  I assume you want all of this for the Status field, correct?
>>
>> As for the options, some of these overlap with the Stage field. For
>> instance, if something has been set to any stage other than
>> "accepted/rejected" it means it needs to be looked into, so that duplicates
>> your "Request Approved" status. Similar thing with the review stages and
>> "Under Final Review".
>>
>> But a general "under development" status would probably be worth adding.
>> That way if an issue is "open" it needs attention, "Under development" means
>> someone is working on a solution, "pending" means someone is blocking the
>> issue for more information, and "closed" means closed.
>>
>> One thing to watch out for, Tennessee, is getting too specific. Like your
>> "Request Approved" and "Specification Approved" just seems too heavy-handed
>> to my tastes. Personally, I prefer to make sure the issue workflow to be
>> somewhat simple so that people don't end up arguing over stuff like whether
>> something has been specified well enough to have it set to "Spec Approved"
>> even if someone else disagrees (which you know would happen).
>>
>> -Brett
>>
>
> Hi Brett,
>
> Some great points. The "stage" settings do overlap a lot of what I had
> suggested. What I'm trying to reach is a way to flag quite clearly whether a
> tracker issue is:
>   * In need of a first response
>   * Being taken care of by someone
>   * Needs the attention of core developers, just about ready
>
> At the moment, the open/closed settings are a bit useless -- issues are
> "Open" until they are finished, at which point they are "Closed". I
> personally suspect that "Pending feedback" is little-used.
>

For now. But once auto-closing of pending issues is set up then it will get
used consistently (don't ask me when that is happening, I just know Daniel
and Martin have been talking about it).


> At the moment, many "Open" issues are in various states of maturity or
> disrepair, so the only really useful information is whether an issue is
> "Closed".
>
> Examining the stage options more closely, they are very code-oriented.
>

Right, because we are dealing with code. =)


> Really they relate to implementation, not other parts of the process. It's
> difficult for me as someone who wants to help out with the Issue resolution
> process as much as coding to take an atomic action to help advance the issue
> through a process.
>
> What do you think about the following "Status" flags:
>
>   * Open: Means newly-created, needs early attention
>   * + Request for Comment: Means that a discussion regarding functionality
> is requested
>   * + Under Development: Means that a caring developer is taking care of it
>   * Pending Feedback: blocking on new information
>   * Closed: Means closed
>
> Once the issue is marked as under development, more information is then
> provided by the "Stage" flag.
>
> "Request for Comment" means that anyone who is stuck on trying to get their
> ideas sorted or a loose specification agreed on can call for feedback
> through this flag. There's still some risk of arguments blocking the
> progression to "Under development", but the issue owner can at least pass to
> "Under development" at their own will without requiring universal agreement.
>

There is also a worry of abuse if anyone can set this. If only people in the
Developer role can set this then that's fine. But if people start using this
just to get attention because they are unhappy with how slowly something is
going it will kill its usefulness.


>
> Core developers can then search on 'commit review' or 'patch review' to see
> what could use their attention, while people who are happy to contribute to
> the debate can search on "Open" and "Request for Comment" issues. Obviously
> the RFC flag is redundant when an issue is first raised, since the submitter
> always has the option of simply emailing this list. However, for older,
> potentially stale issues, it is a useful indicator that the issue may be
> either closed out or could use refreshing.
>
> I don't mind what approach is taken -- I'm happy to work within the current
> infrastructure if someone can suggest a good way. I really just want to be
> able to start distinguishing between issues that are essentially new and
> under debate versus issues that most people agree are a "Good Thing To Do",
> and then helping those issues advance to the point of implementation by
> discussing and, if necessary, formalising or recording requirements against
> them.
>

I have always viewed it that if Stage is set to anything other than its
empty default it is worth looking into. But it seems to me what you are
after is some obvious way to disambiguate issues that are feature requests
that someone has just asked for and anyone can work on if they want, and
issues that require attention, i.e. bugs and patches. Is that accurate?

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090309/2dbf1b7b/attachment-0001.htm>

From brett at python.org  Tue Mar 10 05:29:13 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 9 Mar 2009 21:29:13 -0700
Subject: [Python-Dev] reviewing patches
In-Reply-To: <gp4oce$461$1@ger.gmane.org>
References: <43c8685c0903091701m6413ded5qa15f1ba4d3c8a0bb@mail.gmail.com>
	<49B5B702.5050709@v.loewis.de> <gp4oce$461$1@ger.gmane.org>
Message-ID: <bbaeab100903092129g6e1cccd0i5c2c85ba9639999f@mail.gmail.com>

On Mon, Mar 9, 2009 at 20:55, Terry Reedy <tjreedy at udel.edu> wrote:

> Martin v. L?wis wrote:
>
>> I have seen it said that one very useful activity is reviewing patches.
>>> Of the issues in the tracker, it is not immediately clear to me what is
>>> required of such a review. Many of these patches appear to be bundled in
>>> with feature requests, leaving the question of whether the review it
>>> judging the quality of the code or the merits of the feature request.
>>>
>>
> Both may be needed, depending on the issue.  But see below.
>
>>
>> That's correct.
>>
>>  I
>>> do realise that these issues have probably been previously discussed on
>>> this list. Let's take as an example the following issue:
>>>
>>>  http://bugs.python.org/issue1818
>>>
>>
> Briefly, as I read it: Raymond proposed that his named tuples be used with
> Barry and Skip's csv module.  He got a short reply from Skip but went on to
> other things.  A year later, two new people entered and energized the
> discussion, Barry said 'useful', Skip commented on the details.  The second
> half of the discussion has been thrashing out and getting consesus on the
> devilish details.  What is devilish is that the merits of feature details
> sometimes interacts with code quality.  This is pretty typical.  It looks to
> me like this will be in 3.1.
>
> > often discussion is carried out entirely in the tracker (and
> > if you aren't on the nosy list of the issue, you will miss the
> > discussion completely)
>
> Every Friday a list of opened and closed issues is posted.  Even if you
> have nothing to say, you can add yourself to the nosy list.
>
> >> I feel a lot more
> >> qualified to evaluate code quality than desirability of function.
>
> Good, in the sense that I believe the former is rarer and more needed.
>
> There are 5 broad areas covered in the tracker:
> 1. C code, interpreter core
> 2. C code, extension modules
> 3. Python code, modules
> 4. Tests of the above
> 5. Documentation of above
>

This is somewhat covered by components, but it's implicit. Would it be worth
making this explicit? I have always wondered if people would be more willing
to help out if they could easily search for pure Python code issues if that
is as far as they feel comfortable. We could then have the components merge
into keywords or just take out the implicit type of issue part.

Or we just don't worry about auto-assignment. I think Georg is the only one
getting auto-assignment and I think we all know to assign him doc stuff. =)

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090309/c3dda599/attachment.htm>

From tleeuwenburg at gmail.com  Tue Mar 10 06:34:15 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Tue, 10 Mar 2009 16:34:15 +1100
Subject: [Python-Dev] Review of Issue http://bugs.python.org/issue2706,
	timedelta operators
Message-ID: <43c8685c0903092234k38976501k15e9d534fd857bc@mail.gmail.com>

Hi all,

I'm just trying to find my feet with regards to the proper process for
reviewing. I'm not sure what's best:
  * Post the review as a comment on the associated issue. Only nosies will
be updated.
  * Post the review as a comment on the issue, then post a one-line
information update to this list
  * Post the review to this list for consideration
  * ???? other

http://bugs.python.org/issue2706
General topic: Math operations on timedelta objects
Last activity: 12 Dec 08
Original owner: webograph

This issue covers a number of discrete functional changes. I here review
each in turn:

1) Allow truediv, the operator '/', to be applied to two timedeltas (e.g.
td1 / td2). The return value is a float representing the number of times
that td2 goes into td1.

Evaluation: Since both time deltas are quantitative values of the same unit,
it makes sense they should be able to support basic math operations. In this
case, operation to be added is '/' or truediv. I would personally find this
functionality useful, and believe it is a natural addition to the code.

Recommendation: That this functionality be recommended for development

2) Allow truediv to be applied to a timedelta and an int or float. The
return value is a timedelta representing the fractional proportion of the
original timedelta.

Evaluation: This makes total sense.
Recommendation: That this functionality be recommended for development

2) Allow divmod, the operator '%', to be applied to two timedeltas (e.g. td1
% td2). I think there is some debate here about whether the return value be
an integer in microsends, or a timedelta. I personally believe that a
timedelta should be returned, representing the amount of time remaining
after (td1 // td2)  * td2 has been subtracted.

The alternative is an integer, but due to a lack of immediate clarity over
what time quanta this integer represents, I would suggest returning a unit
amount. There is also some discussion of returning (long, timedelta), but I
personally fail to see the merits of returning the long without some unit
attached.

3) Allow divmod, the operator '%', to be applied to a timedelta and an
integer or float. (e.g. <timedelta> % 3). I'm not quite as sold on this.  I
suggest that more discussion is required to flesh this out further.


A patch has been attached which implements some of this behaviour. I would
suggest that it's valuable to start doing this work in discrete chunks so
that progress can be begun before the issues under debate are resolved. I
suggest that it would be appropriate to create three smaller issues, each
tackling just one piece of functionality. This should make the changes more
atomic and the code patch simpler.

-T

-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090310/40de2897/attachment.htm>

From tleeuwenburg at gmail.com  Tue Mar 10 06:39:29 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Tue, 10 Mar 2009 16:39:29 +1100
Subject: [Python-Dev] Addition of further status options to tracker
In-Reply-To: <bbaeab100903092123v5aede825wcb80e6f80e63edb7@mail.gmail.com>
References: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>
	<bbaeab100903092039m7dcd73a4j93bb9f3255bf963e@mail.gmail.com>
	<43c8685c0903092100i3457c5b0wbdaae31272a47895@mail.gmail.com>
	<bbaeab100903092123v5aede825wcb80e6f80e63edb7@mail.gmail.com>
Message-ID: <43c8685c0903092239n79bf013cle7b7e0c4135027d9@mail.gmail.com>

>
>  I don't mind what approach is taken -- I'm happy to work within the
>> current infrastructure if someone can suggest a good way. I really just want
>> to be able to start distinguishing between issues that are essentially new
>> and under debate versus issues that most people agree are a "Good Thing To
>> Do", and then helping those issues advance to the point of implementation by
>> discussing and, if necessary, formalising or recording requirements against
>> them.
>>
>
> I have always viewed it that if Stage is set to anything other than its
> empty default it is worth looking into. But it seems to me what you are
> after is some obvious way to disambiguate issues that are feature requests
> that someone has just asked for and anyone can work on if they want, and
> issues that require attention, i.e. bugs and patches. Is that accurate?


Pretty much. I've got two views. One is that I'd like to search for issues
that are up for grabs which I could take over, hack on, and generally not
get underfoot of core development activity. The other is that I think I can
help with issue gardening -- splitting issues up into those which still need
some more thought, those which someone should pick up and start working on,
etc.

   * Ideas needing more consideration
   * Ideas being hotly debated
   * Good ideas with no owning developer
   * Ideas currently under initial development
   * Ideas ready for consideration by the core developers

In order to make the most use of experienced developers, I'd say it's
important that they spend most of their time concentrated at the bottom of
that list. Bugs and patches more or less automatically qualify for that, and
they can be searched on pretty effectively now. However, doing gardening on
the issues which aren't quite there yet is currently pretty clumsy.

Say I'm a core developer and very busy. I'd like to quickly address bugs if
possible, and I'm happy to help out on important new issues. I certainly
don't want to trawl through a whole lot of immature issues and do the triage
myself -- what a waste of time!  Right now, what can such a person search on
to find the relevant issues? It looks like "patch review" or "commit review"
will do nicely. This isn't going to change, so that is still supported. Bugs
are essentially everything not a feature request, so that doesn't change
either.

Okay, so the developer workflow is simple and supported. But what about
pre-core development workflow?

How about for those involved in performing triage and specification? Where I
work, at least 50% of the time is spent just working out what to do. There's
a lot of real work in triage and specification, and it really shouldn't be
done by core developers who could be churning out shiny new stuff. That
means that the triage team (or issue janitors) need to be able to support a
workflow this is pretty easy on them, too. At this stage, we're not dealing
with code, we're just dealing with problems.

If we want people with great ideas to get to the top of the heap quickly, we
need some way to facilitate that, while issues that are either inappropriate
or being hotly contested don't suck time away from more important things.

Anyway, I really do just want to fit in. I'm just butting into some workflow
things which seem a bit clumsy when doing issue gardening or when finding
coding tasks that are up for grabs for a python-dev newbie...

Cheers,
-T
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090310/ce7bd274/attachment-0001.htm>

From ben+python at benfinney.id.au  Tue Mar 10 09:19:24 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 10 Mar 2009 19:19:24 +1100
Subject: [Python-Dev] Tracker cleanup - Roundup hacking report :)
References: <2d75d7660903050917n340ede0u4746bc8395e07b12@mail.gmail.com>
Message-ID: <874oy123jn.fsf@benfinney.id.au>

"Daniel (ajax) Diniz" <ajaksu at gmail.com> writes:

> Feedback on meta-tracker open issues, as well as new RFEs and
> replies to tracker-discuss threads should make the tracker meeting
> your needs a bit more likely :)

Thank you for <URL:http://issues.roundup-tracker.org/issue2550523>. I
have wanted to participate on the Python issue tracker, but Roundup's
lack of OpenID login support has made it more rewarding to simply
discuss on mailing lists.

I think that allowing people to log in using their *existing*
identities will lower the barrier significantly to reporting bugs or
contributing information.

In short: +1 :-)


From amauryfa at gmail.com  Tue Mar 10 11:43:22 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 10 Mar 2009 11:43:22 +0100
Subject: [Python-Dev] Integrate lxml into the stdlib?
In-Reply-To: <49B19134.2070803@v.loewis.de>
References: <49ABCFC9.1070508@gmail.com>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<gop8qa$csc$1@ger.gmane.org>
	<ca471dc20903051135j35ef8bdax30be5f85c09f0ed4@mail.gmail.com>
	<gopd12$t0k$1@ger.gmane.org> <49B05F43.5030706@v.loewis.de>
	<gor3lk$gao$1@ger.gmane.org> <goro00$okr$1@ger.gmane.org>
	<ca471dc20903061025v3bb9e9c4n9f37dcd1a05c60dc@mail.gmail.com>
	<49B19134.2070803@v.loewis.de>
Message-ID: <e27efe130903100343m4e7d05b2j83342966e9a5dbb3@mail.gmail.com>

On Fri, Mar 6, 2009 at 22:10, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> libs/_sqlite3.lib 2K
>>
>> I think this is a summary of the entry points into one of the above
>> DLLs for the benefit of other code wanting to link against it, but I'm
>> not sure.
>
> Correct. I don't know why I include them in the MSI - they are there
> because they were also shipped with the Wise installer. I see no
> use - nobody should be linking against an extension module.

They even cause trouble.
Just yesterday I (well, not me: the pypy translation process) was
caught by the presence of the "bz2.lib" file,
which pypy found there, just because the linker lists c:\python25\LIBs
before other directories.
Of course the real bz2.lib, which defines the compression routines,
was installed somewhere else, and compilation failed.

-- 
Amaury Forgeot d'Arc

From solipsis at pitrou.net  Tue Mar 10 12:36:50 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 10 Mar 2009 11:36:50 +0000 (UTC)
Subject: [Python-Dev] draft 3.1 release schedule
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>
	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>
	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>
	<gp3q9m$glr$1@ger.gmane.org>
	<18869.44602.432059.2350@montanaro.dyndns.org>
	<21E2F630429946A9A80448A23E1FAA8D@RaymondLaptop1>
	<1afaf6160903091725n4cf47084t3177e8bd8f33f834@mail.gmail.com>
	<A9C5B7EB749841AE80F09D9CD12814A0@RaymondLaptop1>
Message-ID: <loom.20090310T113623-420@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:

> 
> >>> You might also want to collect a list of serious changes that you
> >>> want in this release;
> 
> I'm making minor updates to the decimal module to match the 1.68 version of
the spec.

What about decimal-in-C, by the way? Is anyone still working on it?

Regards

Antoine.



From solipsis at pitrou.net  Tue Mar 10 12:44:54 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 10 Mar 2009 11:44:54 +0000 (UTC)
Subject: [Python-Dev] Addition of further status options to tracker
References: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>
Message-ID: <loom.20090310T114103-23@post.gmane.org>

Tennessee Leeuwenburg <tleeuwenburg <at> gmail.com> writes:
> 
> Hi all,I am beginning reviewing some more issues in the tracker. I think it
would be useful to have the following status options (new status options marked
with a '+'):

I have to point out that the more alternatives there are to choose from, the
more choosing from them can feel like a bureaucratic burden. Since Python has no
formal development process (as opposed to, say, Twisted), I think the tracker
should match today's quite informal and flexible way of working.




From mrts.pydev at gmail.com  Tue Mar 10 13:00:32 2009
From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=)
Date: Tue, 10 Mar 2009 14:00:32 +0200
Subject: [Python-Dev] Deprecated __cmp__ and total ordering
Message-ID: <ad1f81530903100500w11db61a8m65787d73f3cc5eda@mail.gmail.com>

__cmp__ used to provide a convenient way to make all ordering operators work
by defining a single method. For better or worse, it's gone in 3.0.

To provide total ordering without __cmp__ one has to implement all of
__lt__, __gt__, __le__, __ge__, __eq__ and __ne__. However, in all but a few
cases it suffices only to provide a "real" implementation for e.g. __lt__
and define all the other methods in terms of it as follows:

class TotalOrderMixin(object):
    def __lt__(self, other):
        raise NotImplemented # override this

    def __gt__(self, other):
        return other < self

    def __le__(self, other):
        return not (other < self)

    def __ge__(self, other):
        return not (self < other)

__eq__ and __ne__ are somewhat special, although it is possible to define
them in terms of __lt__

    def __eq__(self, other):
        return not (self == other)

    def __ne__(self, other):
        return self < other or other < self

it may be inefficient.

So, to avoid the situation where all the projects that match
http://www.google.com/codesearch?q=__cmp__+lang%3Apython have to implement
their own TotalOrderMixin, perhaps one could be provided in the stdlib? Or
even better, shouldn't a class grow automagic __gt__, __le__, __ge__ if
__lt__ is provided, and, in a similar vein, __ne__ if __eq__ is provided?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090310/d0ee96dd/attachment.htm>

From fuzzyman at voidspace.org.uk  Tue Mar 10 14:19:43 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 10 Mar 2009 13:19:43 +0000
Subject: [Python-Dev] Deprecated __cmp__ and total ordering
In-Reply-To: <ad1f81530903100500w11db61a8m65787d73f3cc5eda@mail.gmail.com>
References: <ad1f81530903100500w11db61a8m65787d73f3cc5eda@mail.gmail.com>
Message-ID: <49B668EF.1010406@voidspace.org.uk>

Hello Mart,

This has been discussed before. Guido was against automatically filling 
in these methods based I think on the fact that this may not be what you 
want - worth searching the archives for.

See here for a class decorator that provides all rich comparison methods 
for classes that only implement one (e.g. __lt__):

http://code.activestate.com/recipes/576529/

Michael Foord

Mart S?mermaa wrote:
> __cmp__ used to provide a convenient way to make all ordering 
> operators work by defining a single method. For better or worse, it's 
> gone in 3.0.
>
> To provide total ordering without __cmp__ one has to implement all of 
> __lt__, __gt__, __le__, __ge__, __eq__ and __ne__. However, in all but 
> a few cases it suffices only to provide a "real" implementation for 
> e.g. __lt__ and define all the other methods in terms of it as follows:
>
> class TotalOrderMixin(object):
>     def __lt__(self, other):
>         raise NotImplemented # override this
>
>     def __gt__(self, other):
>         return other < self
>
>     def __le__(self, other):
>         return not (other < self)
>
>     def __ge__(self, other):
>         return not (self < other)
>
> __eq__ and __ne__ are somewhat special, although it is possible to 
> define them in terms of __lt__
>
>     def __eq__(self, other):
>         return not (self == other)
>
>     def __ne__(self, other):
>         return self < other or other < self
>
> it may be inefficient.
>
> So, to avoid the situation where all the projects that match 
> http://www.google.com/codesearch?q=__cmp__+lang%3Apython have to 
> implement their own TotalOrderMixin, perhaps one could be provided in 
> the stdlib? Or even better, shouldn't a class grow automagic __gt__, 
> __le__, __ge__ if __lt__ is provided, and, in a similar vein, __ne__ 
> if __eq__ is provided?
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From ncoghlan at gmail.com  Tue Mar 10 14:21:48 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 10 Mar 2009 23:21:48 +1000
Subject: [Python-Dev] Decimal in C status (was Re: draft 3.1 release
	schedule)
In-Reply-To: <loom.20090310T113623-420@post.gmane.org>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>	<gp3q9m$glr$1@ger.gmane.org>	<18869.44602.432059.2350@montanaro.dyndns.org>	<21E2F630429946A9A80448A23E1FAA8D@RaymondLaptop1>	<1afaf6160903091725n4cf47084t3177e8bd8f33f834@mail.gmail.com>	<A9C5B7EB749841AE80F09D9CD12814A0@RaymondLaptop1>
	<loom.20090310T113623-420@post.gmane.org>
Message-ID: <49B6696C.808@gmail.com>

Antoine Pitrou wrote:
> Raymond Hettinger <python <at> rcn.com> writes:
> 
>>>>> You might also want to collect a list of serious changes that you
>>>>> want in this release;
>> I'm making minor updates to the decimal module to match the 1.68 version of
> the spec.
> 
> What about decimal-in-C, by the way? Is anyone still working on it?

There was a bit of a difference in philosophy between Mark and Raymond
as to how best to proceed with speeding it up (Mark's patch [1] focuses
solely on speeding up mantissa manipulation, while Raymond suggested a
more extensive rewrite in C would be preferable), so progress on the
issue stalled.

It would probably require at least a comment from Raymond saying to
proceed with the custom mantissa type approach to get that patch moving
again. Alternatively, if Raymond actually gets the project sponsorship
he mentioned a couple of months ago, that would also get things moving
again (just in a different direction).

Cheers,
Nick.

[1] http://bugs.python.org/issue2486

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From python at rcn.com  Tue Mar 10 14:46:57 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 10 Mar 2009 06:46:57 -0700
Subject: [Python-Dev] Decimal in C status (was Re: draft 3.1
	releaseschedule)
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>	<gp3q9m$glr$1@ger.gmane.org>	<18869.44602.432059.2350@montanaro.dyndns.org>	<21E2F630429946A9A80448A23E1FAA8D@RaymondLaptop1>	<1afaf6160903091725n4cf47084t3177e8bd8f33f834@mail.gmail.com>	<A9C5B7EB749841AE80F09D9CD12814A0@RaymondLaptop1><loom.20090310T113623-420@post.gmane.org>
	<49B6696C.808@gmail.com>
Message-ID: <1BA21CC529C7487E942873DE1E7C0D0E@RaymondLaptop1>


[Nick Coghlan]
>> What about decimal-in-C, by the way? Is anyone still working on it?

I'm seeking funding for the project. If it is forthcoming, I intend to do
a pure C version that simply implements the spec and then adds
wrappers for the pure python interface.  That will save the cost
of constantly creating and modifying many pyobjects that currently
arise during intermediate calculations.  

It's possible to focus just of the mantissa calculations but the cost of
the actual arithmetic work is swapped by the overhead of managing
contexts, checking for special values, and memory allocations.
Without addressing those, I think decimal will remain critically
performance challenged compared to native floats (decimals
will never be that fast, but they can get close enough to make
them a viable alternative for many kinds of work).



Raymond

From python at rcn.com  Tue Mar 10 14:53:26 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 10 Mar 2009 06:53:26 -0700
Subject: [Python-Dev] Deprecated __cmp__ and total ordering
References: <ad1f81530903100500w11db61a8m65787d73f3cc5eda@mail.gmail.com>
Message-ID: <B1A76387858E45638B3C051F98788331@RaymondLaptop1>


[Mart S?mermaa]
> To provide total ordering without __cmp__ one has to implement all of
>  __lt__, __gt__, __le__, __ge__, __eq__ and __ne__. However, in all
> but a few cases it suffices only to provide a "real" implementation for
> e.g. __lt__ and define all the other methods in terms of it as follows:

FWIW, I'm working on a solution for the problem using class decorators.
The idea is that it would scan a class and fill-in missing methods based
on the ones already there.  That way, any one of the four ordering
relations can be provided as a starting point and we won't have to
annoint one like __lt__ as the one true underlying method.

When it's ready, I'll bring it to python-dev for discussion.


Raymond 


From fuzzyman at voidspace.org.uk  Tue Mar 10 14:57:15 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 10 Mar 2009 13:57:15 +0000
Subject: [Python-Dev] Deprecated __cmp__ and total ordering
In-Reply-To: <B1A76387858E45638B3C051F98788331@RaymondLaptop1>
References: <ad1f81530903100500w11db61a8m65787d73f3cc5eda@mail.gmail.com>
	<B1A76387858E45638B3C051F98788331@RaymondLaptop1>
Message-ID: <49B671BB.1030207@voidspace.org.uk>

Raymond Hettinger wrote:
>
> [Mart S?mermaa]
>> To provide total ordering without __cmp__ one has to implement all of
>>  __lt__, __gt__, __le__, __ge__, __eq__ and __ne__. However, in all
>> but a few cases it suffices only to provide a "real" implementation for
>> e.g. __lt__ and define all the other methods in terms of it as follows:
>
> FWIW, I'm working on a solution for the problem using class decorators.
> The idea is that it would scan a class and fill-in missing methods based
> on the ones already there.  That way, any one of the four ordering
> relations can be provided as a starting point and we won't have to
> annoint one like __lt__ as the one true underlying method.
>
> When it's ready, I'll bring it to python-dev for discussion.

Is there something you don't like about this one: 
http://code.activestate.com/recipes/576529/


Michael

>
>
> Raymond
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk 
>


-- 
http://www.ironpythoninaction.com/


From ncoghlan at gmail.com  Tue Mar 10 14:58:28 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 10 Mar 2009 23:58:28 +1000
Subject: [Python-Dev] Decimal in C status (was Re: draft 3.1
	releaseschedule)
In-Reply-To: <1BA21CC529C7487E942873DE1E7C0D0E@RaymondLaptop1>
References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com>	<ca471dc20902150859j32908dbcg1507e48c9a3799e3@mail.gmail.com>	<1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com>	<gp3q9m$glr$1@ger.gmane.org>	<18869.44602.432059.2350@montanaro.dyndns.org>	<21E2F630429946A9A80448A23E1FAA8D@RaymondLaptop1>	<1afaf6160903091725n4cf47084t3177e8bd8f33f834@mail.gmail.com>	<A9C5B7EB749841AE80F09D9CD12814A0@RaymondLaptop1><loom.20090310T113623-420@post.gmane.org>
	<49B6696C.808@gmail.com>
	<1BA21CC529C7487E942873DE1E7C0D0E@RaymondLaptop1>
Message-ID: <49B67204.8050703@gmail.com>

Raymond Hettinger wrote:
> 
> [Nick Coghlan]
>>> What about decimal-in-C, by the way? Is anyone still working on it?
> 
> I'm seeking funding for the project. If it is forthcoming, I intend to do
> a pure C version that simply implements the spec and then adds
> wrappers for the pure python interface.  That will save the cost
> of constantly creating and modifying many pyobjects that currently
> arise during intermediate calculations. 
> It's possible to focus just of the mantissa calculations but the cost of
> the actual arithmetic work is swapped by the overhead of managing
> contexts, checking for special values, and memory allocations.
> Without addressing those, I think decimal will remain critically
> performance challenged compared to native floats (decimals
> will never be that fast, but they can get close enough to make
> them a viable alternative for many kinds of work).

I actually agree with all that, but do you agree the mantissa work is
still worthwhile in the near term (i.e. 3.1) to address the 25% or so
slowdown between decimal in 2.x (with the mantissa as an 8-bit str) and
decimal in 3.0 (with the mantissa as a unicode str, because using a
bytes object to store the digits is actually slower due to the lack of a
bytes->long fast path)?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From python at rcn.com  Tue Mar 10 15:05:27 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 10 Mar 2009 07:05:27 -0700
Subject: [Python-Dev] Deprecated __cmp__ and total ordering
References: <ad1f81530903100500w11db61a8m65787d73f3cc5eda@mail.gmail.com>
	<B1A76387858E45638B3C051F98788331@RaymondLaptop1>
	<49B671BB.1030207@voidspace.org.uk>
Message-ID: <F17A12D4CC6440C7AC114D09338C940E@RaymondLaptop1>


[Mart S?mermaa]
>>> To provide total ordering without __cmp__ one has to implement all of
>>>  __lt__, __gt__, __le__, __ge__, __eq__ and __ne__. However, in all
>>> but a few cases it suffices only to provide a "real" implementation for
>>> e.g. __lt__ and define all the other methods in terms of it as follows:

[Raymond Hettinger]
>> FWIW, I'm working on a solution for the problem using class decorators.
>> The idea is that it would scan a class and fill-in missing methods based
>> on the ones already there.  That way, any one of the four ordering
>> relations can be provided as a starting point and we won't have to
>> annoint one like __lt__ as the one true underlying method.
>>
>> When it's ready, I'll bring it to python-dev for discussion.

[Michael Foord]
> Is there something you don't like about this one: http://code.activestate.com/recipes/576529/

Yes, that recipe has the basic idea!

I think the implementation can be cleaned-up quite a bit
and it can be made as fast as hand-written code (not
the setup time, but the actual introduced method).


Raymond 


From fuzzyman at voidspace.org.uk  Tue Mar 10 15:40:36 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 10 Mar 2009 14:40:36 +0000
Subject: [Python-Dev] Deprecated __cmp__ and total ordering
In-Reply-To: <F17A12D4CC6440C7AC114D09338C940E@RaymondLaptop1>
References: <ad1f81530903100500w11db61a8m65787d73f3cc5eda@mail.gmail.com>
	<B1A76387858E45638B3C051F98788331@RaymondLaptop1>
	<49B671BB.1030207@voidspace.org.uk>
	<F17A12D4CC6440C7AC114D09338C940E@RaymondLaptop1>
Message-ID: <49B67BE4.9090308@voidspace.org.uk>

Raymond Hettinger wrote:
>
> [Mart S?mermaa]
>>>> To provide total ordering without __cmp__ one has to implement all of
>>>>  __lt__, __gt__, __le__, __ge__, __eq__ and __ne__. However, in all
>>>> but a few cases it suffices only to provide a "real" implementation 
>>>> for
>>>> e.g. __lt__ and define all the other methods in terms of it as 
>>>> follows:
>
> [Raymond Hettinger]
>>> FWIW, I'm working on a solution for the problem using class decorators.
>>> The idea is that it would scan a class and fill-in missing methods 
>>> based
>>> on the ones already there.  That way, any one of the four ordering
>>> relations can be provided as a starting point and we won't have to
>>> annoint one like __lt__ as the one true underlying method.
>>>
>>> When it's ready, I'll bring it to python-dev for discussion.
>
> [Michael Foord]
>> Is there something you don't like about this one: 
>> http://code.activestate.com/recipes/576529/
>
> Yes, that recipe has the basic idea!
>

It was originally written after you issued a challenge at PyCon UK last 
year.

> I think the implementation can be cleaned-up quite a bit
> and it can be made as fast as hand-written code (not
> the setup time, but the actual introduced method).

OK

I'll take it back to Christian and Menno and see what we can come up with.

Michael


>
>
> Raymond 


-- 
http://www.ironpythoninaction.com/


From dan.mahn at digidescorp.com  Tue Mar 10 15:50:15 2009
From: dan.mahn at digidescorp.com (Dan Mahn)
Date: Tue, 10 Mar 2009 09:50:15 -0500
Subject: [Python-Dev] More on Py3K urllib -- urlencode()
In-Reply-To: <98678.1236450982@parc.com>
References: <49A99E7B.7030602@digidescorp.com> <34291.1235858913@parc.com>
	<35441.1235863997@parc.com> <49B28B20.7000509@digidescorp.com>
	<20090307152424.GA9464@panix.com> <98678.1236450982@parc.com>
Message-ID: <49B67E27.6050404@digidescorp.com>

I submitted an explanation of this and my proposed modification as issue 
5468.

http://bugs.python.org/issue5468

- Dan


Bill Janssen wrote:
> Aahz <aahz at pythoncraft.com> wrote:
> 
>> On Sat, Mar 07, 2009, Dan Mahn wrote:
>>> After a harder look, I concluded there was a bit more work to be done,  
>>> but still very basic modifications.
>>>
>>> Attached is a version of urlencode() which seems to make the most sense  
>>> to me.
>>>
>>> I wonder how I could officially propose at least some of these  
>>> modifications.
>> Submit a patch to bugs.python.org
> 
> And it would help if it included a lot of test cases.
> 
> Bill


From janssen at parc.com  Tue Mar 10 16:35:39 2009
From: janssen at parc.com (Bill Janssen)
Date: Tue, 10 Mar 2009 08:35:39 PDT
Subject: [Python-Dev] More on Py3K urllib -- urlencode()
In-Reply-To: <49B5DBFE.3060505@digidescorp.com>
References: <49A99E7B.7030602@digidescorp.com> <34291.1235858913@parc.com>
	<35441.1235863997@parc.com> <49B28B20.7000509@digidescorp.com>
	<20090307152424.GA9464@panix.com> <98678.1236450982@parc.com>
	<49B5DBFE.3060505@digidescorp.com>
Message-ID: <37669.1236699339@parc.com>

Dan Mahn <dan.mahn at digidescorp.com> wrote:

> Yes, that was a good idea.  I found some problems, and attached a new
> version.  It looks more complicated than I wanted, but it is a very
> regular repetition, so I hope it is generally readable.

That's great, but I was hoping for more tests in lib/test/test_urllib.py.

Bill

From alexander.belopolsky at gmail.com  Tue Mar 10 17:14:41 2009
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Tue, 10 Mar 2009 11:14:41 -0500
Subject: [Python-Dev] Review of Issue http://bugs.python.org/issue2706,
	timedelta operators
In-Reply-To: <43c8685c0903092234k38976501k15e9d534fd857bc@mail.gmail.com>
References: <43c8685c0903092234k38976501k15e9d534fd857bc@mail.gmail.com>
Message-ID: <d38f5330903100914p26ca1545l56860f2af488734c@mail.gmail.com>

On Tue, Mar 10, 2009 at 12:34 AM, Tennessee Leeuwenburg
<tleeuwenburg at gmail.com> wrote:
> .. I'm not sure what's best:
> ? * Post the review as a comment on the associated issue. Only nosies will
> be updated.

You should always do that.  If you feel that the nosy list is too
small and you want to raise awareness for the issue, you can either CC
the list or post a note with a link to the issue.  Please keep in mind
that tracker summaries are posted on this list every week and updated
issues move to the top of the default tracker view, so just posting a
new comment will already give it additional visibility.

> ? * Post the review as a comment on the issue, then post a one-line
> information update to this list

All comments should be posted to the tracker.  Comments summarizing
long tracker discussions and proposing a resolution may be appropriate
for a python-dev post, but if you do that, please reference the list
thread on the tracker as it develops here.

> ? * Post the review to this list for consideration

Please do this only after a tracker update.  This way your review will
not be lost.

From mrts.pydev at gmail.com  Tue Mar 10 17:58:19 2009
From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=)
Date: Tue, 10 Mar 2009 18:58:19 +0200
Subject: [Python-Dev] Deprecated __cmp__ and total ordering
In-Reply-To: <49B671BB.1030207@voidspace.org.uk>
References: <ad1f81530903100500w11db61a8m65787d73f3cc5eda@mail.gmail.com>
	<B1A76387858E45638B3C051F98788331@RaymondLaptop1>
	<49B671BB.1030207@voidspace.org.uk>
Message-ID: <ad1f81530903100958hb585e11u99453f8be7de297b@mail.gmail.com>

On Tue, Mar 10, 2009 at 3:57 PM, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

> Is there something you don't like about this one:
> http://code.activestate.com/recipes/576529/
>

Yes -- it is not in the standard library. As I said, eventually all the
15,000 matches on Google Code need to update their code and copy that
snippet to their util/, write tests for it etc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090310/1b9adc9b/attachment.htm>

From mahesh.sayibabu at gmail.com  Tue Mar 10 18:11:33 2009
From: mahesh.sayibabu at gmail.com (Mahesh S)
Date: Tue, 10 Mar 2009 22:41:33 +0530
Subject: [Python-Dev] PyS60 - mailbox
Message-ID: <984438e90903101011n7ce06e9av18b219ab328bf087@mail.gmail.com>

Hi,

I am from the PyS60 (Python port on S60 platform) team. We have ported the
Python 2.5.4 core in the latest 1.9.x series of PyS60.
http://wiki.opensource.nokia.com/projects/PyS60

Currently we have a problem with the mailbox module. Check the code snippet
below, from the mailbox module.

*def clean(self):
        """Delete old files in "tmp"."""
        now = time.time()
        for entry in os.listdir(os.path.join(self._**path, 'tmp')):
            path = os.path.join(self._path, 'tmp', entry)
            if now - os.path.getatime(path) > 129600:   # 60 * 60 * 36
                os.remove(path)*

The code in red, tries to delete the files that are not accessed in the last
36 hours. Any idea as to how this is supposed to work on platforms that do
not support access time, for example Symbian/S60 ? Any work around ?

- Mahesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090310/3bf2efb8/attachment.htm>

From brett at python.org  Tue Mar 10 18:18:07 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 10 Mar 2009 10:18:07 -0700
Subject: [Python-Dev] Addition of further status options to tracker
In-Reply-To: <43c8685c0903092239n79bf013cle7b7e0c4135027d9@mail.gmail.com>
References: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>
	<bbaeab100903092039m7dcd73a4j93bb9f3255bf963e@mail.gmail.com>
	<43c8685c0903092100i3457c5b0wbdaae31272a47895@mail.gmail.com>
	<bbaeab100903092123v5aede825wcb80e6f80e63edb7@mail.gmail.com>
	<43c8685c0903092239n79bf013cle7b7e0c4135027d9@mail.gmail.com>
Message-ID: <bbaeab100903101018q463803b1j3479bc33662feb5a@mail.gmail.com>

On Mon, Mar 9, 2009 at 22:39, Tennessee Leeuwenburg
<tleeuwenburg at gmail.com>wrote:

>  I don't mind what approach is taken -- I'm happy to work within the
>>> current infrastructure if someone can suggest a good way. I really just want
>>> to be able to start distinguishing between issues that are essentially new
>>> and under debate versus issues that most people agree are a "Good Thing To
>>> Do", and then helping those issues advance to the point of implementation by
>>> discussing and, if necessary, formalising or recording requirements against
>>> them.
>>>
>>
>> I have always viewed it that if Stage is set to anything other than its
>> empty default it is worth looking into. But it seems to me what you are
>> after is some obvious way to disambiguate issues that are feature requests
>> that someone has just asked for and anyone can work on if they want, and
>> issues that require attention, i.e. bugs and patches. Is that accurate?
>
>
> Pretty much. I've got two views. One is that I'd like to search for issues
> that are up for grabs which I could take over, hack on, and generally not
> get underfoot of core development activity.
>

OK, let's do what is necessary to flag issues like this so that people can
do this. That's why I like the "Under Development" status. Could rename
"open" to "available" or "unsolved" to more clearly mark those issues as up
for grabs.


> The other is that I think I can help with issue gardening -- splitting
> issues up into those which still need some more thought, those which someone
> should pick up and start working on, etc.
>
>    * Ideas needing more consideration
>    * Ideas being hotly debated
>    * Good ideas with no owning developer
>    * Ideas currently under initial development
>    * Ideas ready for consideration by the core developers
>
> In order to make the most use of experienced developers, I'd say it's
> important that they spend most of their time concentrated at the bottom of
> that list. Bugs and patches more or less automatically qualify for that, and
> they can be searched on pretty effectively now. However, doing gardening on
> the issues which aren't quite there yet is currently pretty clumsy.
>

Typically we use nosy lists to get specific people's attention. That or the
priority gets bumped if it turns out to be an important issue. Lastly,
people can email python-dev directly asking for input if all other attempts
to get attention have failed.


>
> Say I'm a core developer and very busy. I'd like to quickly address bugs if
> possible, and I'm happy to help out on important new issues. I certainly
> don't want to trawl through a whole lot of immature issues and do the triage
> myself -- what a waste of time!  Right now, what can such a person search on
> to find the relevant issues? It looks like "patch review" or "commit review"
> will do nicely. This isn't going to change, so that is still supported. Bugs
> are essentially everything not a feature request, so that doesn't change
> either.
>
> Okay, so the developer workflow is simple and supported. But what about
> pre-core development workflow?
>
> How about for those involved in performing triage and specification? Where
> I work, at least 50% of the time is spent just working out what to do.
> There's a lot of real work in triage and specification, and it really
> shouldn't be done by core developers who could be churning out shiny new
> stuff. That means that the triage team (or issue janitors) need to be able
> to support a workflow this is pretty easy on them, too. At this stage, we're
> not dealing with code, we're just dealing with problems.
>

In other words you want some way to flag an issue that just needs to be
talked about but is not ready to be coded. So status would go "open/new" ->
"chatting/needs help" -> "under dev" -> "closed" with "pending" fitting in
when necessary. Sound about right?

My worry with this is making sure the field gets updated.


>
> If we want people with great ideas to get to the top of the heap quickly,
> we need some way to facilitate that, while issues that are either
> inappropriate or being hotly contested don't suck time away from more
> important things.
>

Doesn't seem to have been much of a problem so far.


>
> Anyway, I really do just want to fit in. I'm just butting into some
> workflow things which seem a bit clumsy when doing issue gardening or when
> finding coding tasks that are up for grabs for a python-dev newbie...
>

What I have been telling people is to search for open issues that match up
with what you are looking for based on Stage (whether it be docs, testing,
writing patches), what it affects (based on Component so you can skip
C-related issues) and if it is flagged as easy or not. Your point is that's
fine, but that also turns up issues that people are already working on and
so people constantly bump up against issues that are being dealt with.

I can understand that, but I would worry that if we flag something as under
development it will simply be ignored by other people when they could
actually help out (write the docs for someone, etc.). Or even worse that
someone gets to a certain point with the development, walks away short of
finishing the work (say doesn't get the docs finished) and everyone
continues to ignore the issue because it is still flagged as under
development.

If we can come up with a simple solution to this problem (perhaps have
issues set to under development with no activity shift down a status level
after a month) then maybe we will have something everyone can be happy with.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090310/ecd5ac94/attachment-0001.htm>

From guido at python.org  Tue Mar 10 18:23:26 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Mar 2009 10:23:26 -0700
Subject: [Python-Dev] PyS60 - mailbox
In-Reply-To: <984438e90903101011n7ce06e9av18b219ab328bf087@mail.gmail.com>
References: <984438e90903101011n7ce06e9av18b219ab328bf087@mail.gmail.com>
Message-ID: <ca471dc20903101023k7767353ex86d9679410004aa@mail.gmail.com>

Great news on the port!

On your issue: Is this module relevant for the platform? Do you know
of any existing app that uses the qmail mailbox format?

As a work-around, I can think of several values to substitute for
atime: the current time, or the mtime, or perhaps even the max of
mtime and ctime.

--Guido

On Tue, Mar 10, 2009 at 10:11 AM, Mahesh S <mahesh.sayibabu at gmail.com> wrote:
> Hi,
>
> I am from the PyS60 (Python port on S60 platform) team. We have ported the
> Python 2.5.4 core in the latest 1.9.x series of PyS60.
> http://wiki.opensource.nokia.com/projects/PyS60
>
> Currently we have a problem with the mailbox module. Check the code snippet
> below, from the mailbox module.
>
> def clean(self):
> ??????? """Delete old files in "tmp"."""
> ??????? now = time.time()
> ??????? for entry in os.listdir(os.path.join(self._path, 'tmp')):
> ??????????? path = os.path.join(self._path, 'tmp', entry)
> ??????????? if now - os.path.getatime(path) > 129600:?? # 60 * 60 * 36
> ??????????????? os.remove(path)
>
> The code in red, tries to delete the files that are not accessed in the last
> 36 hours. Any idea as to how this is supposed to work on platforms that do
> not support access time, for example Symbian/S60 ? Any work around ?
>
> - Mahesh
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rowen at u.washington.edu  Tue Mar 10 18:24:07 2009
From: rowen at u.washington.edu (Russell E. Owen)
Date: Tue, 10 Mar 2009 10:24:07 -0700
Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions
References: <DA195505-BDC0-4AF4-B707-4839BA87075B@python.org>
	<ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com>
	<nad-34F90E.03222214022009@news.gmane.org>
	<rowen-8731E0.13531325022009@news.gmane.org>
	<nad-BE5DEE.16573426022009@news.gmane.org>
	<AD5BF985-3C9B-4D49-BFE6-0CFF1B57A00A@mac.com>
Message-ID: <rowen-B5D057.10240710032009@news.gmane.org>

In article <AD5BF985-3C9B-4D49-BFE6-0CFF1B57A00A at mac.com>,
 Ronald Oussoren <ronaldoussoren at mac.com> wrote:

> On 27 Feb, 2009, at 1:57, Ned Deily wrote:
> 
> > In article <rowen-8731E0.13531325022009 at news.gmane.org>,
> > "Russell E. Owen" <rowen at u.washington.edu> wrote:
> >> I want to follow up on this a bit. In the past if the Mac Python
> >> installer was built on a machine that did NOT have a locally  
> >> installed
> >> Tcl/Tk then it would fail to work with a locally installed Tcl/Tk:
> >> Python would segfault when trying to use Tkinter.
> >>
> >> The solution was to build the Mac python installer on a machine  
> >> with a
> >> locally installed Tcl/Tk. The resulting installer package would  
> >> work on
> >> all systems -- with or without locally installed Tcl/Tk.
> >>
> >> So...has this problem been worked around, or is the Mac installer  
> >> still
> >> built on a machine that has a locally installed Tcl/Tk?
> >
> > Ronald will have to answer that for sure since he built the installer
> > for 3.0.1.
> >
> > However, it seems to be true that the most recent python.org OS X
> > installers will always favor a /System/Library/Frameworks/{Tcl/Tk}:
> 
> That's correct, I don't have a locally installed Tcl/Tk on my laptop  
> at the moment and couldn't arrange for one in time for the 3.0.1  
> release.
> 
> BTW. The fact that this should result in crashes when a user does have  
> a locally installed Tcl/Tk is new to me. The reason earlier builds of  
> the OSX installer were build with a locally installed Tcl/Tk is that  
> several Tkinter users indicated that the system version is  
> significantly less useful than a local install.

Any hope of an updated Mac installer for 2.5.4 and/or 2.6 that is built 
with a locally installed Tcl/Tk? If/when you do that, I strongly 
recommend ActiveState Tcl/Tk 8.4.19 -- the last 8.4 release -- it has 
few bugs and very good performance.

(For Python 2.6 you could install Tcl/Tk 8.5 instead, but it might not 
work for folks using Apple's built-in Tcl/Tk 8.4. Maybe somebody else 
knows; if not it would require some testing.)

Note that even Python 2.5 built against Tcl/Tk 8.5 and most of it 
worked, but there were some known bugs.

-- Russell


From fuzzyman at voidspace.org.uk  Tue Mar 10 18:27:54 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 10 Mar 2009 17:27:54 +0000
Subject: [Python-Dev] Deprecated __cmp__ and total ordering
In-Reply-To: <ad1f81530903100958hb585e11u99453f8be7de297b@mail.gmail.com>
References: <ad1f81530903100500w11db61a8m65787d73f3cc5eda@mail.gmail.com>	<B1A76387858E45638B3C051F98788331@RaymondLaptop1>	<49B671BB.1030207@voidspace.org.uk>
	<ad1f81530903100958hb585e11u99453f8be7de297b@mail.gmail.com>
Message-ID: <49B6A31A.1050109@voidspace.org.uk>

Mart S?mermaa wrote:
>
>
> On Tue, Mar 10, 2009 at 3:57 PM, Michael Foord 
> <fuzzyman at voidspace.org.uk <mailto:fuzzyman at voidspace.org.uk>> wrote:
>
>     Is there something you don't like about this one:
>     http://code.activestate.com/recipes/576529/
>
>
> Yes -- it is not in the standard library. As I said, eventually all 
> the 15,000 matches on Google Code need to update their code and copy 
> that snippet to their util/, write tests for it etc.

That question was in reply to Raymond who said he was working on his own 
version.

Michael
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From rdmurray at bitdance.com  Tue Mar 10 18:47:21 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 10 Mar 2009 13:47:21 -0400 (EDT)
Subject: [Python-Dev] PyS60 - mailbox
In-Reply-To: <984438e90903101011n7ce06e9av18b219ab328bf087@mail.gmail.com>
References: <984438e90903101011n7ce06e9av18b219ab328bf087@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0903101323040.2596@kimball.webabinitio.net>

On Tue, 10 Mar 2009 at 22:41, Mahesh S wrote:
> I am from the PyS60 (Python port on S60 platform) team. We have ported the
> Python 2.5.4 core in the latest 1.9.x series of PyS60.
> http://wiki.opensource.nokia.com/projects/PyS60
>
> Currently we have a problem with the mailbox module. Check the code snippet
> below, from the mailbox module.
>
> *def clean(self):
>        """Delete old files in "tmp"."""
>        now = time.time()
>        for entry in os.listdir(os.path.join(self._**path, 'tmp')):
>            path = os.path.join(self._path, 'tmp', entry)
>            if now - os.path.getatime(path) > 129600:   # 60 * 60 * 36
>                os.remove(path)*
>
> The code in red, tries to delete the files that are not accessed in the last
> 36 hours. Any idea as to how this is supposed to work on platforms that do
> not support access time, for example Symbian/S60 ? Any work around ?

Many unix systems (especially laptops) now run with the 'noatime' option
set on their file systems.  On my system with noatime set, atime appears
to be equal to mtime, so assuming you have mtime, returning that for
atime would appear to be de facto standards compliant.

--
R. David Murray           http://www.bitdance.com

From jared.grubb at gmail.com  Tue Mar 10 19:08:36 2009
From: jared.grubb at gmail.com (Jared Grubb)
Date: Tue, 10 Mar 2009 11:08:36 -0700
Subject: [Python-Dev] Regexp 2.7
In-Reply-To: <loom.20090309T230621-950@post.gmane.org>
References: <loom.20090307T101047-371@post.gmane.org>
	<e04bdf310903091408g5db702b5l922a3907d68bb563@mail.gmail.com>
	<loom.20090309T230621-950@post.gmane.org>
Message-ID: <70F7487F-56C8-4CC3-A690-B626754FFF05@gmail.com>

Would there be any interest in augmenting the test case library for  
the regex stuff?

When I was working on PyPy, we were using a simplified regular  
expression matcher to implement the tokenizer for Python. I was able  
to take a lot of PCRE's regex tests and port them to test our regular  
expression implementation (to make sure the DFA's were being optimized  
properly, etc).

I believe the PCRE test library was under a very liberal license, and  
so we may be able to do the same here. If there's interest in it, I  
can do the same for Python.

Jared

On 9 Mar 2009, at 16:07, Antoine Pitrou wrote:

> Facundo Batista <facundobatista <at> gmail.com> writes:
>>
>>> Matthew Barnett has been doing a lot of work on the regular  
>>> expressions
> engine
>>> (it seems he hasn't finished yet) under http://bugs.python.org/issue2636 
>>> .
>>> However, the patches are really huge and touch all of the sre  
>>> internals. I
>>> wonder what the review process can be for such patches? Is there  
>>> someone
>>> knowledgeable enough to be able to review them?
>>
>> All test cases run ok? How well covered is that library?
>
> I don't know, I haven't even tried it.
>
> Regards
> Antoine.


From guido at python.org  Tue Mar 10 19:32:10 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Mar 2009 11:32:10 -0700
Subject: [Python-Dev] Regexp 2.7
In-Reply-To: <70F7487F-56C8-4CC3-A690-B626754FFF05@gmail.com>
References: <loom.20090307T101047-371@post.gmane.org>
	<e04bdf310903091408g5db702b5l922a3907d68bb563@mail.gmail.com>
	<loom.20090309T230621-950@post.gmane.org>
	<70F7487F-56C8-4CC3-A690-B626754FFF05@gmail.com>
Message-ID: <ca471dc20903101132u712001e3p50a2d662bcdf9e21@mail.gmail.com>

Hm, what's wrong with the existing set of regex test cases? This is
one of the most complete set of test cases in our test suite.

On Tue, Mar 10, 2009 at 11:08 AM, Jared Grubb <jared.grubb at gmail.com> wrote:
> Would there be any interest in augmenting the test case library for the
> regex stuff?
>
> When I was working on PyPy, we were using a simplified regular expression
> matcher to implement the tokenizer for Python. I was able to take a lot of
> PCRE's regex tests and port them to test our regular expression
> implementation (to make sure the DFA's were being optimized properly, etc).
>
> I believe the PCRE test library was under a very liberal license, and so we
> may be able to do the same here. If there's interest in it, I can do the
> same for Python.
>
> Jared
>
> On 9 Mar 2009, at 16:07, Antoine Pitrou wrote:
>
>> Facundo Batista <facundobatista <at> gmail.com> writes:
>>>
>>>> Matthew Barnett has been doing a lot of work on the regular expressions
>>
>> engine
>>>>
>>>> (it seems he hasn't finished yet) under
>>>> http://bugs.python.org/issue2636.
>>>> However, the patches are really huge and touch all of the sre internals.
>>>> I
>>>> wonder what the review process can be for such patches? Is there someone
>>>> knowledgeable enough to be able to review them?
>>>
>>> All test cases run ok? How well covered is that library?
>>
>> I don't know, I haven't even tried it.
>>
>> Regards
>> Antoine.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tjreedy at udel.edu  Tue Mar 10 19:43:47 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 10 Mar 2009 14:43:47 -0400
Subject: [Python-Dev] reviewing patches
In-Reply-To: <bbaeab100903092129g6e1cccd0i5c2c85ba9639999f@mail.gmail.com>
References: <43c8685c0903091701m6413ded5qa15f1ba4d3c8a0bb@mail.gmail.com>	<49B5B702.5050709@v.loewis.de>
	<gp4oce$461$1@ger.gmane.org>
	<bbaeab100903092129g6e1cccd0i5c2c85ba9639999f@mail.gmail.com>
Message-ID: <gp6cd4$tr4$1@ger.gmane.org>

Brett Cannon wrote:
>
> This is somewhat covered by components, but it's implicit. Would it be 
> worth making this explicit? I have always wondered if people would be 
> more willing to help out if they could easily search for pure Python 
> code issues if that is as far as they feel comfortable.

If and when I am ready to move from working on documentation issues 
(which seem to becoming fewer as the 3.x transition is completed) to 
code issues, that would be helpful.  What would be really helpful is to 
have library issues tagged and sorted by specific modules (or modules, 
if more than one), but I do not know how that might be done.


From dan.mahn at digidescorp.com  Tue Mar 10 19:45:54 2009
From: dan.mahn at digidescorp.com (Dan Mahn)
Date: Tue, 10 Mar 2009 13:45:54 -0500
Subject: [Python-Dev] More on Py3K urllib -- urlencode()
In-Reply-To: <37669.1236699339@parc.com>
References: <49A99E7B.7030602@digidescorp.com> <34291.1235858913@parc.com>
	<35441.1235863997@parc.com> <49B28B20.7000509@digidescorp.com>
	<20090307152424.GA9464@panix.com> <98678.1236450982@parc.com>
	<49B5DBFE.3060505@digidescorp.com> <37669.1236699339@parc.com>
Message-ID: <49B6B562.3060402@digidescorp.com>

Ahh ... I see.  I should have done a bit more digging to find where the 
standard tests were.

I created a few new tests that could be included in that test suite -- 
see the attached file.  Do you think that this would be sufficient?

- Dan


Bill Janssen wrote:
> Dan Mahn <dan.mahn at digidescorp.com> wrote:
> 
>> Yes, that was a good idea.  I found some problems, and attached a new
>> version.  It looks more complicated than I wanted, but it is a very
>> regular repetition, so I hope it is generally readable.
> 
> That's great, but I was hoping for more tests in lib/test/test_urllib.py.
> 
> Bill
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: new_urlencode_tests.py
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090310/8a6683cf/attachment.txt>

From aahz at pythoncraft.com  Tue Mar 10 20:13:39 2009
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 10 Mar 2009 12:13:39 -0700
Subject: [Python-Dev] More on Py3K urllib -- urlencode()
In-Reply-To: <49B6B562.3060402@digidescorp.com>
References: <49A99E7B.7030602@digidescorp.com> <34291.1235858913@parc.com>
	<35441.1235863997@parc.com> <49B28B20.7000509@digidescorp.com>
	<20090307152424.GA9464@panix.com> <98678.1236450982@parc.com>
	<49B5DBFE.3060505@digidescorp.com> <37669.1236699339@parc.com>
	<49B6B562.3060402@digidescorp.com>
Message-ID: <20090310191339.GA9166@panix.com>

On Tue, Mar 10, 2009, Dan Mahn wrote:
>
> Ahh ... I see.  I should have done a bit more digging to find where the  
> standard tests were.
>
> I created a few new tests that could be included in that test suite --  
> see the attached file.  Do you think that this would be sufficient?

First of all, please notice from the list traffic that except for Guido
(who gets special dispensation because he's BDFL), most messages do not
use top-posting:

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet?

Second, please follow the advice to put ALL patches on the tracker.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of     
indirection."  --Butler Lampson

From amk at amk.ca  Tue Mar 10 20:39:37 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Tue, 10 Mar 2009 15:39:37 -0400
Subject: [Python-Dev] Regexp 2.7
In-Reply-To: <ca471dc20903101132u712001e3p50a2d662bcdf9e21@mail.gmail.com>
References: <loom.20090307T101047-371@post.gmane.org>
	<e04bdf310903091408g5db702b5l922a3907d68bb563@mail.gmail.com>
	<loom.20090309T230621-950@post.gmane.org>
	<70F7487F-56C8-4CC3-A690-B626754FFF05@gmail.com>
	<ca471dc20903101132u712001e3p50a2d662bcdf9e21@mail.gmail.com>
Message-ID: <20090310193937.GA12968@amk-desktop.matrixgroup.net>

On Tue, Mar 10, 2009 at 11:32:10AM -0700, Guido van Rossum wrote:
> Hm, what's wrong with the existing set of regex test cases? This is
> one of the most complete set of test cases in our test suite.

There's never anything wrong with having more test cases!  However, if
you have a choice of which test suite to improve, I would choose
another module to work on.  The re module was once based on PCRE and a
lot of its test suite was ported into the Python one, so there may not
much relevant to add.  But please do whatever you like...

--amk

From jared.grubb at gmail.com  Tue Mar 10 20:52:25 2009
From: jared.grubb at gmail.com (Jared Grubb)
Date: Tue, 10 Mar 2009 12:52:25 -0700
Subject: [Python-Dev] Regexp 2.7
In-Reply-To: <ca471dc20903101132u712001e3p50a2d662bcdf9e21@mail.gmail.com>
References: <loom.20090307T101047-371@post.gmane.org>
	<e04bdf310903091408g5db702b5l922a3907d68bb563@mail.gmail.com>
	<loom.20090309T230621-950@post.gmane.org>
	<70F7487F-56C8-4CC3-A690-B626754FFF05@gmail.com>
	<ca471dc20903101132u712001e3p50a2d662bcdf9e21@mail.gmail.com>
Message-ID: <1EE4E769-B036-4196-A9FC-1623149EC989@gmail.com>

I'm not criticizing the current battery of tests, nor am I arguing  
that we replace them.

There's a comment in the test_re.py that says that "these tests were  
carefully modeled to cover most of the code"... That is a very  
difficult statement to maintain and/or verify, especially if the  
library gets a major revision (which it appears the original post's  
patch is).

PCRE has _thousands_ of detailed regular expression tests, testing  
everything from matching to parsing to extended regular expression  
syntax to encoding and locales. (It's been a while since I've looked  
at the details, but of course there are tests that dont apply to  
Python's implmentation.)

So, if there's interest in investigating how much of the PCRE tests  
can augment the existing tests, I am offering to do so. (I already did  
a simple translation utility to parse the PCRE test format into  
something we could use in the PyPy test suite; I could try to do  
something similar for test_re, if there's interest).

Jared

On 10 Mar 2009, at 11:32, Guido van Rossum wrote:

> Hm, what's wrong with the existing set of regex test cases? This is
> one of the most complete set of test cases in our test suite.
>
> On Tue, Mar 10, 2009 at 11:08 AM, Jared Grubb  
> <jared.grubb at gmail.com> wrote:
>> Would there be any interest in augmenting the test case library for  
>> the
>> regex stuff?
>>
>> On 9 Mar 2009, at 16:07, Antoine Pitrou wrote:
>>
>>> Facundo Batista <facundobatista <at> gmail.com> writes:
>>>>
>>>>> Matthew Barnett has been doing a lot of work on the regular  
>>>>> expressions engine
>>>>>
>>>>> (it seems he hasn't finished yet) under http://bugs.python.org/issue2636 
>>>>> .
>>>>> However, the patches are really huge and touch all of the sre  
>>>>> internals.
>>>>> I wonder what the review process can be for such patches? Is  
>>>>> there someone
>>>>> knowledgeable enough to be able to review them?
>>>>
>>>> All test cases run ok? How well covered is that library?
>>>
>>> I don't know, I haven't even tried it.
>>>

From brett at python.org  Tue Mar 10 21:02:08 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 10 Mar 2009 13:02:08 -0700
Subject: [Python-Dev] reviewing patches
In-Reply-To: <gp6cd4$tr4$1@ger.gmane.org>
References: <43c8685c0903091701m6413ded5qa15f1ba4d3c8a0bb@mail.gmail.com>
	<49B5B702.5050709@v.loewis.de> <gp4oce$461$1@ger.gmane.org>
	<bbaeab100903092129g6e1cccd0i5c2c85ba9639999f@mail.gmail.com>
	<gp6cd4$tr4$1@ger.gmane.org>
Message-ID: <bbaeab100903101302p5e30bd28p9d85fca0a0aa6e5f@mail.gmail.com>

On Tue, Mar 10, 2009 at 11:43, Terry Reedy <tjreedy at udel.edu> wrote:

> Brett Cannon wrote:
>
>>
>> This is somewhat covered by components, but it's implicit. Would it be
>> worth making this explicit? I have always wondered if people would be more
>> willing to help out if they could easily search for pure Python code issues
>> if that is as far as they feel comfortable.
>>
>
> If and when I am ready to move from working on documentation issues (which
> seem to becoming fewer as the 3.x transition is completed) to code issues,
> that would be helpful.  What would be really helpful is to have library
> issues tagged and sorted by specific modules (or modules, if more than one),
> but I do not know how that might be done.


It would have to be a text field that people just filled in. Making a list
that had to be kept up-to-date would be a disaster.

But one would hope that simply searching for an issue with a specific module
name would be enough to warrant not having to have the field.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090310/b1a5db3c/attachment-0001.htm>

From lists at cheimes.de  Tue Mar 10 21:11:38 2009
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 10 Mar 2009 21:11:38 +0100
Subject: [Python-Dev] Ext4 data loss
Message-ID: <gp6hhq$hss$1@ger.gmane.org>

Multiple blogs and news sites are swamped with a discussion about ext4
and KDE 4.0. Theodore Ts'o - the developer of ext4 - explains the issue
at
https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/317781/comments/54.


Python's file type doesn't use fsync() and be the victim of the very
same issue, too. Should we do anything about it?

Christian


From guido at python.org  Tue Mar 10 21:23:13 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Mar 2009 13:23:13 -0700
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <gp6hhq$hss$1@ger.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>
Message-ID: <ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>

On Tue, Mar 10, 2009 at 1:11 PM, Christian Heimes <lists at cheimes.de> wrote:
> Multiple blogs and news sites are swamped with a discussion about ext4
> and KDE 4.0. Theodore Ts'o - the developer of ext4 - explains the issue
> at
> https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/317781/comments/54.
>
>
> Python's file type doesn't use fsync() and be the victim of the very
> same issue, too. Should we do anything about it?

If I understand the post properly, it's up to the app to call fsync(),
and it's only necessary when you're doing one of the rename dances, or
updating a file in place. Basically, as he explains, fsync() is a very
heavyweight operation; I'm against calling it by default anywhere.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tjreedy at udel.edu  Tue Mar 10 21:45:27 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 10 Mar 2009 16:45:27 -0400
Subject: [Python-Dev] Regexp 2.7
In-Reply-To: <1EE4E769-B036-4196-A9FC-1623149EC989@gmail.com>
References: <loom.20090307T101047-371@post.gmane.org>	<e04bdf310903091408g5db702b5l922a3907d68bb563@mail.gmail.com>	<loom.20090309T230621-950@post.gmane.org>	<70F7487F-56C8-4CC3-A690-B626754FFF05@gmail.com>	<ca471dc20903101132u712001e3p50a2d662bcdf9e21@mail.gmail.com>
	<1EE4E769-B036-4196-A9FC-1623149EC989@gmail.com>
Message-ID: <gp6jh8$p27$1@ger.gmane.org>

Jared Grubb wrote:
> I'm not criticizing the current battery of tests, nor am I arguing that 
> we replace them.
> 
> There's a comment in the test_re.py that says that "these tests were 
> carefully modeled to cover most of the code"... That is a very difficult 
> statement to maintain and/or verify, especially if the library gets a 
> major revision (which it appears the original post's patch is).
> 
> PCRE has _thousands_ of detailed regular expression tests, testing 
> everything from matching to parsing to extended regular expression 
> syntax to encoding and locales. (It's been a while since I've looked at 
> the details, but of course there are tests that dont apply to Python's 
> implmentation.)
> 
> So, if there's interest in investigating how much of the PCRE tests can 
> augment the existing tests, I am offering to do so. (I already did a 
> simple translation utility to parse the PCRE test format into something 
> we could use in the PyPy test suite; I could try to do something similar 
> for test_re, if there's interest).

There is a conflict between running a thorough test of everything 
possible and not having the test suite run for hours.  I believe a 
couple of other modules have a regular sanity-check test and an extended 
patch-check test.  Something like that might be appropriate for re.


From nyamatongwe at gmail.com  Tue Mar 10 21:46:25 2009
From: nyamatongwe at gmail.com (Neil Hodgson)
Date: Wed, 11 Mar 2009 07:46:25 +1100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <gp6hhq$hss$1@ger.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>
Message-ID: <50862ebd0903101346t216028cgf7f01de7167fbb72@mail.gmail.com>

   The technique advocated by Theodore Ts'o (save to temporary then
rename) discards metadata. What would be useful is a simple, generic
way in Python to copy all the appropriate metadata (ownership, ACLs,
...) to another file so the temporary-and-rename technique could be
used.

   On Windows, there is a hack in the file system that tries to track
the use of temporary-and-rename and reapply ACLs and on OS X there is
a function FSPathReplaceObject but I don't know how to do this
correctly on Linux.

   Neil

From barry at python.org  Tue Mar 10 21:49:32 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 10 Mar 2009 16:49:32 -0400
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org>
	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
Message-ID: <0D51757C-F9B3-488A-9B50-667ADF1CA955@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 10, 2009, at 4:23 PM, Guido van Rossum wrote:

> On Tue, Mar 10, 2009 at 1:11 PM, Christian Heimes <lists at cheimes.de>  
> wrote:
>> Multiple blogs and news sites are swamped with a discussion about  
>> ext4
>> and KDE 4.0. Theodore Ts'o - the developer of ext4 - explains the  
>> issue
>> at
>> https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/317781/comments/54 
>> .
>>
>>
>> Python's file type doesn't use fsync() and be the victim of the very
>> same issue, too. Should we do anything about it?
>
> If I understand the post properly, it's up to the app to call fsync(),
> and it's only necessary when you're doing one of the rename dances, or
> updating a file in place. Basically, as he explains, fsync() is a very
> heavyweight operation; I'm against calling it by default anywhere.

Right.  Python /applications/ should call fsync() and do the rename  
dance if appropriate, and fortunately it's easy enough to implement in  
Python.  Mailman's queue runner has done exactly this for ages.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCUAwUBSbbSXHEjvBPtnXfVAQLrsQP2NxJL+js6fMDgluoSpB6kW+VCJfSS0G58
KaDiRniinl3E9qH9w+hvNE7Es9JzPSiOP79KkuqRkzvCCmkrQRvsY6dKukOs/1zq
KNpTB4I3bGzUHgM+OwAh2KuxJ3pXzOPhrPwLLXLq7k1OuGRODmPxWXZ+i8FirR7C
8fpV6wNFAQ==
=JIdS
-----END PGP SIGNATURE-----

From guido at python.org  Tue Mar 10 21:54:54 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Mar 2009 13:54:54 -0700
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <50862ebd0903101346t216028cgf7f01de7167fbb72@mail.gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org>
	<50862ebd0903101346t216028cgf7f01de7167fbb72@mail.gmail.com>
Message-ID: <ca471dc20903101354p8e68c6dy446ffb8e1b0f1a8e@mail.gmail.com>

On Tue, Mar 10, 2009 at 1:46 PM, Neil Hodgson <nyamatongwe at gmail.com> wrote:
> ? The technique advocated by Theodore Ts'o (save to temporary then
> rename) discards metadata. What would be useful is a simple, generic
> way in Python to copy all the appropriate metadata (ownership, ACLs,
> ...) to another file so the temporary-and-rename technique could be
> used.
>
> ? On Windows, there is a hack in the file system that tries to track
> the use of temporary-and-rename and reapply ACLs and on OS X there is
> a function FSPathReplaceObject but I don't know how to do this
> correctly on Linux.

I don't know how to implement this for metadata beyond the traditional
stat metadata, but the API could be an extension of shutil.copystat().

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Tue Mar 10 22:30:25 2009
From: barry at python.org (Barry Warsaw)
Date: Tue, 10 Mar 2009 17:30:25 -0400
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <50862ebd0903101346t216028cgf7f01de7167fbb72@mail.gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org>
	<50862ebd0903101346t216028cgf7f01de7167fbb72@mail.gmail.com>
Message-ID: <BAF87B91-3102-4374-8A70-021FFD77A058@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 10, 2009, at 4:46 PM, Neil Hodgson wrote:

>   The technique advocated by Theodore Ts'o (save to temporary then
> rename) discards metadata. What would be useful is a simple, generic
> way in Python to copy all the appropriate metadata (ownership, ACLs,
> ...) to another file so the temporary-and-rename technique could be
> used.
>
>   On Windows, there is a hack in the file system that tries to track
> the use of temporary-and-rename and reapply ACLs and on OS X there is
> a function FSPathReplaceObject but I don't know how to do this
> correctly on Linux.

Of course, a careful *nix application can ensure that the file owners  
and mod bits are set the way it needs them to be set.  A convenience  
function might be useful though.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSbbb8XEjvBPtnXfVAQLxvgP/SDnwzcKHI9E9K/ShAVWk3aShsDvJSztH
wHRQlOkbxxG/xcGJ7hGYaxJh5/TszU4wvtSc7JV5p6tRWrk/FRvAPW9lFBrlVQ8I
ZTV/bsNRJLSDxEXe7H4S2/c0L8LuGu58RGWtQzFH0UlnIFYIDwxxVGjfpVckXAc4
l54OAhDPFSk=
=njh4
-----END PGP SIGNATURE-----

From janssen at parc.com  Tue Mar 10 22:46:41 2009
From: janssen at parc.com (Bill Janssen)
Date: Tue, 10 Mar 2009 14:46:41 PDT
Subject: [Python-Dev] More on Py3K urllib -- urlencode()
In-Reply-To: <20090310191339.GA9166@panix.com>
References: <49A99E7B.7030602@digidescorp.com> <34291.1235858913@parc.com>
	<35441.1235863997@parc.com> <49B28B20.7000509@digidescorp.com>
	<20090307152424.GA9464@panix.com> <98678.1236450982@parc.com>
	<49B5DBFE.3060505@digidescorp.com> <37669.1236699339@parc.com>
	<49B6B562.3060402@digidescorp.com>
	<20090310191339.GA9166@panix.com>
Message-ID: <82503.1236721601@pippin.parc.xerox.com>

Aahz <aahz at pythoncraft.com> wrote:

> Second, please follow the advice to put ALL patches on the tracker.

I don't care about top-posting, but I second the Second point.  Let's move this
thread to the issue tracker.

Bill

From martin at v.loewis.de  Tue Mar 10 23:03:08 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 10 Mar 2009 23:03:08 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org>
	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
Message-ID: <49B6E39C.9040902@v.loewis.de>

> If I understand the post properly, it's up to the app to call fsync(),

Correct.

> and it's only necessary when you're doing one of the rename dances, or
> updating a file in place. 

No. It's in general necessary when you want to be sure that the data is
on disk, even if the power is lost. So even if you write a file (say, a
.pyc) only once - if the lights go out, and on again, your .pyc might be
corrupted, as the file system may have chosen to flush the metadata onto
disk, but not the actual data (or only parts of it). This may
happen even if the close(2) operation was successful.

In the specific case of config files, that's unfortunate because you
then can't revert to the old state, either - because that may be gone.
Ideally, you want transactional updates - you get either the old config
or the new config after a crash. You can get that with explicit
fdatasync, or with a transactional database (which may chose to sync
only infrequently, but then will be able to rollback the old state if
the new one wasn't written completely).

But yes, I agree, it's the applications' responsibility to properly
sync. If I had to place sync calls into the standard library, they would
go into dumbdbm.

I somewhat disagree that it is the application's fault entirely, and not
the operating system's/file system's fault. Ideally, there would be an
option of specifying transaction brackets for file operations, so that
the system knows it cannot flush the unlink operation of the old file
before it has flushed the data of the new file. This would still allow
the system to schedule IO fairly freely, but also guarantee that not all
gets lost in a crash. I thought that the data=ordered ext3 mount option
was going in that direction - not sure what happened to it in ext4.

Regards,
Martin

From amk at amk.ca  Tue Mar 10 23:09:15 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Tue, 10 Mar 2009 18:09:15 -0400
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <gp6hhq$hss$1@ger.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>
Message-ID: <20090310220915.GA15302@amk-desktop.matrixgroup.net>

On Tue, Mar 10, 2009 at 09:11:38PM +0100, Christian Heimes wrote:
> Python's file type doesn't use fsync() and be the victim of the very
> same issue, too. Should we do anything about it?

The mailbox module tries to be careful and always fsync() before
closing files, because mail messages are pretty important.  The
various *dbm modules mostly have .sync() method.  

dumbdbm.py doesn't call fsync(), AFAICT; _commit() writes stuff and
closes the file, but doesn't call fsync().

sqlite3 doesn't have a sync() or flush() call.  Does SQLite handle
this itself?

The tarfile, zipfile, and gzip/bzip2 classes don't seem to use fsync()
at all, either implicitly or by having methods for calling them.
Should they?  What about cookielib.CookieJar?

--amk

From ncoghlan at gmail.com  Tue Mar 10 23:27:47 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Mar 2009 08:27:47 +1000
Subject: [Python-Dev] Regexp 2.7
In-Reply-To: <gp6jh8$p27$1@ger.gmane.org>
References: <loom.20090307T101047-371@post.gmane.org>	<e04bdf310903091408g5db702b5l922a3907d68bb563@mail.gmail.com>	<loom.20090309T230621-950@post.gmane.org>	<70F7487F-56C8-4CC3-A690-B626754FFF05@gmail.com>	<ca471dc20903101132u712001e3p50a2d662bcdf9e21@mail.gmail.com>	<1EE4E769-B036-4196-A9FC-1623149EC989@gmail.com>
	<gp6jh8$p27$1@ger.gmane.org>
Message-ID: <49B6E963.2040703@gmail.com>

Terry Reedy wrote:
> There is a conflict between running a thorough test of everything
> possible and not having the test suite run for hours.  I believe a
> couple of other modules have a regular sanity-check test and an extended
> patch-check test.  Something like that might be appropriate for re.

Correct - a new regrtest resource (e.g. -uregex) could be added if the
enhanced regex tests took too much time to run. (-udecimal and
-ucompiler are the two existing examples of this that you're likely
thinking off)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Tue Mar 10 23:33:08 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Mar 2009 08:33:08 +1000
Subject: [Python-Dev] Addition of further status options to tracker
In-Reply-To: <bbaeab100903101018q463803b1j3479bc33662feb5a@mail.gmail.com>
References: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>	<bbaeab100903092039m7dcd73a4j93bb9f3255bf963e@mail.gmail.com>	<43c8685c0903092100i3457c5b0wbdaae31272a47895@mail.gmail.com>	<bbaeab100903092123v5aede825wcb80e6f80e63edb7@mail.gmail.com>	<43c8685c0903092239n79bf013cle7b7e0c4135027d9@mail.gmail.com>
	<bbaeab100903101018q463803b1j3479bc33662feb5a@mail.gmail.com>
Message-ID: <49B6EAA4.601@gmail.com>

Brett Cannon wrote:
> If we can come up with a simple solution to this problem (perhaps have
> issues set to under development with no activity shift down a status
> level after a month) then maybe we will have something everyone can be
> happy with.

If an issue is assigned, then somebody has claimed it and is working on
it (or it has at least been sent to a specific person for consideration).

Deciding that someone has dropped off the face of the planet and that
their issues should be unassigned is something that a human tracker
janitor would probably have to decide to do (the original assignee would
get email about the assignment changes and have the option of reclaiming
issues they still cared about).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From brett at python.org  Tue Mar 10 23:43:59 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 10 Mar 2009 15:43:59 -0700
Subject: [Python-Dev] Addition of further status options to tracker
In-Reply-To: <49B6EAA4.601@gmail.com>
References: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>
	<bbaeab100903092039m7dcd73a4j93bb9f3255bf963e@mail.gmail.com>
	<43c8685c0903092100i3457c5b0wbdaae31272a47895@mail.gmail.com>
	<bbaeab100903092123v5aede825wcb80e6f80e63edb7@mail.gmail.com>
	<43c8685c0903092239n79bf013cle7b7e0c4135027d9@mail.gmail.com>
	<bbaeab100903101018q463803b1j3479bc33662feb5a@mail.gmail.com>
	<49B6EAA4.601@gmail.com>
Message-ID: <bbaeab100903101543j1086cac6r6dd3d181a2ff7817@mail.gmail.com>

On Tue, Mar 10, 2009 at 15:33, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Brett Cannon wrote:
> > If we can come up with a simple solution to this problem (perhaps have
> > issues set to under development with no activity shift down a status
> > level after a month) then maybe we will have something everyone can be
> > happy with.
>
> If an issue is assigned, then somebody has claimed it and is working on
> it (or it has at least been sent to a specific person for consideration).
>

Right, but that is for core developers only. I think Tennessee is worrying
about non-core folks.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090310/3a26946c/attachment.htm>

From solipsis at pitrou.net  Wed Mar 11 00:20:58 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 10 Mar 2009 23:20:58 +0000 (UTC)
Subject: [Python-Dev] Regexp 2.7
References: <loom.20090307T101047-371@post.gmane.org>
	<e04bdf310903091408g5db702b5l922a3907d68bb563@mail.gmail.com>
	<loom.20090309T230621-950@post.gmane.org>
	<70F7487F-56C8-4CC3-A690-B626754FFF05@gmail.com>
	<ca471dc20903101132u712001e3p50a2d662bcdf9e21@mail.gmail.com>
	<1EE4E769-B036-4196-A9FC-1623149EC989@gmail.com>
Message-ID: <loom.20090310T231809-606@post.gmane.org>

Hello,

> So, if there's interest in investigating how much of the PCRE tests  
> can augment the existing tests, I am offering to do so.

IMO there's nothing wrong with having more tests, provided that:
- they don't make the test suite slower than it should be
- they aren't too implementation-specific

By the way, having tests is good, but having well-organized and documented tests
is even better (something which caught us when working on the io rewrite).

Regards

Antoine.



From tleeuwenburg at gmail.com  Wed Mar 11 00:35:04 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Wed, 11 Mar 2009 10:35:04 +1100
Subject: [Python-Dev] Fwd:  Addition of further status options to tracker
In-Reply-To: <43c8685c0903101554n7b05bb3ka78866f64972556f@mail.gmail.com>
References: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>
	<bbaeab100903092039m7dcd73a4j93bb9f3255bf963e@mail.gmail.com>
	<43c8685c0903092100i3457c5b0wbdaae31272a47895@mail.gmail.com>
	<bbaeab100903092123v5aede825wcb80e6f80e63edb7@mail.gmail.com>
	<43c8685c0903092239n79bf013cle7b7e0c4135027d9@mail.gmail.com>
	<bbaeab100903101018q463803b1j3479bc33662feb5a@mail.gmail.com>
	<49B6EAA4.601@gmail.com>
	<bbaeab100903101543j1086cac6r6dd3d181a2ff7817@mail.gmail.com>
	<43c8685c0903101554n7b05bb3ka78866f64972556f@mail.gmail.com>
Message-ID: <43c8685c0903101635w7b17abfdq11ead1e2eb9fe098@mail.gmail.com>

On Wed, Mar 11, 2009 at 9:43 AM, Brett Cannon <brett at python.org> wrote:

>
>
> On Tue, Mar 10, 2009 at 15:33, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> Brett Cannon wrote:
>> > If we can come up with a simple solution to this problem (perhaps have
>> > issues set to under development with no activity shift down a status
>> > level after a month) then maybe we will have something everyone can be
>> > happy with.
>>
>> If an issue is assigned, then somebody has claimed it and is working on
>> it (or it has at least been sent to a specific person for consideration).
>>
>
> Right, but that is for core developers only. I think Tennessee is worrying
> about non-core folks.
>

Absolutely, I don't have any issue with the way the most important issues
are being worked on now, I just think that less-experienced developers could
do a lot more to help out with simple tasks / early-stage tasks. If the rest
of us can help ease the burden by getting issues properly sorted out before
they go to core developers (writing unit tests, sorting out requirements
clearly, documentation, patch suggestions etc) then they won't need to spend
as much time on simple maintenance.

Cheers,
-T



-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090311/834175ae/attachment.htm>

From solipsis at pitrou.net  Wed Mar 11 00:34:59 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 10 Mar 2009 23:34:59 +0000 (UTC)
Subject: [Python-Dev] Ext4 data loss
References: <gp6hhq$hss$1@ger.gmane.org>
	<50862ebd0903101346t216028cgf7f01de7167fbb72@mail.gmail.com>
Message-ID: <loom.20090310T233359-584@post.gmane.org>

Neil Hodgson <nyamatongwe <at> gmail.com> writes:
> 
> What would be useful is a simple, generic
> way in Python to copy all the appropriate metadata (ownership, ACLs,
> ...) to another file so the temporary-and-rename technique could be
> used.

How about shutil.copystat()?



From brett at python.org  Wed Mar 11 01:15:21 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 10 Mar 2009 17:15:21 -0700
Subject: [Python-Dev] Addition of further status options to tracker
In-Reply-To: <43c8685c0903101551q51a7fee8yb5a6af61e5642649@mail.gmail.com>
References: <43c8685c0903092025j14883a1cwa6a182535a6fe8a8@mail.gmail.com>
	<bbaeab100903092039m7dcd73a4j93bb9f3255bf963e@mail.gmail.com>
	<43c8685c0903092100i3457c5b0wbdaae31272a47895@mail.gmail.com>
	<bbaeab100903092123v5aede825wcb80e6f80e63edb7@mail.gmail.com>
	<43c8685c0903092239n79bf013cle7b7e0c4135027d9@mail.gmail.com>
	<bbaeab100903101018q463803b1j3479bc33662feb5a@mail.gmail.com>
	<43c8685c0903101551q51a7fee8yb5a6af61e5642649@mail.gmail.com>
Message-ID: <bbaeab100903101715w1dc1cb3fk16dc101442d16445@mail.gmail.com>

[adding python-dev back on to the email]

On Tue, Mar 10, 2009 at 15:51, Tennessee Leeuwenburg <tleeuwenburg at gmail.com
> wrote:

>
>>> Pretty much. I've got two views. One is that I'd like to search for
>>> issues that are up for grabs which I could take over, hack on, and generally
>>> not get underfoot of core development activity.
>>>
>>
>> OK, let's do what is necessary to flag issues like this so that people can
>> do this. That's why I like the "Under Development" status. Could rename
>> "open" to "available" or "unsolved" to more clearly mark those issues as up
>> for grabs.
>>
>
> Yep. I like that too.
>
>
>>  Typically we use nosy lists to get specific people's attention. That or
>> the priority gets bumped if it turns out to be an important issue. Lastly,
>> people can email python-dev directly asking for input if all other attempts
>> to get attention have failed.
>>
>
> Now that I am understanding the tracker system better, I think it's fine to
> just add reviews to the tracker issue and that will be enough to grab
> attention. There is always the option of emailing the list.
>
>
>>
>> In other words you want some way to flag an issue that just needs to be
>> talked about but is not ready to be coded. So status would go "open/new" ->
>> "chatting/needs help" -> "under dev" -> "closed" with "pending" fitting in
>> when necessary. Sound about right?
>>
>> My worry with this is making sure the field gets updated.
>>
>
> Sounds exactly right. I'm not so concerned about this field being updated.
> If it doesn't, but someone is clearly working on it, then it's not really
> holding anyone back. Tracker janitors (although I do prefer gardeners!) can
> worry about whether the field is set correctly, and developers can just get
> on with doing their work.
>
> I like:
>    "Open/New"
>    "Needs help / Chatting"
>    "Under development"
>    "Pending feedback"
>    "Closed"
>
> very much.
>
>

As long as "Under Dev" and "Pending" are time-based to switch to "chatting"
or "closed" respectively, I am fine with this. What do other people think?
Too heavy-handed? Just right to help people get people involved?

-Brett



> <snip> ...I can understand that, but I would worry that if we flag
>> something as under development it will simply be ignored by other people
>> when they could actually help out (write the docs for someone, etc.). Or
>> even worse that someone gets to a certain point with the development, walks
>> away short of finishing the work (say doesn't get the docs finished) and
>> everyone continues to ignore the issue because it is still flagged as under
>> development.
>
>
>
>> If we can come up with a simple solution to this problem (perhaps have
>> issues set to under development with no activity shift down a status level
>> after a month) then maybe we will have something everyone can be happy with.
>
>
>
> Maybe we can just revert anything that is under development back to "needs
> help" after a month of inactivity?
>
> Cheers,
> -T
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090310/5ffc587f/attachment.htm>

From cs at zip.com.au  Wed Mar 11 01:31:52 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 11 Mar 2009 11:31:52 +1100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <20090310220915.GA15302@amk-desktop.matrixgroup.net>
Message-ID: <20090311003152.GA15370@cskk.homeip.net>

On 10Mar2009 18:09, A.M. Kuchling <amk at amk.ca> wrote:
| On Tue, Mar 10, 2009 at 09:11:38PM +0100, Christian Heimes wrote:
| > Python's file type doesn't use fsync() and be the victim of the very
| > same issue, too. Should we do anything about it?

IMHO, beyond _offering_ an fsync method, no.

| The mailbox module tries to be careful and always fsync() before
| closing files, because mail messages are pretty important.

Can it be turned off? I hadn't realised this.

| The
| various *dbm modules mostly have .sync() method.  
| 
| dumbdbm.py doesn't call fsync(), AFAICT; _commit() writes stuff and
| closes the file, but doesn't call fsync().
| 
| sqlite3 doesn't have a sync() or flush() call.  Does SQLite handle
| this itself?

Yeah, most obnoxiously. There's a longstanding firefox bug about the
horrendous performance side effects of sqlite's zeal in this regard:

  https://bugzilla.mozilla.org/show_bug.cgi?id=421482

At least there's now an (almost undocumented) preference to disable it,
which I do on a personal basis.

| The tarfile, zipfile, and gzip/bzip2 classes don't seem to use fsync()
| at all, either implicitly or by having methods for calling them.
| Should they?  What about cookielib.CookieJar?

I think they should not do this implicitly. By all means let a user
issue policy.

In case you hadn't guessed, I fall into the "never fsync" group,
something of a simplification of my real position. In my opinion,
deciding to fsync is almost always a user policy decision, not an app
decision. An app talks to the OS; if the OS' filesystem has accepted
responsibility for the data (as it has after a successful fflush, for
example) then normally the app should have no further responsibility;
that is _exactly_ what the OS is responsible for.

Recovery is what backups are for, generally speaking.
All this IMHO, of course.

Of course there are some circumstances where one might fsync, as part
of one's risk mitigation policies (eg database checkpointing etc). But
whenever you do this you're basicly saying you don't trust the OS
abstraction of the hardware and also imposing an inherent performance
bottleneck.

With things like ext3 (and ext4 may well be the same - I have not
checked) an fsync doesn't just sync that file data and metadata, it does
a whole-filesystem sync. Really expensive. If underlying libraries do that
quietly and without user oversight/control then this failure to trust the
OS puts an unresolvable bottlneck on various things, and as an app scales
up in I/O or operational throughput or as a library or facility becomes
"higher level" (i.e. _involving_ more and more underlying complexity or
number of basic operations) the more intrusive and unfixable such a low
level "auto-fsync" would become.

Also, how far do you want to go to assure integrity for particular
filesystems' integrity issues/behaviours? Most filesystems sync to disc
regularly (or frequently, at any rate) anyway. What's too big a window
of potential loss?

For myself, I'm against libraries that implicitly do fsyncs, especially
if the user can't issue policy about it.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

If it can't be turned off, it's not a feature. - Karl Heuer

From lists at cheimes.de  Wed Mar 11 02:01:55 2009
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 11 Mar 2009 02:01:55 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org>
	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
Message-ID: <49B70D83.7030104@cheimes.de>

Guido van Rossum wrote:
> If I understand the post properly, it's up to the app to call fsync(),
> and it's only necessary when you're doing one of the rename dances, or
> updating a file in place. Basically, as he explains, fsync() is a very
> heavyweight operation; I'm against calling it by default anywhere.

I agree with you, fsync() shouldn't be called by default. I didn't plan
on adding fsync() calls all over our code. However I like to suggest a
file.sync() method and a synced flag for files to make the job of
application developers easier.

When a file is opened for writing and has the synced flag set, fsync()
is called immediately before the FILE *fp is closed.

Suggested syntax:

    >>> f = open("somefile", "ws")
    >>> f.synced
    True

or:

    >>> f = open(somefile, "w")
    >>> f.synced
    False
    >>> f.synced = True
    >>> f.synced
    True


The sync() method of a file object flushes the internal buffer(fflush()
for Python 2's file object) and fsync() the file descriptor.

Finally the documentation should give the user a hint that close() does
not necessarily mean the data is written to disk. It's not our
responsibility to teach Python users how to deal with low level stuff.
On the other hand a short warning doesn't hurt us and may help Python
users to write better programs.

For the rest I concur with MvL and AMK.

From amk at amk.ca  Wed Mar 11 03:14:51 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Tue, 10 Mar 2009 22:14:51 -0400
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <20090311003152.GA15370@cskk.homeip.net>
References: <20090310220915.GA15302@amk-desktop.matrixgroup.net>
	<20090311003152.GA15370@cskk.homeip.net>
Message-ID: <20090311021451.GA21406@amk.local>

On Wed, Mar 11, 2009 at 11:31:52AM +1100, Cameron Simpson wrote:
> On 10Mar2009 18:09, A.M. Kuchling <amk at amk.ca> wrote:
> | The mailbox module tries to be careful and always fsync() before
> | closing files, because mail messages are pretty important.
> 
> Can it be turned off? I hadn't realised this.

No, there's no way to turn it off (well, you could delete 'fsync' from
the os module).

> | The tarfile, zipfile, and gzip/bzip2 classes don't seem to use fsync()
> | at all, either implicitly or by having methods for calling them.
> | Should they?  What about cookielib.CookieJar?
> 
> I think they should not do this implicitly. By all means let a user
> issue policy.

The problem is that in some cases the user can't issue policy.  For
example, look at dumbdbm._commit().  It renames a file to a backup,
opens a new file object, writes to it, and closes it.  A caller can't
fsync() because the file object is created, used, and closed
internally.  With zipfile, you could at least access the .fp attribute
to sync it (though is the .fp documented as part of the interface?).

In other words, do we need to ensure that all the relevant library
modules expose an interface to allow requesting a sync, or getting the
file descriptor in order to sync it?

--amk

From solipsis at pitrou.net  Wed Mar 11 03:20:39 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 11 Mar 2009 02:20:39 +0000 (UTC)
Subject: [Python-Dev] Ext4 data loss
References: <gp6hhq$hss$1@ger.gmane.org>
	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
	<49B70D83.7030104@cheimes.de>
Message-ID: <loom.20090311T021758-280@post.gmane.org>

Christian Heimes <lists <at> cheimes.de> writes:
> 
> I agree with you, fsync() shouldn't be called by default. I didn't plan
> on adding fsync() calls all over our code. However I like to suggest a
> file.sync() method and a synced flag for files to make the job of
> application developers easier.

We already have os.fsync() and os.fdatasync(). Should the sync() (and
datasync()?) method be added as an object-oriented convenience?




From python at rcn.com  Wed Mar 11 03:29:30 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 10 Mar 2009 19:29:30 -0700
Subject: [Python-Dev] Deprecated __cmp__ and total ordering
References: <ad1f81530903100500w11db61a8m65787d73f3cc5eda@mail.gmail.com><B1A76387858E45638B3C051F98788331@RaymondLaptop1><49B671BB.1030207@voidspace.org.uk>
	<ad1f81530903100958hb585e11u99453f8be7de297b@mail.gmail.com>
Message-ID: <AC20FEE00E6F46D3B222875B62491425@RaymondLaptop1>


[Michael Foord]
> > Is there something you don't like about this one: http://code.activestate.com/recipes/576529/

[Mart S?mermaa ]
> Yes -- it is not in the standard library. As I said, eventually all the 15,000 matches
> on Google Code need to update their code and copy that snippet to their
> util/, write tests for it etc.

FWIW, my version is http://code.activestate.com/recipes/576685/

If you want to push for inclusion in the standard library, I would support
your effort.  The basic idea isn't controversial, but there probably would
be a lengthy discussion on what to call it (total_ordering is one possibilty)
and where to put it (functools is a possibility).


Raymond 


From lists at cheimes.de  Wed Mar 11 03:45:03 2009
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 11 Mar 2009 03:45:03 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090311T021758-280@post.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>
	<loom.20090311T021758-280@post.gmane.org>
Message-ID: <49B725AF.4090705@cheimes.de>

Antoine Pitrou wrote:
> Christian Heimes <lists <at> cheimes.de> writes:
>> I agree with you, fsync() shouldn't be called by default. I didn't plan
>> on adding fsync() calls all over our code. However I like to suggest a
>> file.sync() method and a synced flag for files to make the job of
>> application developers easier.
> 
> We already have os.fsync() and os.fdatasync(). Should the sync() (and
> datasync()?) method be added as an object-oriented convenience?

It's more than an object oriented convenience. fsync() takes a file
descriptor as argument. Therefore I assume fsync() only syncs the data
to disk that was written to the file descriptor. [*] In Python 2.x we
are using a FILE* based stream. In Python 3.x we have our own buffered
writer class.

In order to write all data to disk the FILE* stream must be flushed
first before fsync() is called:

    PyFileObject *f;
    if (fflush(f->f_fp) != 0) {
        /* report error */
    }
    if (fsync(fileno(f->f_fp)) != 0) {
        /* report error */
    }


Christian

[*] Is my assumption correct, anybody?

From guido at python.org  Wed Mar 11 03:55:34 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Mar 2009 19:55:34 -0700
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B725AF.4090705@cheimes.de>
References: <gp6hhq$hss$1@ger.gmane.org>
	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
	<49B70D83.7030104@cheimes.de>
	<loom.20090311T021758-280@post.gmane.org>
	<49B725AF.4090705@cheimes.de>
Message-ID: <ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>

On Tue, Mar 10, 2009 at 7:45 PM, Christian Heimes <lists at cheimes.de> wrote:
> Antoine Pitrou wrote:
>> Christian Heimes <lists <at> cheimes.de> writes:
>>> I agree with you, fsync() shouldn't be called by default. I didn't plan
>>> on adding fsync() calls all over our code. However I like to suggest a
>>> file.sync() method and a synced flag for files to make the job of
>>> application developers easier.
>>
>> We already have os.fsync() and os.fdatasync(). Should the sync() (and
>> datasync()?) method be added as an object-oriented convenience?
>
> It's more than an object oriented convenience. fsync() takes a file
> descriptor as argument. Therefore I assume fsync() only syncs the data
> to disk that was written to the file descriptor. [*] In Python 2.x we
> are using a FILE* based stream. In Python 3.x we have our own buffered
> writer class.
>
> In order to write all data to disk the FILE* stream must be flushed
> first before fsync() is called:
>
> ? ?PyFileObject *f;
> ? ?if (fflush(f->f_fp) != 0) {
> ? ? ? ?/* report error */
> ? ?}
> ? ?if (fsync(fileno(f->f_fp)) != 0) {
> ? ? ? ?/* report error */
> ? ?}

Let's not think too Unix-specific. If we add such an API it should do
something on Windows too -- the app shouldn't have to test for the
presence of the API. (And thus the API probably shouldn't be called
fsync.)

> Christian
>
> [*] Is my assumption correct, anybody?

It seems to be, at least it's ambiguous.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From cs at zip.com.au  Wed Mar 11 03:59:00 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 11 Mar 2009 13:59:00 +1100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <20090311021451.GA21406@amk.local>
Message-ID: <20090311025900.GA13613@cskk.homeip.net>

On 10Mar2009 22:14, A.M. Kuchling <amk at amk.ca> wrote:
| On Wed, Mar 11, 2009 at 11:31:52AM +1100, Cameron Simpson wrote:
| > On 10Mar2009 18:09, A.M. Kuchling <amk at amk.ca> wrote:
| > | The mailbox module tries to be careful and always fsync() before
| > | closing files, because mail messages are pretty important.
| > 
| > Can it be turned off? I hadn't realised this.
| 
| No, there's no way to turn it off (well, you could delete 'fsync' from
| the os module).

Ah. For myself, were I writing a high load mailbox tool (eg a mail filer
or more to the point, a mail refiler - which I do actually intend to) I
would want to be able to do a huge mass of mailbox stuff and then
possibly issue a sync at the end. For "unix mbox" that might be ok but
for maildirs I'd imagine it leads to an fsync per message.

| > | The tarfile, zipfile, and gzip/bzip2 classes don't seem to use fsync()
| > | at all, either implicitly or by having methods for calling them.
| > | Should they?  What about cookielib.CookieJar?
| > 
| > I think they should not do this implicitly. By all means let a user
| > issue policy.
| 
| The problem is that in some cases the user can't issue policy.  For
| example, look at dumbdbm._commit().  It renames a file to a backup,
| opens a new file object, writes to it, and closes it.  A caller can't
| fsync() because the file object is created, used, and closed
| internally.  With zipfile, you could at least access the .fp attribute
| to sync it (though is the .fp documented as part of the interface?).

I didn't so much mean giving the user an fsync hook so much as publishing a
flag such as ".do_critical_fsyncs" inside the dbm or zipfile object. If true,
issue fsyncs at appropriate times.

| In other words, do we need to ensure that all the relevant library
| modules expose an interface to allow requesting a sync, or getting the
| file descriptor in order to sync it?

With a policy flag you could solve the control issue even for things
which don't expose the fd such as your dumbdbm._commit() example.
If you supply both a flag and an fsync() method it becomes easy for
a user of a module to go:

  obj = get_dbm_handle(....)
  obj.do_critical_fsyncs = False
  ... do lots and lots of stuff ...
  obj.fsync()
  obj.close()

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

In the end, winning is the only safety. - Kerr Avon

From cs at zip.com.au  Wed Mar 11 04:02:12 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Wed, 11 Mar 2009 14:02:12 +1100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090311T021758-280@post.gmane.org>
Message-ID: <20090311030212.GA15763@cskk.homeip.net>

On 11Mar2009 02:20, Antoine Pitrou <solipsis at pitrou.net> wrote:
| Christian Heimes <lists <at> cheimes.de> writes:
| > I agree with you, fsync() shouldn't be called by default. I didn't plan
| > on adding fsync() calls all over our code. However I like to suggest a
| > file.sync() method and a synced flag for files to make the job of
| > application developers easier.
| 
| We already have os.fsync() and os.fdatasync(). Should the sync() (and
| datasync()?) method be added as an object-oriented convenience?

I can imagine plenty of occasions when there may not be an available
file descriptor to hand to os.fsync() et al. Having sync() and
datasync() methods in the object would obviate the need for the caller
to know the object internals.
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

I must construct my own System, or be enslaved to another Man's.
        - William Blake

From Scott.Daniels at Acm.Org  Wed Mar 11 08:17:55 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Wed, 11 Mar 2009 00:17:55 -0700
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <20090311021451.GA21406@amk.local>
References: <20090310220915.GA15302@amk-desktop.matrixgroup.net>	<20090311003152.GA15370@cskk.homeip.net>
	<20090311021451.GA21406@amk.local>
Message-ID: <gp7odc$inp$1@ger.gmane.org>

A.M. Kuchling wrote:
> ....  With zipfile, you could at least access the .fp attribute
> to sync it (though is the .fp documented as part of the interface?).

For this one, I'd like to add the sync as a method (so that Zip-inside-
Zip is eventually possible).  In fact, a sync on an exposed writable
for a single file should probably push back out to a full sync.  This
would be trickier to accomplish if the using code had to suss out how
to get to the fp.  Clearly I have plans for a ZipFile expansion, but
this could only conceivably hit 2.7, and 2.8 / 3.2 is a lot more likely.

--Scott David Daniels
Scott.Daniels at Acm.Org


From martin at v.loewis.de  Wed Mar 11 09:47:36 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 11 Mar 2009 09:47:36 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B725AF.4090705@cheimes.de>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>
	<49B725AF.4090705@cheimes.de>
Message-ID: <49B77AA8.4090808@v.loewis.de>

>> We already have os.fsync() and os.fdatasync(). Should the sync() (and
>> datasync()?) method be added as an object-oriented convenience?
> 
> It's more than an object oriented convenience. fsync() takes a file
> descriptor as argument. Therefore I assume fsync() only syncs the data
> to disk that was written to the file descriptor. [*] 
[...]
> [*] Is my assumption correct, anybody?

Not necessarily. In Linux, for many releases, fsync() was really
equivalent to sync() (i.e. flushing all data for all files on all
file systems to disk). It may be that some systems still implement
it that way today.

However, even it it was true, I don't see why a .sync method would
be more than a convenience. An application wishing to sync a file
before close can do

    f.flush()
    os.fsync(f.fileno)
    f.close()

With a sync method, it would become

    f.flush()
    f.sync()
    f.close()

which is *really* nothing more than convenience.

O'd also like to point to the O_SYNC/O_DSYNC/O_RSYNC open(2)
flags. Applications that require durable writes can also chose
to set those on open, and be done.

Regrds,
Martin

From him at online.de  Wed Mar 11 10:09:43 2009
From: him at online.de (=?ISO-8859-1?Q?Joachim_K=F6nig?=)
Date: Wed, 11 Mar 2009 10:09:43 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org>
	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
Message-ID: <49B77FD7.8030807@online.de>

Guido van Rossum wrote:
> On Tue, Mar 10, 2009 at 1:11 PM, Christian Heimes <lists at cheimes.de> wrote:
>   
>> [...]
>> https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/317781/comments/54.
>> [...]
>>     
> If I understand the post properly, it's up to the app to call fsync(),
> and it's only necessary when you're doing one of the rename dances, or
> updating a file in place. Basically, as he explains, fsync() is a very
> heavyweight operation; I'm against calling it by default anywhere.
>
>   
To me, the flaw seem to be in the close() call (of the operating 
system). I'd expect the data to be
in a persistent state once the close() returns. So there would be no 
need to fsync if the file gets closed anyway.

Of course the close() call could take a while (up to 30 seconds in 
laptop mode), but if one does
not want to wait that long, than one can continue without calling 
close() and take the risk.

Of course, if the data should be on a persistant storage without closing 
the file (e.g. for database
applications), than one has to carefully call the different sync 
methods, but that's an other story.

Why has this ext4 problem not come up for other filesystems?




From nyamatongwe at gmail.com  Wed Mar 11 10:14:51 2009
From: nyamatongwe at gmail.com (Neil Hodgson)
Date: Wed, 11 Mar 2009 20:14:51 +1100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090310T233359-584@post.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>
	<50862ebd0903101346t216028cgf7f01de7167fbb72@mail.gmail.com>
	<loom.20090310T233359-584@post.gmane.org>
Message-ID: <50862ebd0903110214q54b8b7b4y846982c1a1e83a27@mail.gmail.com>

Antoine Pitrou:

> How about shutil.copystat()?

   shutil.copystat does not copy over the owner, group or ACLs.

   Modeling a copymetadata method on copystat would provide an easy to
understand API and should be implementable on Windows and POSIX.
Reading the OS X documentation shows a set of low-level POSIX
functions for ACLs. Since there are multiple pieces of metadata and an
application may not want to copy all pieces there could be multiple
methods (copygroup ...) or one method with options
shutil.copymetadata(src, dst, group=True, resource_fork=False)

   Neil

From hrvoje.niksic at avl.com  Wed Mar 11 10:55:29 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Wed, 11 Mar 2009 10:55:29 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <15090177.2970205.1236762890023.JavaMail.xicrypt@atgrzls001>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
	<15090177.2970205.1236762890023.JavaMail.xicrypt@atgrzls001>
Message-ID: <49B78A91.8080602@avl.com>

Joachim K?nig wrote:
> To me, the flaw seem to be in the close() call (of the operating 
> system). I'd expect the data to be
> in a persistent state once the close() returns.

I wouldn't, because that would mean that every cp -r would effectively 
do an fsync() for each individual file it copies, which would bog down 
in the case of copying many small files.  Operating systems aggressively 
buffer file systems for good reason: performance of the common case.

> Why has this ext4 problem not come up for other filesystems?

It has come up for XFS many many times, for example 
https://launchpad.net/ubuntu/+bug/37435

ext3 was resillient to the problem because of its default allocation 
policy; now that ext4 has implemented the same optimization XFS had 
before, it shares the problems.

From solipsis at pitrou.net  Wed Mar 11 12:43:33 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 11 Mar 2009 11:43:33 +0000 (UTC)
Subject: [Python-Dev] Ext4 data loss
References: <gp6hhq$hss$1@ger.gmane.org>
	<50862ebd0903101346t216028cgf7f01de7167fbb72@mail.gmail.com>
	<loom.20090310T233359-584@post.gmane.org>
	<50862ebd0903110214q54b8b7b4y846982c1a1e83a27@mail.gmail.com>
Message-ID: <loom.20090311T114053-723@post.gmane.org>

Neil Hodgson <nyamatongwe <at> gmail.com> writes:
> 
>    shutil.copystat does not copy over the owner, group or ACLs.

It depends on what you call "ACLs". It does copy the chmod permission bits.
As for owner and group, I think there is a very good reason that it doesn't copy
them: under Linux, only root can change these properties.




From phd at phd.pp.ru  Wed Mar 11 12:46:52 2009
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Wed, 11 Mar 2009 14:46:52 +0300
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090311T114053-723@post.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>
	<50862ebd0903101346t216028cgf7f01de7167fbb72@mail.gmail.com>
	<loom.20090310T233359-584@post.gmane.org>
	<50862ebd0903110214q54b8b7b4y846982c1a1e83a27@mail.gmail.com>
	<loom.20090311T114053-723@post.gmane.org>
Message-ID: <20090311114652.GB27554@phd.pp.ru>

On Wed, Mar 11, 2009 at 11:43:33AM +0000, Antoine Pitrou wrote:
> As for owner and group, I think there is a very good reason that it doesn't copy
> them: under Linux, only root can change these properties.

   Only root can change file ownership - and yes, there are scripts that
run with root privileges, so why not copy? As for group ownership - any
user can change group if [s]he belongs to the group.

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From solipsis at pitrou.net  Wed Mar 11 12:47:05 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 11 Mar 2009 11:47:05 +0000 (UTC)
Subject: [Python-Dev] Ext4 data loss
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>
	<loom.20090311T021758-280@post.gmane.org>
	<49B725AF.4090705@cheimes.de>
Message-ID: <loom.20090311T114541-270@post.gmane.org>

Christian Heimes <lists <at> cheimes.de> writes:
> 
> It's more than an object oriented convenience. fsync() takes a file
> descriptor as argument. Therefore I assume fsync() only syncs the data
> to disk that was written to the file descriptor.

Ok, I agree that a .sync() method makes sense.



From solipsis at pitrou.net  Wed Mar 11 12:56:13 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 11 Mar 2009 11:56:13 +0000 (UTC)
Subject: [Python-Dev] Ext4 data loss
References: <gp6hhq$hss$1@ger.gmane.org>
	<50862ebd0903101346t216028cgf7f01de7167fbb72@mail.gmail.com>
	<loom.20090310T233359-584@post.gmane.org>
	<50862ebd0903110214q54b8b7b4y846982c1a1e83a27@mail.gmail.com>
	<loom.20090311T114053-723@post.gmane.org>
	<20090311114652.GB27554@phd.pp.ru>
Message-ID: <loom.20090311T115512-392@post.gmane.org>

Oleg Broytmann <phd <at> phd.pp.ru> writes:
> 
>    Only root can change file ownership - and yes, there are scripts that
> run with root privileges, so why not copy?

Because the new function would then be useless for non-root scripts, and
encouraging people to run their scripts as root would be rather bad.



From phd at phd.pp.ru  Wed Mar 11 13:09:38 2009
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Wed, 11 Mar 2009 15:09:38 +0300
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090311T115512-392@post.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>
	<50862ebd0903101346t216028cgf7f01de7167fbb72@mail.gmail.com>
	<loom.20090310T233359-584@post.gmane.org>
	<50862ebd0903110214q54b8b7b4y846982c1a1e83a27@mail.gmail.com>
	<loom.20090311T114053-723@post.gmane.org>
	<20090311114652.GB27554@phd.pp.ru>
	<loom.20090311T115512-392@post.gmane.org>
Message-ID: <20090311120938.GC27554@phd.pp.ru>

On Wed, Mar 11, 2009 at 11:56:13AM +0000, Antoine Pitrou wrote:
> Oleg Broytmann <phd <at> phd.pp.ru> writes:
> >    Only root can change file ownership - and yes, there are scripts that
> > run with root privileges, so why not copy?
> 
> Because the new function would then be useless for non-root scripts

   That's easy to fix - only copy ownership if the effective user id == 0.

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From skippy.hammond at gmail.com  Wed Mar 11 13:12:50 2009
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Wed, 11 Mar 2009 23:12:50 +1100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>
	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
Message-ID: <49B7AAC2.6040504@gmail.com>

On 11/03/2009 1:55 PM, Guido van Rossum wrote:
> On Tue, Mar 10, 2009 at 7:45 PM, Christian Heimes<lists at cheimes.de>  wrote:
>> Antoine Pitrou wrote:
>>> Christian Heimes<lists<at>  cheimes.de>  writes:
...
> Let's not think too Unix-specific. If we add such an API it should do
> something on Windows too -- the app shouldn't have to test for the
> presence of the API. (And thus the API probably shouldn't be called
> fsync.)

This is especially true given Windows has recently introduced a 
transactional API for NTFS.  Although the tone is - err - gushing - it 
(a) should give some information on what is available, and (b) was high 
on my google search list <wink>

http://msdn.microsoft.com/en-us/magazine/cc163388.aspx
http://msdn.microsoft.com/en-us/library/aa363764(VS.85).aspx

Cheers,

Mark

From solipsis at pitrou.net  Wed Mar 11 13:25:08 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 11 Mar 2009 12:25:08 +0000 (UTC)
Subject: [Python-Dev] Ext4 data loss
References: <gp6hhq$hss$1@ger.gmane.org>
	<50862ebd0903101346t216028cgf7f01de7167fbb72@mail.gmail.com>
	<loom.20090310T233359-584@post.gmane.org>
	<50862ebd0903110214q54b8b7b4y846982c1a1e83a27@mail.gmail.com>
	<loom.20090311T114053-723@post.gmane.org>
	<20090311114652.GB27554@phd.pp.ru>
	<loom.20090311T115512-392@post.gmane.org>
	<20090311120938.GC27554@phd.pp.ru>
Message-ID: <loom.20090311T122237-548@post.gmane.org>

Oleg Broytmann <phd <at> phd.pp.ru> writes:
> 
>    That's easy to fix - only copy ownership if the effective user id == 0.

But errors should not pass silently. If the user intended the function to copy
ownership information and the function fails to do so, it should raise an 
exception.
Having implicit special cases in an API is usually bad, especially when it has
an impact on security.



From lists at cheimes.de  Wed Mar 11 15:12:10 2009
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 11 Mar 2009 15:12:10 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>
	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
Message-ID: <49B7C6BA.8010100@cheimes.de>

Guido van Rossum wrote:
> Let's not think too Unix-specific. If we add such an API it should do
> something on Windows too -- the app shouldn't have to test for the
> presence of the API. (And thus the API probably shouldn't be called
> fsync.)

In my initial proposal one and a half hour earlier I suggested 'sync()'
as the name of the method and 'synced' as the name of the flag that
forces a fsync() call during the close operation.

Christian

From hrvoje.niksic at avl.com  Wed Mar 11 15:14:49 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Wed, 11 Mar 2009 15:14:49 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <31764696.2987478.1236780758847.JavaMail.xicrypt@atgrzls001>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
	<31764696.2987478.1236780758847.JavaMail.xicrypt@atgrzls001>
Message-ID: <49B7C759.1080106@avl.com>

Christian Heimes wrote:
> Guido van Rossum wrote:
>> Let's not think too Unix-specific. If we add such an API it should do
>> something on Windows too -- the app shouldn't have to test for the
>> presence of the API. (And thus the API probably shouldn't be called
>> fsync.)
> 
> In my initial proposal one and a half hour earlier I suggested 'sync()'
> as the name of the method and 'synced' as the name of the flag that
> forces a fsync() call during the close operation.

Maybe it would make more sense for "synced" to force fsync() on each 
flush, not only on close.  I'm not sure how useful it is, but that's 
what "synced" would imply to me.  Maybe it would be best to avoid having 
such a variable, and expose a close_sync() method instead?

From solipsis at pitrou.net  Wed Mar 11 15:21:25 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 11 Mar 2009 14:21:25 +0000 (UTC)
Subject: [Python-Dev] Ext4 data loss
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>
	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
	<49B7C6BA.8010100@cheimes.de>
Message-ID: <loom.20090311T141849-464@post.gmane.org>

Christian Heimes <lists <at> cheimes.de> writes:
> 
> In my initial proposal one and a half hour earlier I suggested 'sync()'
> as the name of the method and 'synced' as the name of the flag that
> forces a fsync() call during the close operation.

I think your "synced" flag is too vague. Some applications may need the file to
be synced on close(), but some others may need it to be synced at regular
intervals, or after each write(), etc.

Calling the flag "sync_on_close" would be much more explicit. Also, given the
current API I think it should be an argument to open() rather than a writable
attribute.



From solipsis at pitrou.net  Wed Mar 11 15:39:21 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 11 Mar 2009 14:39:21 +0000 (UTC)
Subject: [Python-Dev] Ext4 data loss
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>
	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
	<49B7C6BA.8010100@cheimes.de>
	<loom.20090311T141849-464@post.gmane.org>
Message-ID: <loom.20090311T143816-333@post.gmane.org>

After Hrvoje's message, let me rephrase my suggestion. Let's instead allow:
   open(..., sync_on="close")
   open(..., sync_on="flush")

with a default of None meaning no implicit syncs.

Regards

Antoine.



From lists at cheimes.de  Wed Mar 11 15:58:49 2009
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 11 Mar 2009 15:58:49 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090311T143816-333@post.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>	<49B7C6BA.8010100@cheimes.de>	<loom.20090311T141849-464@post.gmane.org>
	<loom.20090311T143816-333@post.gmane.org>
Message-ID: <49B7D1A9.7040309@cheimes.de>

Antoine Pitrou schrieb:
> After Hrvoje's message, let me rephrase my suggestion. Let's instead allow:
>    open(..., sync_on="close")
>    open(..., sync_on="flush")
> 
> with a default of None meaning no implicit syncs.

And sync_on="flush" implies sync_on="close"? Your suggestion sounds like
the right way to me!

Christian

From solipsis at pitrou.net  Wed Mar 11 16:07:24 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 11 Mar 2009 15:07:24 +0000 (UTC)
Subject: [Python-Dev] Ext4 data loss
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>	<49B7C6BA.8010100@cheimes.de>	<loom.20090311T141849-464@post.gmane.org>
	<loom.20090311T143816-333@post.gmane.org>
	<49B7D1A9.7040309@cheimes.de>
Message-ID: <loom.20090311T150550-377@post.gmane.org>

Christian Heimes <lists <at> cheimes.de> writes:
> 
> And sync_on="flush" implies sync_on="close"?

close() implies flush(), so by construction yes.

> Your suggestion sounds like
> the right way to me!

I'm glad I brought something constructive to the discussion :-))



From aahz at pythoncraft.com  Wed Mar 11 16:34:34 2009
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 11 Mar 2009 08:34:34 -0700
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090311T143816-333@post.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>
	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
	<49B70D83.7030104@cheimes.de>
	<loom.20090311T021758-280@post.gmane.org>
	<49B725AF.4090705@cheimes.de>
	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
	<49B7C6BA.8010100@cheimes.de>
	<loom.20090311T141849-464@post.gmane.org>
	<loom.20090311T143816-333@post.gmane.org>
Message-ID: <20090311153434.GA18918@panix.com>

On Wed, Mar 11, 2009, Antoine Pitrou wrote:
>
> After Hrvoje's message, let me rephrase my suggestion. Let's instead allow:
>    open(..., sync_on="close")
>    open(..., sync_on="flush")
> 
> with a default of None meaning no implicit syncs.

That looks good, though I'd prefer using named constants rather than
strings.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of     
indirection."  --Butler Lampson

From scott+python-dev at scottdial.com  Wed Mar 11 16:38:26 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 11 Mar 2009 11:38:26 -0400
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <20090311153434.GA18918@panix.com>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>	<49B7C6BA.8010100@cheimes.de>	<loom.20090311T141849-464@post.gmane.org>	<loom.20090311T143816-333@post.gmane.org>
	<20090311153434.GA18918@panix.com>
Message-ID: <49B7DAF2.8040403@scottdial.com>

Aahz wrote:
> On Wed, Mar 11, 2009, Antoine Pitrou wrote:
>> After Hrvoje's message, let me rephrase my suggestion. Let's instead allow:
>>    open(..., sync_on="close")
>>    open(..., sync_on="flush")
>>
>> with a default of None meaning no implicit syncs.
> 
> That looks good, though I'd prefer using named constants rather than
> strings.

I would agree, but where do you put them? Since open is a built-in,
where would you suggest placing such constants (assuming we don't want
to pollute the built-in namespace)?

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From aahz at pythoncraft.com  Wed Mar 11 17:04:41 2009
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 11 Mar 2009 09:04:41 -0700
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B7DAF2.8040403@scottdial.com>
References: <ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
	<49B70D83.7030104@cheimes.de>
	<loom.20090311T021758-280@post.gmane.org>
	<49B725AF.4090705@cheimes.de>
	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
	<49B7C6BA.8010100@cheimes.de>
	<loom.20090311T141849-464@post.gmane.org>
	<loom.20090311T143816-333@post.gmane.org>
	<20090311153434.GA18918@panix.com> <49B7DAF2.8040403@scottdial.com>
Message-ID: <20090311160441.GA28165@panix.com>

On Wed, Mar 11, 2009, Scott Dial wrote:
> Aahz wrote:
>> On Wed, Mar 11, 2009, Antoine Pitrou wrote:
>>> After Hrvoje's message, let me rephrase my suggestion. Let's instead allow:
>>>    open(..., sync_on="close")
>>>    open(..., sync_on="flush")
>>>
>>> with a default of None meaning no implicit syncs.
>> 
>> That looks good, though I'd prefer using named constants rather than
>> strings.
> 
> I would agree, but where do you put them? Since open is a built-in,
> where would you suggest placing such constants (assuming we don't want
> to pollute the built-in namespace)?

The os module, of course, like the existing O_* constants.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of     
indirection."  --Butler Lampson

From eric at trueblade.com  Wed Mar 11 17:05:32 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 11 Mar 2009 12:05:32 -0400
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090311T141849-464@post.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>	<49B7C6BA.8010100@cheimes.de>
	<loom.20090311T141849-464@post.gmane.org>
Message-ID: <49B7E14C.5050809@trueblade.com>

Antoine Pitrou wrote:
> I think your "synced" flag is too vague. Some applications may need the file to
> be synced on close(), but some others may need it to be synced at regular
> intervals, or after each write(), etc.

Why wouldn't sync just be an optional argument to close(), at least for 
the "sync_on_close" case?

Eric.

From solipsis at pitrou.net  Wed Mar 11 17:20:25 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 11 Mar 2009 16:20:25 +0000 (UTC)
Subject: [Python-Dev] Ext4 data loss
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>	<49B7C6BA.8010100@cheimes.de>
	<loom.20090311T141849-464@post.gmane.org>
	<49B7E14C.5050809@trueblade.com>
Message-ID: <loom.20090311T161952-397@post.gmane.org>

Eric Smith <eric <at> trueblade.com> writes:
> 
> Why wouldn't sync just be an optional argument to close(), at least for 
> the "sync_on_close" case?

It wouldn't work with the "with" statement.



From lie.1296 at gmail.com  Wed Mar 11 17:20:30 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Thu, 12 Mar 2009 03:20:30 +1100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B7DAF2.8040403@scottdial.com>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>	<49B7C6BA.8010100@cheimes.de>	<loom.20090311T141849-464@post.gmane.org>	<loom.20090311T143816-333@post.gmane.org>	<20090311153434.GA18918@panix.com>
	<49B7DAF2.8040403@scottdial.com>
Message-ID: <gp8ocf$sra$1@ger.gmane.org>

Scott Dial wrote:
> Aahz wrote:
>> On Wed, Mar 11, 2009, Antoine Pitrou wrote:
>>> After Hrvoje's message, let me rephrase my suggestion. Let's instead allow:
>>>    open(..., sync_on="close")
>>>    open(..., sync_on="flush")
>>>
>>> with a default of None meaning no implicit syncs.
>> That looks good, though I'd prefer using named constants rather than
>> strings.
> 
> I would agree, but where do you put them? Since open is a built-in,
> where would you suggest placing such constants (assuming we don't want
> to pollute the built-in namespace)?
> 

I actually prefer strings. Just like 'w' or 'r' in open().

Or why not add "f" "c" as modes?

open('file.txt', 'wf')

open for writing, sync on flush


From eric at trueblade.com  Wed Mar 11 18:05:54 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 11 Mar 2009 13:05:54 -0400
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090311T161952-397@post.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>	<49B7C6BA.8010100@cheimes.de>	<loom.20090311T141849-464@post.gmane.org>	<49B7E14C.5050809@trueblade.com>
	<loom.20090311T161952-397@post.gmane.org>
Message-ID: <49B7EF72.5080602@trueblade.com>

Antoine Pitrou wrote:
> Eric Smith <eric <at> trueblade.com> writes:
>> Why wouldn't sync just be an optional argument to close(), at least for 
>> the "sync_on_close" case?
> 
> It wouldn't work with the "with" statement.


Well, that is a good reason, then!

From martin at v.loewis.de  Wed Mar 11 19:19:28 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 11 Mar 2009 19:19:28 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B7AAC2.6040504@gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
	<49B7AAC2.6040504@gmail.com>
Message-ID: <49B800B0.9010304@v.loewis.de>

> This is especially true given Windows has recently introduced a
> transactional API for NTFS.  Although the tone is - err - gushing - it
> (a) should give some information on what is available, and (b) was high
> on my google search list <wink>
> 
> http://msdn.microsoft.com/en-us/magazine/cc163388.aspx
> http://msdn.microsoft.com/en-us/library/aa363764(VS.85).aspx

Of course, we don't have to go to transactional NTFS to find an
equivalent to fsync: applications can call FlushFileBuffers.
Likewise, if applications want every write call to be synchronized,
they can pass FILE_FLAG_WRITE_THROUGH to CreateFile (similar to
what O_SYNC does on POSIX).

Regards,
Martin

From martin at v.loewis.de  Wed Mar 11 19:20:50 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 11 Mar 2009 19:20:50 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B7C759.1080106@avl.com>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>	<31764696.2987478.1236780758847.JavaMail.xicrypt@atgrzls001>
	<49B7C759.1080106@avl.com>
Message-ID: <49B80102.6010706@v.loewis.de>

> Maybe it would make more sense for "synced" to force fsync() on each
> flush, not only on close.  I'm not sure how useful it is, but that's
> what "synced" would imply to me.

That should be implement by passing O_SYNC on open, rather than
explicitly calling fsync.

Regards,
Martin

From eric at trueblade.com  Wed Mar 11 20:23:56 2009
From: eric at trueblade.com (Eric Smith)
Date: Wed, 11 Mar 2009 15:23:56 -0400
Subject: [Python-Dev] Can I modify string.Formatter._vformat?
Message-ID: <49B80FCC.2020307@trueblade.com>

I'm implementing support for auto-numbering of str.format() fields (see 
http://bugs.python.org/issue5237). I'm reasonably sure that when I'm 
done modifying the C implementation I'll need to change the signatures 
of string.Formatter._vformat, str._formatter_parser, and/or 
str._formatter_field_name_split. (They need to support the state needed 
to track the auto-number field counter.)

I've always considered these internal implementation details of 
str.format and string.Formatter. They begin with underscores and are not 
documented.

Is there any problem with modifying these in 2.7 and 3.1? I assume not, 
but I want to make sure it doesn't give anyone heartburn.

Eric.

From benjamin at python.org  Wed Mar 11 21:20:17 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 11 Mar 2009 15:20:17 -0500
Subject: [Python-Dev] Can I modify string.Formatter._vformat?
In-Reply-To: <49B80FCC.2020307@trueblade.com>
References: <49B80FCC.2020307@trueblade.com>
Message-ID: <1afaf6160903111320i32859e08r8533138a305cfd8a@mail.gmail.com>

2009/3/11 Eric Smith <eric at trueblade.com>:
> I'm implementing support for auto-numbering of str.format() fields (see
> http://bugs.python.org/issue5237). I'm reasonably sure that when I'm done
> modifying the C implementation I'll need to change the signatures of
> string.Formatter._vformat, str._formatter_parser, and/or
> str._formatter_field_name_split. (They need to support the state needed to
> track the auto-number field counter.)
>
> I've always considered these internal implementation details of str.format
> and string.Formatter. They begin with underscores and are not documented.
>
> Is there any problem with modifying these in 2.7 and 3.1? I assume not, but
> I want to make sure it doesn't give anyone heartburn.

Certainly sounds fine with me.


-- 
Regards,
Benjamin

From brett at python.org  Wed Mar 11 21:49:15 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 11 Mar 2009 13:49:15 -0700
Subject: [Python-Dev] Can I modify string.Formatter._vformat?
In-Reply-To: <1afaf6160903111320i32859e08r8533138a305cfd8a@mail.gmail.com>
References: <49B80FCC.2020307@trueblade.com>
	<1afaf6160903111320i32859e08r8533138a305cfd8a@mail.gmail.com>
Message-ID: <bbaeab100903111349n1dc9f942pdc976e9c00d5cbbe@mail.gmail.com>

On Wed, Mar 11, 2009 at 13:20, Benjamin Peterson <benjamin at python.org>wrote:

> 2009/3/11 Eric Smith <eric at trueblade.com>:
> > I'm implementing support for auto-numbering of str.format() fields (see
> > http://bugs.python.org/issue5237). I'm reasonably sure that when I'm
> done
> > modifying the C implementation I'll need to change the signatures of
> > string.Formatter._vformat, str._formatter_parser, and/or
> > str._formatter_field_name_split. (They need to support the state needed
> to
> > track the auto-number field counter.)
> >
> > I've always considered these internal implementation details of
> str.format
> > and string.Formatter. They begin with underscores and are not documented.
> >
> > Is there any problem with modifying these in 2.7 and 3.1? I assume not,
> but
> > I want to make sure it doesn't give anyone heartburn.
>
> Certainly sounds fine with me.


Even though the Great Release Manager of 3.1 said it was fine, I will toss
in my support with it being okay to modify them.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090311/ca64ee5a/attachment-0001.htm>

From nyamatongwe at gmail.com  Wed Mar 11 22:11:08 2009
From: nyamatongwe at gmail.com (Neil Hodgson)
Date: Thu, 12 Mar 2009 08:11:08 +1100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090311T114053-723@post.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>
	<50862ebd0903101346t216028cgf7f01de7167fbb72@mail.gmail.com>
	<loom.20090310T233359-584@post.gmane.org>
	<50862ebd0903110214q54b8b7b4y846982c1a1e83a27@mail.gmail.com>
	<loom.20090311T114053-723@post.gmane.org>
Message-ID: <50862ebd0903111411s187271ccu676c5d2a5aee69e9@mail.gmail.com>

Antoine Pitrou:

> It depends on what you call "ACLs". It does copy the chmod permission bits.

    Access Control Lists are fine grained permissions. Perhaps you
want to allow Sam to read a file and for Ted to both read and write
it. These permissions should not need to be reset every time you
modify the file.

> As for owner and group, I think there is a very good reason that it doesn't copy
> them: under Linux, only root can change these properties.

   Since I am a member of both "staff" and "everyone", I can set group
on one of my files from "staff" to "everyone" or back again:

$ chown :everyone x.pl
$ ls -la x.pl
-rwxrwxrwx  1 nyamatongwe  everyone  269 Mar 11  2008 x.pl
$ chown :staff x.pl
$ ls -la x.pl
-rwxrwxrwx  1 nyamatongwe  staff  269 Mar 11  2008 x.pl

   Neil

From greg.ewing at canterbury.ac.nz  Wed Mar 11 22:30:30 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 12 Mar 2009 10:30:30 +1300
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <BAF87B91-3102-4374-8A70-021FFD77A058@python.org>
References: <gp6hhq$hss$1@ger.gmane.org>
	<50862ebd0903101346t216028cgf7f01de7167fbb72@mail.gmail.com>
	<BAF87B91-3102-4374-8A70-021FFD77A058@python.org>
Message-ID: <49B82D76.2060900@canterbury.ac.nz>

Barry Warsaw wrote:

> Of course, a careful *nix application can ensure that the file owners  
> and mod bits are set the way it needs them to be set.  A convenience  
> function might be useful though.

A specialised function would also provide a place for
dealing with platform-specific extensions, such as
MacOSX Finder attributes.

-- 
Greg

From cs at zip.com.au  Wed Mar 11 22:43:44 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Thu, 12 Mar 2009 08:43:44 +1100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B77FD7.8030807@online.de>
Message-ID: <20090311214344.GA28718@cskk.homeip.net>

On 11Mar2009 10:09, Joachim K?nig <him at online.de> wrote:
> Guido van Rossum wrote:
>> On Tue, Mar 10, 2009 at 1:11 PM, Christian Heimes <lists at cheimes.de> wrote:
>>> [...]
>>> https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/317781/comments/54.
>>> [...]
>> If I understand the post properly, it's up to the app to call fsync(),
>> and it's only necessary when you're doing one of the rename dances, or
>> updating a file in place. Basically, as he explains, fsync() is a very
>> heavyweight operation; I'm against calling it by default anywhere.
>>   
> To me, the flaw seem to be in the close() call (of the operating  
> system). I'd expect the data to be
> in a persistent state once the close() returns. So there would be no  
> need to fsync if the file gets closed anyway.

Not really. On the whole, flush() means "the object has handed all data
to the OS".  close() means "the object has handed all data to the OS
and released the control data structures" (OS file descriptor release;
like the OS, the python interpreter may release python stuff later too).

By contrast, fsync() means "the OS has handed filesystem changes to the
disc itself". Really really slow, by comparison with memory. It is Very
Expensive, and a very different operation to close().

[...snip...]
> Why has this ext4 problem not come up for other filesystems?

The same problems exist for all disc based filesystems to a greater of
lesser degree; the OS always does some buffering and therefore there
is a gap between what the OS has accepted from you (and thus made
visible to other apps using the OS) and the physical data structures
on disc. Ext2/3/4 tend to do whole disc sync when just asked to fsync,
probably because it really is only feasible to say "get to a particular
checkpoint in the journal". Many other filesystems will have similar
degrees of granularity, perhaps not all.

Anyway, fsync is a much bigger ask than close, and should be used very
sparingly.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

If I repent anything, it is very likely to be my good behavior.
    What demon possessed me that I behaved so well?     - Henry David Thoreau

From steve at pearwood.info  Wed Mar 11 23:48:27 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 12 Mar 2009 09:48:27 +1100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090311T141849-464@post.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org> <49B7C6BA.8010100@cheimes.de>
	<loom.20090311T141849-464@post.gmane.org>
Message-ID: <200903120948.28250.steve@pearwood.info>

On Thu, 12 Mar 2009 01:21:25 am Antoine Pitrou wrote:
> Christian Heimes <lists <at> cheimes.de> writes:
> > In my initial proposal one and a half hour earlier I suggested
> > 'sync()' as the name of the method and 'synced' as the name of the
> > flag that forces a fsync() call during the close operation.
>
> I think your "synced" flag is too vague. Some applications may need
> the file to be synced on close(), but some others may need it to be
> synced at regular intervals, or after each write(), etc.
>
> Calling the flag "sync_on_close" would be much more explicit. Also,
> given the current API I think it should be an argument to open()
> rather than a writable attribute.

Perhaps we should have a module containing rich file tools, e.g. classes 
FileSyncOnWrite, FileSyncOnClose, functions for common file-related 
operations, etc. This will make it easy for conscientious programmers 
to do the right thing for their app without needing to re-invent the 
wheel all the time, but without handcuffing them into a single "one 
size fits all" solution.

File operations are *hard*, because many error conditions are uncommon, 
and consequently many (possibly even the majority) of programmers never 
learn that something like this:

f = open('myfile', 'w')
f.write(data)
f.close()

(or the equivalent in whatever language they use) may cause data loss. 
Worse, we train users to accept that data loss as normal instead of 
reporting it as a bug -- possibly because it is unclear whether it is a 
bug in the application, the OS, the file system, or all three. (It's 
impossible to avoid *all* risk of data loss, of course -- what if the 
computer loses power in the middle of a write? But we can minimize that 
risk significantly.)

Even when programmers try to do the right thing, it is hard to know what 
the right thing is: there are trade-offs to be made, and having made a 
trade-off, the programmer then has to re-invent what usually turns out 
to be a quite complicated wheel. To do the right thing in Python often 
means delving into the world of os.O_* constants and file descriptors, 
which is intimidating and unpythonic. They're great for those who 
want/need them, but perhaps we should expose a Python interface to the 
more common operations? To my mind, that means classes instead of magic 
constants.

Would there be interest in a filetools module? Replies and discussion to 
python-ideas please.


-- 
Steven D'Aprano

From ggpolo at gmail.com  Thu Mar 12 00:04:26 2009
From: ggpolo at gmail.com (Guilherme Polo)
Date: Wed, 11 Mar 2009 20:04:26 -0300
Subject: [Python-Dev] [Python-ideas] Adding a test discovery into Python
In-Reply-To: <7C7FFEC01D01407CBC9BBD84092D42E6@RaymondLaptop1>
References: <ac2200130902010910x385e4fa8m25d119a707ece87d@mail.gmail.com>
	<gm57o1$5g6$1@ger.gmane.org>
	<ca471dc20902011440h15726e07y8fb39b1eb631dfc@mail.gmail.com>
	<gp94jv$caf$1@ger.gmane.org>
	<7C7FFEC01D01407CBC9BBD84092D42E6@RaymondLaptop1>
Message-ID: <ac2200130903111604j5d34a21ex6aa8a668f6bd8130@mail.gmail.com>

On Wed, Mar 11, 2009 at 7:37 PM, Raymond Hettinger <python at rcn.com> wrote:
> [Christian Heimes]
>>>>
>>>> I'm +1 for a simple (!) test discovery system. I'm emphasizing on simple
>>>> because there are enough frameworks for elaborate unit testing.
>
> Test discovery is not the interesting part of the problem.

Interesting or not, it is a problem that is asking for a solution,
this kind of code is being duplicated in several places for no good
reason.

>
> Axiom: ?The more work involved in writing tests, the fewer
> tests that will get written.

At some point you will have to run them too, I don't think you want to
reimplement the discovery part yet another time.


-- 
-- Guilherme H. Polo Goncalves

From greg.ewing at canterbury.ac.nz  Thu Mar 12 00:11:31 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 12 Mar 2009 12:11:31 +1300
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <gp8ocf$sra$1@ger.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>
	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
	<49B70D83.7030104@cheimes.de> <loom.20090311T021758-280@post.gmane.org>
	<49B725AF.4090705@cheimes.de>
	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
	<49B7C6BA.8010100@cheimes.de> <loom.20090311T141849-464@post.gmane.org>
	<loom.20090311T143816-333@post.gmane.org>
	<20090311153434.GA18918@panix.com>
	<49B7DAF2.8040403@scottdial.com> <gp8ocf$sra$1@ger.gmane.org>
Message-ID: <49B84523.7090206@canterbury.ac.nz>

Lie Ryan wrote:

> I actually prefer strings. Just like 'w' or 'r' in open().
> 
> Or why not add "f" "c" as modes?
> 
> open('file.txt', 'wf')

I like this, because it doesn't expand the signature that
file-like objects need to support. If you're wrapping
another file object you just need to pass on the mode
string and it will all work.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Mar 12 00:17:58 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 12 Mar 2009 12:17:58 +1300
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B80102.6010706@v.loewis.de>
References: <gp6hhq$hss$1@ger.gmane.org>
	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
	<49B70D83.7030104@cheimes.de> <loom.20090311T021758-280@post.gmane.org>
	<49B725AF.4090705@cheimes.de>
	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
	<31764696.2987478.1236780758847.JavaMail.xicrypt@atgrzls001>
	<49B7C759.1080106@avl.com> <49B80102.6010706@v.loewis.de>
Message-ID: <49B846A6.9010408@canterbury.ac.nz>

Martin v. L?wis wrote:

> That should be implement by passing O_SYNC on open, rather than
> explicitly calling fsync.

On platforms which have it (MacOSX doesn't seem to,
according to the man page).

This is another good reason to put these things in the
mode string.

-- 
Greg

From p.f.moore at gmail.com  Thu Mar 12 00:31:10 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 11 Mar 2009 23:31:10 +0000
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B84523.7090206@canterbury.ac.nz>
References: <gp6hhq$hss$1@ger.gmane.org> <49B725AF.4090705@cheimes.de>
	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
	<49B7C6BA.8010100@cheimes.de>
	<loom.20090311T141849-464@post.gmane.org>
	<loom.20090311T143816-333@post.gmane.org>
	<20090311153434.GA18918@panix.com> <49B7DAF2.8040403@scottdial.com>
	<gp8ocf$sra$1@ger.gmane.org> <49B84523.7090206@canterbury.ac.nz>
Message-ID: <79990c6b0903111631y2b047d51i3d676eb1e965e08@mail.gmail.com>

2009/3/11 Greg Ewing <greg.ewing at canterbury.ac.nz>:
> Lie Ryan wrote:
>
>> I actually prefer strings. Just like 'w' or 'r' in open().
>>
>> Or why not add "f" "c" as modes?
>>
>> open('file.txt', 'wf')
>
> I like this, because it doesn't expand the signature that
> file-like objects need to support. If you're wrapping
> another file object you just need to pass on the mode
> string and it will all work.

Of course, a file opened for write, in text mode, with auto-sync on
flush, has mode "wtf". I'm in favour just for the chance to use that
mode :-)

Paul.

From solipsis at pitrou.net  Thu Mar 12 00:42:15 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 11 Mar 2009 23:42:15 +0000 (UTC)
Subject: [Python-Dev] Ext4 data loss
References: <gp6hhq$hss$1@ger.gmane.org>
	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
	<49B70D83.7030104@cheimes.de>
	<loom.20090311T021758-280@post.gmane.org>
	<49B725AF.4090705@cheimes.de>
	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
	<49B7C6BA.8010100@cheimes.de>
	<loom.20090311T141849-464@post.gmane.org>
	<loom.20090311T143816-333@post.gmane.org>
	<20090311153434.GA18918@panix.com>
	<49B7DAF2.8040403@scottdial.com> <gp8ocf$sra$1@ger.gmane.org>
	<49B84523.7090206@canterbury.ac.nz>
Message-ID: <loom.20090311T233159-203@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> 
> I like this, because it doesn't expand the signature that
> file-like objects need to support. If you're wrapping
> another file object you just need to pass on the mode
> string and it will all work.

What do you mean? open() doesn't allow you to wrap other file objects.

As for adding options to the mode string, I think it will only make things
unreadable. Better make the option explicit, like others already are (buffering,
newline, encoding).

Besides, file objects still have to support a sync() method, since sync-on-close
doesn't cater for all uses.



From greg.ewing at canterbury.ac.nz  Thu Mar 12 01:34:29 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 12 Mar 2009 13:34:29 +1300
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090311T233159-203@post.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>
	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>
	<49B70D83.7030104@cheimes.de> <loom.20090311T021758-280@post.gmane.org>
	<49B725AF.4090705@cheimes.de>
	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>
	<49B7C6BA.8010100@cheimes.de> <loom.20090311T141849-464@post.gmane.org>
	<loom.20090311T143816-333@post.gmane.org>
	<20090311153434.GA18918@panix.com>
	<49B7DAF2.8040403@scottdial.com> <gp8ocf$sra$1@ger.gmane.org>
	<49B84523.7090206@canterbury.ac.nz>
	<loom.20090311T233159-203@post.gmane.org>
Message-ID: <49B85895.4010209@canterbury.ac.nz>

Antoine Pitrou wrote:

> What do you mean? open() doesn't allow you to wrap other file objects.

I'm talking about things like GzipFile that take a
filename and mode, open the file and then wrap the
file object.

-- 
Greg

From python at rcn.com  Thu Mar 12 01:47:58 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 11 Mar 2009 17:47:58 -0700
Subject: [Python-Dev] Formatting mini-language suggestion
Message-ID: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>

The current formatting mini-language provisions left/right/center alignment, prefixes for 0b 0x 0o, and rules on when to show the 
plus-sign.  I think it would be far more useful to provision a simple way of specifying a thousands separator.

Financial users in particular find the locale approach to be frustrating and non-obvious.  Putting in a thousands separator is a 
common task for output destined to be read by non-programmers.


Raymond 


From ncoghlan at gmail.com  Thu Mar 12 01:54:26 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Mar 2009 10:54:26 +1000
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B85895.4010209@canterbury.ac.nz>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>
	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>	<49B7C6BA.8010100@cheimes.de>
	<loom.20090311T141849-464@post.gmane.org>	<loom.20090311T143816-333@post.gmane.org>	<20090311153434.GA18918@panix.com>	<49B7DAF2.8040403@scottdial.com>
	<gp8ocf$sra$1@ger.gmane.org>	<49B84523.7090206@canterbury.ac.nz>	<loom.20090311T233159-203@post.gmane.org>
	<49B85895.4010209@canterbury.ac.nz>
Message-ID: <49B85D42.9080406@gmail.com>

Greg Ewing wrote:
> Antoine Pitrou wrote:
> 
>> What do you mean? open() doesn't allow you to wrap other file objects.
> 
> I'm talking about things like GzipFile that take a
> filename and mode, open the file and then wrap the
> file object.

The tempfile module would be another example.

For that reason, I think Steven's idea of a filetools module which
provided context managers and the like that wrapped *existing* file-like
objects might be preferable.

Otherwise it may be a while before sync-aware code is able to deal with
anything other than basic files.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Thu Mar 12 02:01:22 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 12 Mar 2009 01:01:22 +0000 (UTC)
Subject: [Python-Dev] Formatting mini-language suggestion
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
Message-ID: <loom.20090312T005653-93@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:
> 
> Financial users in particular find the locale approach to be frustrating and
non-obvious.  Putting in a
> thousands separator is a 
> common task for output destined to be read by non-programmers.

Please note that for it to be useful in all parts of the world, it must also
allow changing the decimal point.




From ncoghlan at gmail.com  Thu Mar 12 02:06:29 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Mar 2009 11:06:29 +1000
Subject: [Python-Dev] Formatting mini-language suggestion
In-Reply-To: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
Message-ID: <49B86015.2030203@gmail.com>

Raymond Hettinger wrote:
> The current formatting mini-language provisions left/right/center
> alignment, prefixes for 0b 0x 0o, and rules on when to show the
> plus-sign.  I think it would be far more useful to provision a simple
> way of specifying a thousands separator.
> 
> Financial users in particular find the locale approach to be frustrating
> and non-obvious.  Putting in a thousands separator is a common task for
> output destined to be read by non-programmers.

+1 for the general idea.

A specific syntax proposal:

  [[fill]align][sign][#][0][minimumwidth][,sep][.precision][type]

'sep' is the new field that defines the thousands separator. It appears
immediately before the precision specifier and starts with a leading comma.

I believe this syntax is unambiguous and backwards compatible because
the only other place a comma might appear (the fill field) is required
to be followed by an alignment character.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From foom at fuhm.net  Thu Mar 12 02:33:55 2009
From: foom at fuhm.net (James Y Knight)
Date: Wed, 11 Mar 2009 21:33:55 -0400
Subject: [Python-Dev] Formatting mini-language suggestion
In-Reply-To: <49B86015.2030203@gmail.com>
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
	<49B86015.2030203@gmail.com>
Message-ID: <E64B88F5-ABC7-492E-A19D-818C523EEA61@fuhm.net>


On Mar 11, 2009, at 9:06 PM, Nick Coghlan wrote:

> Raymond Hettinger wrote:
>> The current formatting mini-language provisions left/right/center
>> alignment, prefixes for 0b 0x 0o, and rules on when to show the
>> plus-sign.  I think it would be far more useful to provision a simple
>> way of specifying a thousands separator.
>>
>> Financial users in particular find the locale approach to be  
>> frustrating
>> and non-obvious.  Putting in a thousands separator is a common task  
>> for
>> output destined to be read by non-programmers.
>
> +1 for the general idea.
>
> A specific syntax proposal:
>
>  [[fill]align][sign][#][0][minimumwidth][,sep][.precision][type]
>
> 'sep' is the new field that defines the thousands separator. It  
> appears
> immediately before the precision specifier and starts with a leading  
> comma.
>
> I believe this syntax is unambiguous and backwards compatible because
> the only other place a comma might appear (the fill field) is required
> to be followed by an alignment character.

You might be interested to know that in India, the commas don't come  
every 3 digits. In india, they come every two digits, after the first  
three. Thus one billion = 1,00,00,00,000. How are you gonna represent  
*that* in a formatting mini-language? :)

See also http://en.wikipedia.org/wiki/Indian_numbering_system

James

From solipsis at pitrou.net  Thu Mar 12 03:03:13 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 12 Mar 2009 02:03:13 +0000 (UTC)
Subject: [Python-Dev] Ext4 data loss
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>
	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>	<49B7C6BA.8010100@cheimes.de>
	<loom.20090311T141849-464@post.gmane.org>	<loom.20090311T143816-333@post.gmane.org>	<20090311153434.GA18918@panix.com>	<49B7DAF2.8040403@scottdial.com>
	<gp8ocf$sra$1@ger.gmane.org>	<49B84523.7090206@canterbury.ac.nz>	<loom.20090311T233159-203@post.gmane.org>
	<49B85895.4010209@canterbury.ac.nz> <49B85D42.9080406@gmail.com>
Message-ID: <loom.20090312T010957-202@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:
> 
> The tempfile module would be another example.

Do you really need your temporary files to survive system crashes? ;)

> For that reason, I think Steven's idea of a filetools module which
> provided context managers and the like that wrapped *existing* file-like
> objects might be preferable.

Well, well, let's clarify things a bit.
If we want to help users with this problem, we can provide two things:
1. a new sync() method on the standard objects provided by the IO lib
2. a facility to automatically call sync() on flush() and/or close() calls

Step 1 may be done with a generic implementation in the IO ABCs calling
self.flush() and then os.fsync(self.fileno()). IMO it is important that it is a
method of IO objects because implementations may want to override it. An
external facility would be too inflexible.

Step 2 may be done with a generic wrapper. However, we could also provide an
open() flag which transparently invokes the wrapper. After all, open() is
already a convenience function creating a raw file object and wrapping it in two
optional layers.

(as a side note, wrappers have a non-zero performance impact, especially on
small ops - e.g. reading or writing a few bytes)



From zooko at zooko.com  Thu Mar 12 02:26:40 2009
From: zooko at zooko.com (zooko)
Date: Wed, 11 Mar 2009 19:26:40 -0600
Subject: [Python-Dev] [Python-ideas]  Ext4 data loss
In-Reply-To: <200903120948.28250.steve@pearwood.info>
References: <gp6hhq$hss$1@ger.gmane.org> <49B7C6BA.8010100@cheimes.de>
	<loom.20090311T141849-464@post.gmane.org>
	<200903120948.28250.steve@pearwood.info>
Message-ID: <C772320F-990D-441F-9989-88C69B5F730B@zooko.com>

> Would there be interest in a filetools module? Replies and  
> discussion to python-ideas please.


I've been using and maintaining a few filesystem hacks for, let's  
see, almost nine years now:

http://allmydata.org/trac/pyutil/browser/pyutil/pyutil/fileutil.py

(The first version of that was probably written by Greg Smith in  
about 1999.)

I'm sure there are many other such packages.  A couple of quick  
searches of pypi turned up these two:

http://pypi.python.org/pypi/Pythonutils
http://pypi.python.org/pypi/fs

I wonder if any of them have the sort of functionality you're  
thinking of.

Regards,

Zooko

From python at rcn.com  Thu Mar 12 03:50:13 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 11 Mar 2009 19:50:13 -0700
Subject: [Python-Dev] Formatting mini-language suggestion
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
	<49B86015.2030203@gmail.com>
	<E64B88F5-ABC7-492E-A19D-818C523EEA61@fuhm.net>
Message-ID: <ED15EA05C80840BEBCE209D5B8DF47DC@RaymondLaptop1>


[James Y Knight]
> You might be interested to know that in India, the commas don't come  
> every 3 digits. In india, they come every two digits, after the first  
> three. Thus one billion = 1,00,00,00,000. How are you gonna represent  
> *that* in a formatting mini-language? :)

It is not the goal to replace locale or to accomodate every
possible convention.  The goal is to make a common task easier
for many users.  The current, default use of the period as a decimal 
point has not proven to be problem eventhough that convention is
not universal.   For a thousands separator, a comma is a decent choice 
that makes it easy follow-on with s.replace(',', '_') or somesuch.

This simple utility could help a lot of programmers make their output
look more professional and readable.  I hope the idea doesn't get
sunk by a desire to over-parameterize and cover every possible use case.

My pocket calculators all support thousands separators but in Python,
we have to do a funky dance for even this most basic bit of formatting.

I'd like to think that in 2009 we could show a little progress beyond
C's printf() or Fortran's write() formats.


Raymond



>>> import locale
>>> locale.setlocale(locale.LC_ALL, 'English_United States.1252')
'English_United States.1252'
>>> conv = locale.localeconv()          # get a mapping of conventions
>>> x = 1234567.8
>>> locale.format("%d", x, grouping=True)
'1,234,567'


From ben+python at benfinney.id.au  Thu Mar 12 04:09:52 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Thu, 12 Mar 2009 14:09:52 +1100
Subject: [Python-Dev] Formatting mini-language suggestion
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
	<49B86015.2030203@gmail.com>
	<E64B88F5-ABC7-492E-A19D-818C523EEA61@fuhm.net>
Message-ID: <87bps7xwqn.fsf@benfinney.id.au>

James Y Knight <foom at fuhm.net> writes:

> You might be interested to know that in India, the commas don't come
> every 3 digits. In india, they come every two digits, after the
> first three. Thus one billion = 1,00,00,00,000. How are you gonna
> represent *that* in a formatting mini-language? :)

Likewise, China uses four-digit groupings (per ?myriad?)
<URL:http://en.wikipedia.org/wiki/Chinese_numerals#Reading_and_transcribing_numbers>.

-- 
 \           ?Self-respect: The secure feeling that no one, as yet, is |
  `\                                    suspicious.? ?Henry L. Mencken |
_o__)                                                                  |
Ben Finney


From guido at python.org  Thu Mar 12 04:20:57 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 11 Mar 2009 20:20:57 -0700
Subject: [Python-Dev] Formatting mini-language suggestion
In-Reply-To: <loom.20090312T005653-93@post.gmane.org>
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
	<loom.20090312T005653-93@post.gmane.org>
Message-ID: <ca471dc20903112020s71612e32h2605aa0db7624349@mail.gmail.com>

On Wed, Mar 11, 2009 at 6:01 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Raymond Hettinger <python <at> rcn.com> writes:
>>
>> Financial users in particular find the locale approach to be frustrating and
> non-obvious. ?Putting in a
>> thousands separator is a
>> common task for output destined to be read by non-programmers.
>
> Please note that for it to be useful in all parts of the world, it must also
> allow changing the decimal point.

Now that this cat is out of the bag (or should I say now that this can
of worms is opened :-) I suggest moving this to python-ideas and
writing a proper PEP. I expect that nobody likes that idea, but it
seems better than the alternative, which is to let the programmer who
gets to implement it design it...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Thu Mar 12 04:23:36 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Mar 2009 13:23:36 +1000
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090312T010957-202@post.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>	<49B7C6BA.8010100@cheimes.de>	<loom.20090311T141849-464@post.gmane.org>	<loom.20090311T143816-333@post.gmane.org>	<20090311153434.GA18918@panix.com>	<49B7DAF2.8040403@scottdial.com>	<gp8ocf$sra$1@ger.gmane.org>	<49B84523.7090206@canterbury.ac.nz>	<loom.20090311T233159-203@post.gmane.org>	<49B85895.4010209@canterbury.ac.nz>
	<49B85D42.9080406@gmail.com>
	<loom.20090312T010957-202@post.gmane.org>
Message-ID: <49B88038.1080003@gmail.com>

Antoine Pitrou wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>> The tempfile module would be another example.
> 
> Do you really need your temporary files to survive system crashes? ;)

No, but they need to provide the full file API. If we add a sync()
method to file objects, that becomes part of the "file-like" API.

On the performance side... the overhead from fsync() itself is going to
dwarf the CPU overhead of going through a wrapper class.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From tleeuwenburg at gmail.com  Thu Mar 12 04:23:50 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Thu, 12 Mar 2009 14:23:50 +1100
Subject: [Python-Dev] Approaches to argument type-checking
Message-ID: <43c8685c0903112023s29444188wc685a67d484b8999@mail.gmail.com>

Hi all,

I am currently looking at issue 5236. This issue regards the exception
raised when a bytes string is passed into time.strptime. In addition to the
specific question I have regarding this issue, I wasn't sure if this was
something for python-dev or for the issue comment. However, it does concern
general Python coding approach, so just give me a pointer over whether this
is better kept on the tracker or whether posting to the list was a good idea
(I'm slowly learning!)

EXAMPLE OF PROBLEM:

>>> time.strptime(b'2009', "%Y")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/tjl/python3/lib/python3.1/_strptime.py", line 454, in
_strptime_time
    return _strptime(data_string, format)[0]
  File "/home/tjl/python3/lib/python3.1/_strptime.py", line 322, in
_strptime
    found = format_regex.match(data_string)
TypeError: can't use a string pattern on a bytes-like object


WHEREAS:

>>> time.strftime(b"%Y")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: strftime() argument 1 must be str, not bytes


What is occurring here is that the arguments to strftime are being
type-checked up-front, whereas in strptime they are not. Further, srtptime
is implemented in a python file, _strptime.py, whiel strftime is implemented
in timemodule.c.

It appears as though it is generally the case (or at least often the case)
that C functions are making use of the vgetargs function which performs a
goodly bit of type checking. However, the same does not seem to hold for the
Python interface functions.

>From the Python interpreter perspective, though, both are in the time module
(time.strftime and time.strptime) so the inconsistency is a bit jarring. I
can see that I could solve this a few ways:
  * Do a false-parse of the arguments using the same vgetargs1 method, but
not do anything with the return value
  * Perform a type-check on the relevant argument, data_string, in Python
and raise a more specific Exception
  * Write some kind of generic type-checking helper method which could be
re-used

Is there a general strategy used in Python development which I should be
aware of? i.e. is it customary to type-check every argument of an interface
method? Or is it customary not to perform type checking up-front and simply
allow the exception to occur deeper in the code? Are there performance
issues surrounding defensive programming?

Regards,
-Tennessee


-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090312/a8069c7b/attachment.htm>

From benjamin at python.org  Thu Mar 12 04:29:34 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 11 Mar 2009 22:29:34 -0500
Subject: [Python-Dev] Approaches to argument type-checking
In-Reply-To: <43c8685c0903112023s29444188wc685a67d484b8999@mail.gmail.com>
References: <43c8685c0903112023s29444188wc685a67d484b8999@mail.gmail.com>
Message-ID: <1afaf6160903112029j217dc631pe751bd694364b614@mail.gmail.com>

2009/3/11 Tennessee Leeuwenburg <tleeuwenburg at gmail.com>:
> Is there a general strategy used in Python development which I should be
> aware of? i.e. is it customary to type-check every argument of an interface
> method? Or is it customary not to perform type checking up-front and simply
> allow the exception to occur deeper in the code? Are there performance
> issues surrounding defensive programming?

Generally we avoid checking types at all in Python because of ducking
typing. The C interface must check types because they have to
translate to the C level equivalents.

If tests are failing from a C implementation on a Python
implementation because of extensive type checking, I would be tempted
to mark those tests as implementation details.

However, in the case of this specific issue, I think rejecting bytes
purposefully is good because it avoids programming errors.



-- 
Regards,
Benjamin

From ncoghlan at gmail.com  Thu Mar 12 04:40:02 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Mar 2009 13:40:02 +1000
Subject: [Python-Dev] Formatting mini-language suggestion
In-Reply-To: <ED15EA05C80840BEBCE209D5B8DF47DC@RaymondLaptop1>
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
	<49B86015.2030203@gmail.com>
	<E64B88F5-ABC7-492E-A19D-818C523EEA61@fuhm.net>
	<ED15EA05C80840BEBCE209D5B8DF47DC@RaymondLaptop1>
Message-ID: <49B88412.9050902@gmail.com>

Raymond Hettinger wrote:
> 
> [James Y Knight]
>> You might be interested to know that in India, the commas don't come 
>> every 3 digits. In india, they come every two digits, after the first 
>> three. Thus one billion = 1,00,00,00,000. How are you gonna represent 
>> *that* in a formatting mini-language? :)
> 
> It is not the goal to replace locale or to accomodate every
> possible convention.  The goal is to make a common task easier
> for many users.  The current, default use of the period as a decimal
> point has not proven to be problem eventhough that convention is
> not universal.   For a thousands separator, a comma is a decent choice
> that makes it easy follow-on with s.replace(',', '_') or somesuch.

In that case, I would simplify my suggestion to:

  [[fill]align][sign][#][0][minimumwidth][,][.precision][type]

Addition to mini language documentation:
  The ',' option indicates that commas should be included in the
 output as a thousands separator. As with locales which do not use a
 period as the decimal point, locales which use a different convention
 for digit separation will need to use the locale module to obtain
 appropriate formatting.

Guido has asked for a PEP to be developed on python-ideas to define the
deliberately limited scope though, so I'm going to bow out of the
conversation now...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From python at rcn.com  Thu Mar 12 04:45:24 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 11 Mar 2009 20:45:24 -0700
Subject: [Python-Dev] Formatting mini-language suggestion
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1><loom.20090312T005653-93@post.gmane.org>
	<ca471dc20903112020s71612e32h2605aa0db7624349@mail.gmail.com>
Message-ID: <7A6631E172AC4D12956AEE832E28F52F@RaymondLaptop1>


[Guido van Rossum]
> I suggest moving this to python-ideas and
> writing a proper PEP. 

Okay, it's moved.

Will write up a PEP, do research on what  other languages 
do and collect everyone's ideas on what to put in the shed.
(hundreds and ten thousands grouping, various choices of 
decimal points, mayan number systems and whatnot).  Will
start with Nick's simple proposal as a starting point.

[Nick Coghlan]
>  [[fill]align][sign][#][0][minimumwidth][,][.precision][type]

Other suggestions and comments welcome.


Raymond

From guido at python.org  Thu Mar 12 04:50:47 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 11 Mar 2009 20:50:47 -0700
Subject: [Python-Dev] Formatting mini-language suggestion
In-Reply-To: <D06880392B41411BB277C75ADC3095D8@RaymondLaptop1>
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
	<loom.20090312T005653-93@post.gmane.org>
	<ca471dc20903112020s71612e32h2605aa0db7624349@mail.gmail.com>
	<D06880392B41411BB277C75ADC3095D8@RaymondLaptop1>
Message-ID: <ca471dc20903112050j7bcb2c62y52e9687251af0c80@mail.gmail.com>

On Wed, Mar 11, 2009 at 8:34 PM, Raymond Hettinger <python at rcn.com> wrote:
>> ?I expect that nobody likes that idea,
>
> Do you mean the idea of a thousands separator
> or the idea of also parameterizing the decimal point
> or both?

Sorry, neither. I meant the idea of having to write a PEP. :-)

(Added back python-dev to clarify for all.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From foom at fuhm.net  Thu Mar 12 05:36:35 2009
From: foom at fuhm.net (James Y Knight)
Date: Thu, 12 Mar 2009 00:36:35 -0400
Subject: [Python-Dev] Formatting mini-language suggestion
In-Reply-To: <49B88412.9050902@gmail.com>
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
	<49B86015.2030203@gmail.com>
	<E64B88F5-ABC7-492E-A19D-818C523EEA61@fuhm.net>
	<ED15EA05C80840BEBCE209D5B8DF47DC@RaymondLaptop1>
	<49B88412.9050902@gmail.com>
Message-ID: <4290AE97-5B4D-4949-8090-7983FDBB9AEB@fuhm.net>

On Mar 11, 2009, at 11:40 PM, Nick Coghlan wrote:
> Raymond Hettinger wrote:
>> It is not the goal to replace locale or to accomodate every
>> possible convention.  The goal is to make a common task easier
>> for many users.  The current, default use of the period as a decimal
>> point has not proven to be problem eventhough that convention is
>> not universal.   For a thousands separator, a comma is a decent  
>> choice
>> that makes it easy follow-on with s.replace(',', '_') or somesuch.
>
> In that case, I would simplify my suggestion to:
>
>  [[fill]align][sign][#][0][minimumwidth][,][.precision][type]
>
> Addition to mini language documentation:
>  The ',' option indicates that commas should be included in the
> output as a thousands separator. As with locales which do not use a
> period as the decimal point, locales which use a different convention
> for digit separation will need to use the locale module to obtain
> appropriate formatting.


This proposal has the advantage that you're not overly specifying the  
behavior in the format string itself.

That is: the "," option is really just indicating "please insert  
separators". With the current locale-ignorant implementation, that'd  
just mean "a comma every 3 digits". But it leaves the door open for a  
locale-sensitive variant of the format to be added in the future  
without conflicting with the instructions in the format string. (as  
the ability to specify an arbitrary character, or the ability to  
specify a comma instead of a period for the decimal point would).

I'm not against Raymond's proposal, just against doing a *bad* job of  
making it work in multiple locales. Locale conventions can be complex,  
and are going to be best represented outside the format string.

(BTW: single quote is used by printf for the grouping flag rather than  
comma)

James

From lie.1296 at gmail.com  Thu Mar 12 06:50:12 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Thu, 12 Mar 2009 16:50:12 +1100
Subject: [Python-Dev] Formatting mini-language suggestion
In-Reply-To: <4290AE97-5B4D-4949-8090-7983FDBB9AEB@fuhm.net>
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>	<49B86015.2030203@gmail.com>	<E64B88F5-ABC7-492E-A19D-818C523EEA61@fuhm.net>	<ED15EA05C80840BEBCE209D5B8DF47DC@RaymondLaptop1>	<49B88412.9050902@gmail.com>
	<4290AE97-5B4D-4949-8090-7983FDBB9AEB@fuhm.net>
Message-ID: <gpa7qk$7h6$1@ger.gmane.org>

James Y Knight wrote:
> On Mar 11, 2009, at 11:40 PM, Nick Coghlan wrote:
>> Raymond Hettinger wrote:
>>> It is not the goal to replace locale or to accomodate every
>>> possible convention.  The goal is to make a common task easier
>>> for many users.  The current, default use of the period as a decimal
>>> point has not proven to be problem eventhough that convention is
>>> not universal.   For a thousands separator, a comma is a decent choice
>>> that makes it easy follow-on with s.replace(',', '_') or somesuch.
>>
>> In that case, I would simplify my suggestion to:
>>
>>  [[fill]align][sign][#][0][minimumwidth][,][.precision][type]
>>
>> Addition to mini language documentation:
>>  The ',' option indicates that commas should be included in the
>> output as a thousands separator. As with locales which do not use a
>> period as the decimal point, locales which use a different convention
>> for digit separation will need to use the locale module to obtain
>> appropriate formatting.
> 
> 
> This proposal has the advantage that you're not overly specifying the 
> behavior in the format string itself.
> 
> That is: the "," option is really just indicating "please insert 
> separators". With the current locale-ignorant implementation, that'd 
> just mean "a comma every 3 digits". But it leaves the door open for a 
> locale-sensitive variant of the format to be added in the future without 
> conflicting with the instructions in the format string. (as the ability 
> to specify an arbitrary character, or the ability to specify a comma 
> instead of a period for the decimal point would).
> 
> I'm not against Raymond's proposal, just against doing a *bad* job of 
> making it work in multiple locales. Locale conventions can be complex, 
> and are going to be best represented outside the format string.

How about having a country code field, e.g. en-us would format according 
to US locale, in to India, ch to China, etc... that way the format 
string would become very simple (although the lib maintainer would need 
to know customs from all over the world). Then have a special country 
code that is a placeholder for whatever the locale the machine is set to.


From python at rcn.com  Thu Mar 12 07:19:05 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 11 Mar 2009 23:19:05 -0700
Subject: [Python-Dev] Formatting mini-language suggestion
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>	<49B86015.2030203@gmail.com>	<E64B88F5-ABC7-492E-A19D-818C523EEA61@fuhm.net>	<ED15EA05C80840BEBCE209D5B8DF47DC@RaymondLaptop1>	<49B88412.9050902@gmail.com><4290AE97-5B4D-4949-8090-7983FDBB9AEB@fuhm.net>
	<gpa7qk$7h6$1@ger.gmane.org>
Message-ID: <C38702A6E3D44D14B005A581C842185E@RaymondLaptop1>


[Lie Ryan]
> How about having a country code field, e.g. en-us would format according 
> to US locale, in to India, ch to China, etc... that way the format 
> string would become very simple (although the lib maintainer would need 
> to know customs from all over the world). Then have a special country 
> code that is a placeholder for whatever the locale the machine is set to.

Am moving the discussion to the python-ideas list (at Guido's request).
My proposal is strictly limited to the builtin, non-locale dependent formatting.
Improvements to the locale module are probably as subject for another day.


Raymond




From asmodai at in-nomine.org  Thu Mar 12 08:59:36 2009
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Thu, 12 Mar 2009 08:59:36 +0100
Subject: [Python-Dev] Formatting mini-language suggestion
In-Reply-To: <gpa7qk$7h6$1@ger.gmane.org>
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
	<49B86015.2030203@gmail.com>
	<E64B88F5-ABC7-492E-A19D-818C523EEA61@fuhm.net>
	<ED15EA05C80840BEBCE209D5B8DF47DC@RaymondLaptop1>
	<49B88412.9050902@gmail.com>
	<4290AE97-5B4D-4949-8090-7983FDBB9AEB@fuhm.net>
	<gpa7qk$7h6$1@ger.gmane.org>
Message-ID: <20090312075936.GL10062@nexus.in-nomine.org>

-On [20090312 06:50], Lie Ryan (lie.1296 at gmail.com) wrote:
>How about having a country code field, e.g. en-us would format according 
>to US locale, in to India, ch to China, etc... that way the format 
>string would become very simple (although the lib maintainer would need 
>to know customs from all over the world). Then have a special country 
>code that is a placeholder for whatever the locale the machine is set to.

Then you are effectively duplicating what is already available via CLDR [1]
and Babel [2].

[1] http://www.unicode.org/cldr/
[2] http://babel.edgewall.org/

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
Any road leads to the end of the world...

From gisle at activestate.com  Thu Mar 12 09:06:26 2009
From: gisle at activestate.com (Gisle Aas)
Date: Thu, 12 Mar 2009 09:06:26 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <20090311214344.GA28718@cskk.homeip.net>
References: <20090311214344.GA28718@cskk.homeip.net>
Message-ID: <04853B4D-A94B-4CC1-8000-81154C7F6917@activestate.com>

On Mar 11, 2009, at 22:43 , Cameron Simpson wrote:

> On 11Mar2009 10:09, Joachim K?nig <him at online.de> wrote:
>> Guido van Rossum wrote:
>>> On Tue, Mar 10, 2009 at 1:11 PM, Christian Heimes  
>>> <lists at cheimes.de> wrote:
>>>> [...]
>>>> https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/317781/comments/54 
>>>> .
>>>> [...]
>>> If I understand the post properly, it's up to the app to call  
>>> fsync(),
>>> and it's only necessary when you're doing one of the rename  
>>> dances, or
>>> updating a file in place. Basically, as he explains, fsync() is a  
>>> very
>>> heavyweight operation; I'm against calling it by default anywhere.
>>>
>> To me, the flaw seem to be in the close() call (of the operating
>> system). I'd expect the data to be
>> in a persistent state once the close() returns. So there would be no
>> need to fsync if the file gets closed anyway.
>
> Not really. On the whole, flush() means "the object has handed all  
> data
> to the OS".  close() means "the object has handed all data to the OS
> and released the control data structures" (OS file descriptor release;
> like the OS, the python interpreter may release python stuff later  
> too).
>
> By contrast, fsync() means "the OS has handed filesystem changes to  
> the
> disc itself". Really really slow, by comparison with memory. It is  
> Very
> Expensive, and a very different operation to close().

...and at least on OS X there is one level more where you actually  
tell the
disc to flush its buffers to permanent storage with:

    fcntl(fd, F_FULLSYNC)

The fsync manpage says:

      Note that while fsync() will flush all data from the host to the  
drive
      (i.e. the "permanent storage device"), the drive itself may not  
physi-
      cally write the data to the platters for quite some time and it  
may be
      written in an out-of-order sequence.

      Specifically, if the drive loses power or the OS crashes, the  
application
      may find that only some or none of their data was written.  The  
disk
      drive may also re-order the data so that later writes may be  
present,
      while earlier writes are not.

      This is not a theoretical edge case.  This scenario is easily  
reproduced
      with real world workloads and drive power failures.

      For applications that require tighter guarantees about the  
integrity of
      their data, Mac OS X provides the F_FULLFSYNC fcntl.  The  
F_FULLFSYNC
      fcntl asks the drive to flush all buffered data to permanent  
storage.
      Applications, such as databases, that require a strict ordering  
of writes
      should use F_FULLFSYNC to ensure that their data is written in  
the order
      they expect.  Please see fcntl(2) for more detail.

It's not obvious what level of syncing is appropriate to automatically  
happen
from Python so I think it's better to let the application deal with it.

--Gisle


From solipsis at pitrou.net  Thu Mar 12 11:39:26 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 12 Mar 2009 10:39:26 +0000 (UTC)
Subject: [Python-Dev] Ext4 data loss
References: <gp6hhq$hss$1@ger.gmane.org>	<ca471dc20903101323k6a9bd700oeed24cd6d70dbcd5@mail.gmail.com>	<49B70D83.7030104@cheimes.de>	<loom.20090311T021758-280@post.gmane.org>	<49B725AF.4090705@cheimes.de>	<ca471dc20903101955m5a1ea935p79a09f36952212a3@mail.gmail.com>	<49B7C6BA.8010100@cheimes.de>	<loom.20090311T141849-464@post.gmane.org>	<loom.20090311T143816-333@post.gmane.org>	<20090311153434.GA18918@panix.com>	<49B7DAF2.8040403@scottdial.com>	<gp8ocf$sra$1@ger.gmane.org>	<49B84523.7090206@canterbury.ac.nz>	<loom.20090311T233159-203@post.gmane.org>	<49B85895.4010209@canterbury.ac.nz>
	<49B85D42.9080406@gmail.com>
	<loom.20090312T010957-202@post.gmane.org>
	<49B88038.1080003@gmail.com>
Message-ID: <loom.20090312T103755-804@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:
> 
> On the performance side... the overhead from fsync() itself is going to
> dwarf the CPU overhead of going through a wrapper class.

The significant overhead is not in calling sync() or flush() or close(), but in
calling methods which are supposed to be fast (read() from internal buffer or
write() to internal buffer, for example).



From python at rcn.com  Thu Mar 12 11:41:24 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 12 Mar 2009 03:41:24 -0700
Subject: [Python-Dev] [Python-ideas] Rough draft: Proposed format
	specifier for a	thousands separator (discussion moved from
	python-dev)
References: <56EE433347824B66BC1D5FF4E3F47A2C@RaymondLaptop1>
	<49B8D702.4040004@trueblade.com>
Message-ID: <1F032CDC26874991B78E47DBDB333917@RaymondLaptop1>

Here's an update incorporating all the comments received so far.

* Put into PEP format
* Fixed typos
* The suggestion for modifying the locale module was dropped.
* The "n" specifier in the local module was referenced
* Fixed minimumwidth --> width
* PERIOD --> DOT
* Added suggestions by Lie Ryan and Eric Smith

-----------------------------------------------------------


PEP: XXX
Title: Format Specifier for Thousands Separator
Version: $Revision$
Last-Modified: $Date$
Author: Raymond Hettinger <python at rcn.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 12-Mar-2009
Post-History: 12-Mar-2009


Motivation
==========

Provide a simple, non-locale aware way to format a number
with a thousands separator.

Adding thousands separators is one of the simplest ways to
improve the professional appearance and readability of output
exposed to end users.

In the finance world, output with commas is the norm.  Finance
users and non-professional programmers find the locale
approach to be frustrating, arcane and non-obvious.

It is not the goal to replace locale or to accommodate every
possible convention.  The goal is to make a common task easier
for many users.


Current Version of the Mini-Language
====================================

* `Python 2.6 docs`_

  .. _Python 2.6 docs: http://docs.python.org/library/string.html#formatstrings

* PEP 3101 Advanced String Formatting


Research so far
===============

Scanning the web, I've found that thousands separators are
usually one of COMMA, DOT, SPACE, or UNDERSCORE.
When a COMMA is the decimal separator, the thousands separator
is typically a DOT or SPACE (see examples from Denis Spir).

James Knight observed that Indian/Pakistani numbering systems
group by hundreds.   Ben Finney noted that Chinese group by
ten-thousands.  Eric Smith pointed-out that these are already
handled by the "n" specifier in the locale module (albiet only
for integers).

Visual Basic and its brethren (like MS Excel) use a completely
different style and have ultra-flexible custom format
specifiers like: "_($* #,##0_)".


Proposal I (from Nick Coghlan)
==============================

A comma will be added to the format() specifier mini-language:

[[fill]align][sign][#][0][width][,][.precision][type]

The ',' option indicates that commas should be included in the
output as a thousands separator. As with locales which do not
use a period as the decimal point, locales which use a
different convention for digit separation will need to use the
locale module to obtain appropriate formatting.

The proposal works well with floats, ints, and decimals.
It also allows easy substitution for other separators.
For example::

  format(n, "6,f").replace(",", "_")

This technique is completely general but it is awkward in the
one case where the commas and periods need to be swapped::

  format(n, "6,f").replace(",", "X").replace(".", ",").replace("X", ".")


Proposal II (to meet Antoine Pitrou's request)
==============================================

Make both the thousands separator and decimal separator user
specifiable but not locale aware.  For simplicity, limit the
choices to a comma, period, space, or underscore.

[[fill]align][sign][#][0][width][T[tsep]][dsep precision][type]

Examples::

  format(1234, "8.1f")    -->     '  1234.0'
  format(1234, "8,1f")    -->     '  1234,0'
  format(1234, "8T.,1f")  -->     ' 1.234,0'
  format(1234, "8T .f")   -->     ' 1 234,0'
  format(1234, "8d")      -->     '    1234'
  format(1234, "8T,d")    -->     '   1,234'

This proposal meets mosts needs (except for people wanting
grouping for hundreds or ten-thousands), but it comes at the
expense of being a little more complicated to learn and
remember.  Also, it makes it more challenging to write custom
__format__ methods that follow the format specification
mini-language.

No change is proposed for the local module.


Other Ideas
===========

* Lie Ryan suggested a convenience function of the form::

    create_format(self, type='i', base=16, seppos=4, sep=':', \
                  charset='0123456789abcdef', maxwidth=32,    \
                  minwidth=32, pad='0')

* Eric Smith would like the C version of the mini-language
  parser to be exposed.  That would make it easier to write
  custom __format__ methods.


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: 


From mail at timgolden.me.uk  Thu Mar 12 11:11:51 2009
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 12 Mar 2009 10:11:51 +0000
Subject: [Python-Dev] Building Py3K branch docs with Sphinx
Message-ID: <49B8DFE7.3040403@timgolden.me.uk>

Can I ask which flavour of Sphinx is being used to build the py3k docs? 
I've taken the naive approach of simply pulling the sources from
branches/py3k and then calling make checkout to fetch the appropriate
sources, but these are from http://svn.python.org/projects and are
the same for 2.x and 3.x (and don't work under 3.x).

The latest sphinx from its mercurial tip repo has the same issues
so I wondered whether built the released docs used some other svn 
source or simply patched. The readme points out that the code won't 
work under Python 3.x but someone's managed to build the docs for 
the already-released versions.

(using the make.bat under Windows, but AFAICT the unix-style Makefile 
would have the same issues).

TJG

From steve at pearwood.info  Thu Mar 12 12:54:21 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 12 Mar 2009 22:54:21 +1100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090312T010957-202@post.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org> <49B85D42.9080406@gmail.com>
	<loom.20090312T010957-202@post.gmane.org>
Message-ID: <200903122254.21316.steve@pearwood.info>

On Thu, 12 Mar 2009 01:03:13 pm Antoine Pitrou wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
> > The tempfile module would be another example.
>
> Do you really need your temporary files to survive system crashes? ;)

It depends on what you mean by "temporary".

Applications like OpenOffice can sometimes recover from an application 
crash or even a systems crash and give you the opportunity to restore 
the temporary files that were left lying around. Firefox does the same 
thing -- after a crash, it offers you the opportunity to open the 
websites you had open before. Konquorer does much the same, except it 
can only recover from application crashes, not system crashes. I can't 
tell you how many times such features have saved my hide!




-- 
Steven D'Aprano

From solipsis at pitrou.net  Thu Mar 12 13:21:31 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 12 Mar 2009 12:21:31 +0000 (UTC)
Subject: [Python-Dev] Ext4 data loss
References: <gp6hhq$hss$1@ger.gmane.org> <49B85D42.9080406@gmail.com>
	<loom.20090312T010957-202@post.gmane.org>
	<200903122254.21316.steve@pearwood.info>
Message-ID: <loom.20090312T122030-705@post.gmane.org>

Steven D'Aprano <steve <at> pearwood.info> writes:
> 
> It depends on what you mean by "temporary".
> 
> Applications like OpenOffice can sometimes recover from an application 
> crash or even a systems crash and give you the opportunity to restore 
> the temporary files that were left lying around.

For such files, you want deterministic naming in order to find them again, so
you won't use the tempfile module...




From mail at timgolden.me.uk  Thu Mar 12 13:27:38 2009
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 12 Mar 2009 12:27:38 +0000
Subject: [Python-Dev] Building Py3K branch docs with Sphinx
In-Reply-To: <49B8DFE7.3040403@timgolden.me.uk>
References: <49B8DFE7.3040403@timgolden.me.uk>
Message-ID: <49B8FFBA.5090708@timgolden.me.uk>

Tim Golden wrote:
> Can I ask which flavour of Sphinx is being used to build the py3k docs? 
> I've taken the naive approach of simply pulling the sources from
> branches/py3k and then calling make checkout to fetch the appropriate
> sources, but these are from http://svn.python.org/projects and are
> the same for 2.x and 3.x (and don't work under 3.x).


... or I could just use an existing Python 2.x installation to build
the 3.x docs. Obviously. (slaps forehead)

TJG

From a.badger at gmail.com  Thu Mar 12 15:43:55 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 12 Mar 2009 07:43:55 -0700
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <loom.20090312T122030-705@post.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>
	<49B85D42.9080406@gmail.com>	<loom.20090312T010957-202@post.gmane.org>	<200903122254.21316.steve@pearwood.info>
	<loom.20090312T122030-705@post.gmane.org>
Message-ID: <49B91FAB.2010101@gmail.com>

Antoine Pitrou wrote:
> Steven D'Aprano <steve <at> pearwood.info> writes:
>> It depends on what you mean by "temporary".
>>
>> Applications like OpenOffice can sometimes recover from an application 
>> crash or even a systems crash and give you the opportunity to restore 
>> the temporary files that were left lying around.
> 
> For such files, you want deterministic naming in order to find them again, so
> you won't use the tempfile module...
> 
Something that doesn't require deterministicly named tempfiles was Ted
T'so's explanation linked to earlier.

read data from important file
modify data
create tempfile
write data to tempfile
*sync tempfile to disk*
mv tempfile to filename of important file

The sync is necessary to ensure that the data is written to the disk
before the old file overwrites the new filename.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090312/6ed3e6e4/attachment.pgp>

From mail at timgolden.me.uk  Thu Mar 12 16:33:23 2009
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 12 Mar 2009 15:33:23 +0000
Subject: [Python-Dev] Building Py3K branch docs with Sphinx
In-Reply-To: <e44eaae0f887ccaa0c357c230062e940.squirrel@localhost>
References: <49B8DFE7.3040403@timgolden.me.uk>
	<49B8FFBA.5090708@timgolden.me.uk>
	<e44eaae0f887ccaa0c357c230062e940.squirrel@localhost>
Message-ID: <49B92B43.40302@timgolden.me.uk>

andrew cooke wrote:
> Tim Golden wrote:
>> Tim Golden wrote:
>>> Can I ask which flavour of Sphinx is being used to build the py3k docs?
>>> I've taken the naive approach of simply pulling the sources from
>>> branches/py3k and then calling make checkout to fetch the appropriate
>>> sources, but these are from http://svn.python.org/projects and are
>>> the same for 2.x and 3.x (and don't work under 3.x).
>>
>> ... or I could just use an existing Python 2.x installation to build
>> the 3.x docs. Obviously. (slaps forehead)
> 
> I asked about this on the Sphinx list a while back.  I didn't get any
> response at the time, but checking now I see that a week later someone
> (the author I assume) commented -
> http://groups.google.com/group/sphinx-dev/browse_thread/thread/9a0286f5deeb2912/778a02c397295add
> 
> So it seems that there is no public solution until release 0.6, and that
> you cannot be able to run doctests when running with a "different" Python
> version (my code should work with 3.0 and 2.6, so tests might work; for
> some reason I can no longer remember I disabled that).


Thanks for the update; the thing's a bit complicated because Sphinx
is based on docutils and docutils makes heavy use of except ABC, def
and of unicode strings. I tried hand-changing it briefly but it all
got a bit cumbersome. Maybe 2to3 will work ok. For now, tho', I've
switched to using 2.x to generate and all is well.

TJG

From python at rcn.com  Thu Mar 12 17:22:23 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 12 Mar 2009 09:22:23 -0700
Subject: [Python-Dev] [Python-ideas] Rough draft: Proposed
	formatspecifier for a	thousands separator (discussion moved
	frompython-dev)
References: <56EE433347824B66BC1D5FF4E3F47A2C@RaymondLaptop1><49B8D702.4040004@trueblade.com>
	<1F032CDC26874991B78E47DBDB333917@RaymondLaptop1>
Message-ID: <B261D4CF204F44C5AB6FBF073329E7F4@RaymondLaptop1>

Fixed typo in the example with spaces and commas.
Discussion draft at: http://www.python.org/dev/peps/pep-0378/

From barry at python.org  Thu Mar 12 17:26:44 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 12 Mar 2009 12:26:44 -0400
Subject: [Python-Dev] py: urls, new bazaar plugin available
Message-ID: <FFBA9E3F-0392-4D00-848D-1184E85A04BC@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Bazaar users!

There's a new Bazaar plugin you can use to more easily access read- 
only or read-write branches on code.python.org.  This plugin provides  
the 'py:' url prefix.  For example, to get the trunk branch with the  
plugin installed, you can now do:

     bzr branch py:trunk

or to get the 2.6 branch you can do:

     bzr branch py:2.6

You can also use this to get user branches, e.g. my email rewrite  
branch:

     bzr branch py:~barry/30email

If you have write access to branches on code.python.org, you can  
either set the environment variable $PYDEV_USER or the Bazaar  
configuration option pydev_user (the value doesn't matter) to get bzr 
+ssh access instead of the standard http access.  py: works both for  
branching and pushing.

Thanks to Karl Fogel for the implementation.  You'll need Karl's pydev  
plugin branch, and instructions on installing this are available here:

     http://tinyurl.com/aq55oc

I've updated the wiki page with additional details:

     http://wiki.python.org/moin/Bazaar

Enjoy!
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSbk3xHEjvBPtnXfVAQIiVgQAt3GwmDSkFjog/mp4PxVKn/F6MQoEDa/A
0nNysiU2oEvUXDBWPlab2V53tqpZ/uvOS17hl7ZhlDe+Z2jUUYiCkH+Hfvpz5F9n
u0Gf+5dMA7GQkLhvOezu7r6ngu2mmBB84ZwAfX4tJM+bBuQEn+U3BuRspkDiCb0S
iZONBdHyk5g=
=Pb2v
-----END PGP SIGNATURE-----

From andrew at acooke.org  Thu Mar 12 16:27:13 2009
From: andrew at acooke.org (andrew cooke)
Date: Thu, 12 Mar 2009 12:27:13 -0300 (CLST)
Subject: [Python-Dev] Building Py3K branch docs with Sphinx
In-Reply-To: <49B8FFBA.5090708@timgolden.me.uk>
References: <49B8DFE7.3040403@timgolden.me.uk>
	<49B8FFBA.5090708@timgolden.me.uk>
Message-ID: <e44eaae0f887ccaa0c357c230062e940.squirrel@localhost>

Tim Golden wrote:
> Tim Golden wrote:
>> Can I ask which flavour of Sphinx is being used to build the py3k docs?
>> I've taken the naive approach of simply pulling the sources from
>> branches/py3k and then calling make checkout to fetch the appropriate
>> sources, but these are from http://svn.python.org/projects and are
>> the same for 2.x and 3.x (and don't work under 3.x).
>
>
> ... or I could just use an existing Python 2.x installation to build
> the 3.x docs. Obviously. (slaps forehead)

I asked about this on the Sphinx list a while back.  I didn't get any
response at the time, but checking now I see that a week later someone
(the author I assume) commented -
http://groups.google.com/group/sphinx-dev/browse_thread/thread/9a0286f5deeb2912/778a02c397295add

So it seems that there is no public solution until release 0.6, and that
you cannot be able to run doctests when running with a "different" Python
version (my code should work with 3.0 and 2.6, so tests might work; for
some reason I can no longer remember I disabled that).

Anyway, I generate docs for 3.x code using 2.x and it does work (without
doctests).

Andrew


From eric at trueblade.com  Thu Mar 12 19:11:40 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 12 Mar 2009 14:11:40 -0400
Subject: [Python-Dev] [Python-ideas] Rough draft: Proposed
 format	specifier for a	thousands separator (discussion moved
 from	python-dev)
In-Reply-To: <1F032CDC26874991B78E47DBDB333917@RaymondLaptop1>
References: <56EE433347824B66BC1D5FF4E3F47A2C@RaymondLaptop1>	<49B8D702.4040004@trueblade.com>
	<1F032CDC26874991B78E47DBDB333917@RaymondLaptop1>
Message-ID: <49B9505C.1050803@trueblade.com>

Raymond Hettinger wrote:
> Eric Smith pointed-out that these are already
> handled by the "n" specifier in the locale module (albiet only
> for integers).

It's supported by float, but it's just not very useful. For Decimal it's
unsupported. Maybe this isn't a distinction worth pointing out.

> Proposal I (from Nick Coghlan)
> ==============================
...
> [[fill]align][sign][#][0][width][,][.precision][type]

> Proposal II (to meet Antoine Pitrou's request)
> ==============================================
...
> [[fill]align][sign][#][0][width][T[tsep]][dsep precision][type]

I was going to suggest that since the locale name for this is 
"grouping", we use "G". But since we're not doing a general-purpose 
grouping implementation, I think "T" better says "we're doing thousands, 
not general grouping. Perhaps this should go in a rationale section if 
we opt for "T". Now that I think about it, "G" is already a valid type, 
so it wouldn't work, anyway.

>  format(1234, "8T,d")    -->     '   1,234'

For proposal 2, this case is unfortunate. Because for integers, there is
no decimal allowed in the mini-language (it's currently illegal to use
"8.1d"), you'd only ever add the thousands, but you'd always need the
"T". It would be nice to come up with a specification that would degrade
for integers such that "8,d" would give '   1,234'. Proposal 1 is much
nicer in that regard, although I definitely like the fact that the
actual characters used for DOT and COMMA can be specified with proposal 2.

Maybe you'd never really use "T,", since the comma is redundant, and 
you'd say:
  format(1234, "8Td")    -->     '   1,234'
in normal use. But "d" is also the default, so it just becomes:
  format(1234, "8T")     -->     '   1,234'

I like approach 2 in general. I'll give some thought to other, similar 
schemes which would allow "8," or "8,d" to work. I think people will 
write "8," and expect "   1,234", not an error.

Eric.

From martin at v.loewis.de  Thu Mar 12 20:15:15 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Mar 2009 20:15:15 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B91FAB.2010101@gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org>	<49B85D42.9080406@gmail.com>	<loom.20090312T010957-202@post.gmane.org>	<200903122254.21316.steve@pearwood.info>	<loom.20090312T122030-705@post.gmane.org>
	<49B91FAB.2010101@gmail.com>
Message-ID: <49B95F43.3000708@v.loewis.de>

> Something that doesn't require deterministicly named tempfiles was Ted
> T'so's explanation linked to earlier.
> 
> read data from important file
> modify data
> create tempfile
> write data to tempfile
> *sync tempfile to disk*
> mv tempfile to filename of important file
> 
> The sync is necessary to ensure that the data is written to the disk
> before the old file overwrites the new filename.

You still wouldn't use the tempfile module in that case. Instead, you
would create a regular file, with the name base on the name of the
important file.

Regards,
Martin

From martin at v.loewis.de  Thu Mar 12 20:18:11 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Mar 2009 20:18:11 +0100
Subject: [Python-Dev] Building Py3K branch docs with Sphinx
In-Reply-To: <49B8DFE7.3040403@timgolden.me.uk>
References: <49B8DFE7.3040403@timgolden.me.uk>
Message-ID: <49B95FF3.9000407@v.loewis.de>

> Can I ask which flavour of Sphinx is being used to build the py3k docs?

The proper procedure to build the documentation is

make update
make htmlhelp #say

Regards,
Martin

From jimjjewett at gmail.com  Thu Mar 12 20:47:38 2009
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 12 Mar 2009 15:47:38 -0400
Subject: [Python-Dev] wait time [was: Ext4 data loss]
Message-ID: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>

It is starting to look as though flush (and close?) should take an
optional wait parameter, to indicate how much re-assurance you're
willing to wait for.

It also looks like we can't know enough to predict all sensible
symbolic constants -- so instead use a floating point numeric value.

f.flush(wait=0)  ==> current behavior
f.flush(wait=1)  ==> Do everything you can.  On a Mac, this would
apparently mean (everything up to and including) fcntl(fd, F_FULLSYNC)

f.flush(wait=0.5) ==> somewhere in between, depending on the operating
system and file system and disk drive and other stuff the devoloper
won't know in advance.

The exact interpretation of intermediate values might depend on the
installation or even change over time; the only invariant would be
that higher values are at least as safe, and lower values are at least
as fast.

-jJ

From martin at v.loewis.de  Thu Mar 12 20:56:03 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Mar 2009 20:56:03 +0100
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>
Message-ID: <49B968D3.7060807@v.loewis.de>

> It is starting to look as though flush (and close?) should take an
> optional wait parameter, to indicate how much re-assurance you're
> willing to wait for.

Unfortunately, such a thing would be unimplementable on most of today's
operating systems.

Regards,
Martin

From greg.ewing at canterbury.ac.nz  Thu Mar 12 21:10:31 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 13 Mar 2009 09:10:31 +1300
Subject: [Python-Dev] Formatting mini-language suggestion
In-Reply-To: <49B86015.2030203@gmail.com>
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
	<49B86015.2030203@gmail.com>
Message-ID: <49B96C37.8030801@canterbury.ac.nz>

Nick Coghlan wrote:

>   [[fill]align][sign][#][0][minimumwidth][,sep][.precision][type]
> 
> 'sep' is the new field that defines the thousands separator.

Wouldn't it be better to use a locale setting for this,
instead of having to specify it in every format string?
If an app is using a particular thousands separator in
one place, it will probably want to use it everywhere.

-- 
Greg

From rdmurray at bitdance.com  Thu Mar 12 21:16:32 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 12 Mar 2009 16:16:32 -0400 (EDT)
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <49B968D3.7060807@v.loewis.de>
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>
	<49B968D3.7060807@v.loewis.de>
Message-ID: <Pine.LNX.4.64.0903121608590.2596@kimball.webabinitio.net>

On Thu, 12 Mar 2009 at 20:56, "Martin v. L?wis" wrote:
>> It is starting to look as though flush (and close?) should take an
>> optional wait parameter, to indicate how much re-assurance you're
>> willing to wait for.
>
> Unfortunately, such a thing would be unimplementable on most of today's
> operating systems.

I read Jim's suggestion as a way to indicate the strength of the desire
of the application programmer for certainty, not as a time value.
In other words, 0.0 would map to 'just flush it', 0.5 might map to
'fsync', and 1.0 map to OS-X's "tell the disk to flush its buffers' call.

Assuming I'm right, I don't like the proposal.  It feels too squishy:
the semantics are not well defined.

By the way, I would not like to see python programmers encouraged to make
the same mistake that sqlite3 made.  The decision about how aggressive
to be on flushing data to disk should be in the hands of the _user_, not
the application.  Of course, the application needs some way to enable
the user to make that decision, which is what I presume we are talking
about supporting here.

--
R. David Murray           http://www.bitdance.com

From greg.ewing at canterbury.ac.nz  Thu Mar 12 21:18:55 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 13 Mar 2009 09:18:55 +1300
Subject: [Python-Dev] Formatting mini-language suggestion
In-Reply-To: <E64B88F5-ABC7-492E-A19D-818C523EEA61@fuhm.net>
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
	<49B86015.2030203@gmail.com>
	<E64B88F5-ABC7-492E-A19D-818C523EEA61@fuhm.net>
Message-ID: <49B96E2F.4080503@canterbury.ac.nz>

James Y Knight wrote:

> You might be interested to know that in India, the commas don't come  
> every 3 digits. In india, they come every two digits, after the first  
> three. Thus one billion = 1,00,00,00,000. How are you gonna represent  
> *that* in a formatting mini-language? :)

We outsource it. Send the number by email to a service centre
in India, where an employee formats it for us and sends it
back.

-- 
Greg

From solipsis at pitrou.net  Thu Mar 12 21:25:59 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 12 Mar 2009 20:25:59 +0000 (UTC)
Subject: [Python-Dev] wait time [was: Ext4 data loss]
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>
	<49B968D3.7060807@v.loewis.de>
	<Pine.LNX.4.64.0903121608590.2596@kimball.webabinitio.net>
Message-ID: <loom.20090312T202311-538@post.gmane.org>

R. David Murray <rdmurray <at> bitdance.com> writes:
> 
> By the way, I would not like to see python programmers encouraged to make
> the same mistake that sqlite3 made.  The decision about how aggressive
> to be on flushing data to disk should be in the hands of the _user_, not
> the application.

I disagree. The user usually does not know which kind of flushing is needed in
order for his data to be safe. Actually, he probably doesn't even know what
flushing means, and that files are ever "closed".

However, I also think that any parameter to flush() or close() is a bad idea,
since it can't be used when flushing and closing is implicit. For example when
the file is used in a "with" statement.



From rdmurray at bitdance.com  Thu Mar 12 21:50:05 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 12 Mar 2009 16:50:05 -0400 (EDT)
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <loom.20090312T202311-538@post.gmane.org>
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>
	<49B968D3.7060807@v.loewis.de>
	<Pine.LNX.4.64.0903121608590.2596@kimball.webabinitio.net>
	<loom.20090312T202311-538@post.gmane.org>
Message-ID: <Pine.LNX.4.64.0903121638290.2596@kimball.webabinitio.net>

On Thu, 12 Mar 2009 at 20:25, Antoine Pitrou wrote:
> R. David Murray <rdmurray <at> bitdance.com> writes:
>>
>> By the way, I would not like to see python programmers encouraged to make
>> the same mistake that sqlite3 made.  The decision about how aggressive
>> to be on flushing data to disk should be in the hands of the _user_, not
>> the application.
>
> I disagree. The user usually does not know which kind of flushing is needed in
> order for his data to be safe. Actually, he probably doesn't even know what
> flushing means, and that files are ever "closed".

Let me try some examples.

Suppose I'm running my applications on a laptop and I don't want the
disk to be spinning continually while I work.  I'm willing to take the
risk of data loss in order to extend my battery life.

And then there's the high performance server situation, where all the
hardware is at least double redundancy, and we want the fastest disk
performance possible, with data reliability being taken care of by
the redundancy in the system.  (Is this actually possible with today's
hardware and software?  I don't know, but it _should_ be.)

In between there is the medium to low performance, non-redundant server,
where we are willing to trade performance for data integrity.

In all three of these situations I might be running the exact same
application software.

So, the user needs to be in control.  Of course, for users who don't
understand the tradeoffs, there should be a sane default.

Oh, and the user doesn't need to understand flushing, they just
need to be in control of the performance versus data-integrity-
in-the-face-of-crashes tradeoff.

--
R. David Murray           http://www.bitdance.com

From jimjjewett at gmail.com  Thu Mar 12 22:01:27 2009
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 12 Mar 2009 17:01:27 -0400
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <49B968D3.7060807@v.loewis.de>
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>
	<49B968D3.7060807@v.loewis.de>
Message-ID: <fb6fbf560903121401m7d53fdb3v49f1f3c376b5b0f1@mail.gmail.com>

On 3/12/09, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> It is starting to look as though flush (and close?) should take an
>> optional wait parameter, to indicate how much re-assurance you're
>> willing to wait for.

> Unfortunately, such a thing would be unimplementable on most of today's
> operating systems.

What am I missing?

_file=file
class file(_file): ...
    def flush(self, wait=0):
        super().flush(self)
        if wait < 0.25:
            return
        if wait < 0.5 and os.fdatasync:
            os.fdatasync(self.fileno())
            return
        os.fsync(self.fileno())
        if wait < 0.75:
            return
        if os.ffullsync:
            os.ffullsync(self.fileno())

(To be honest, I'm not even seeing why it couldn't be done in
Objects/fileobject.c, though I realize extension modules would need to
go through the python interface to take advantage of it.)

-jJ

From martin at v.loewis.de  Thu Mar 12 22:09:38 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Mar 2009 22:09:38 +0100
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <Pine.LNX.4.64.0903121638290.2596@kimball.webabinitio.net>
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>	<49B968D3.7060807@v.loewis.de>	<Pine.LNX.4.64.0903121608590.2596@kimball.webabinitio.net>	<loom.20090312T202311-538@post.gmane.org>
	<Pine.LNX.4.64.0903121638290.2596@kimball.webabinitio.net>
Message-ID: <49B97A12.7080401@v.loewis.de>

> Let me try some examples.
> 
> Suppose I'm running my applications on a laptop and I don't want the
> disk to be spinning continually while I work.  I'm willing to take the
> risk of data loss in order to extend my battery life.

So when you select "Save" in your application, would you like the data
to be saved, or would you accept that they get lost? If the latter,
what kind of interaction would you perform with your application to
indicate that you *do* want the data to appear on disk?

Regards,
Martin

From rdmurray at bitdance.com  Thu Mar 12 22:14:04 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 12 Mar 2009 17:14:04 -0400 (EDT)
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <fb6fbf560903121401m7d53fdb3v49f1f3c376b5b0f1@mail.gmail.com>
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>
	<49B968D3.7060807@v.loewis.de>
	<fb6fbf560903121401m7d53fdb3v49f1f3c376b5b0f1@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0903121711570.2596@kimball.webabinitio.net>

On Thu, 12 Mar 2009 at 17:01, Jim Jewett wrote:
> On 3/12/09, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> It is starting to look as though flush (and close?) should take an
>>> optional wait parameter, to indicate how much re-assurance you're
>>> willing to wait for.
>
>> Unfortunately, such a thing would be unimplementable on most of today's
>> operating systems.
>
> What am I missing?

A less confusing name for your proposed parameter :)

Maybe 'reliability'?

> _file=file
> class file(_file): ...
>    def flush(self, wait=0):
>        super().flush(self)
>        if wait < 0.25:
>            return
>        if wait < 0.5 and os.fdatasync:
>            os.fdatasync(self.fileno())
>            return
>        os.fsync(self.fileno())
>        if wait < 0.75:
>            return
>        if os.ffullsync:
>            os.ffullsync(self.fileno())
>
> (To be honest, I'm not even seeing why it couldn't be done in
> Objects/fileobject.c, though I realize extension modules would need to
> go through the python interface to take advantage of it.)

From martin at v.loewis.de  Thu Mar 12 22:16:34 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Mar 2009 22:16:34 +0100
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <fb6fbf560903121401m7d53fdb3v49f1f3c376b5b0f1@mail.gmail.com>
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>	
	<49B968D3.7060807@v.loewis.de>
	<fb6fbf560903121401m7d53fdb3v49f1f3c376b5b0f1@mail.gmail.com>
Message-ID: <49B97BB2.7000608@v.loewis.de>

Jim Jewett wrote:
> On 3/12/09, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> It is starting to look as though flush (and close?) should take an
>>> optional wait parameter, to indicate how much re-assurance you're
>>> willing to wait for.
> 
>> Unfortunately, such a thing would be unimplementable on most of today's
>> operating systems.
> 
> What am I missing?

As somebody else remarked: I mistook your proposal for a "wait"
parameter to denote a time that you want to wait for the data to appear
on disk, specified, e.g., in seconds.

It didn't occur to me that it might be a unit-less unscaled value, which
I find an ugly API.

Regards,
Martin

From daniel at stutzbachenterprises.com  Thu Mar 12 22:38:57 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Thu, 12 Mar 2009 16:38:57 -0500
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <49B97A12.7080401@v.loewis.de>
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>
	<49B968D3.7060807@v.loewis.de>
	<Pine.LNX.4.64.0903121608590.2596@kimball.webabinitio.net>
	<loom.20090312T202311-538@post.gmane.org>
	<Pine.LNX.4.64.0903121638290.2596@kimball.webabinitio.net>
	<49B97A12.7080401@v.loewis.de>
Message-ID: <eae285400903121438j45f69dd9t1e62b4bf8539fedb@mail.gmail.com>

On Thu, Mar 12, 2009 at 4:09 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> So when you select "Save" in your application, would you like the data
> to be saved, or would you accept that they get lost? If the latter,
> what kind of interaction would you perform with your application to
> indicate that you *do* want the data to appear on disk?
>

I accept that if the computer crashes at just the wrong moment as I click
Save, my changes will not actually be Saved.  No amount of diligence in the
implementation of close() can prevent that since the computer can crash
before the program calls close().

I oppose applications that lose or corrupt both my new save and my
*previous* save if the computer crashes at the wrong moment.  That would
cause me to lose not only my most recent changes (an inconvenience), but
also all the work I have ever done on the file (a major headache for anyone
who doesn't make regular backups).

However, defaulting to calling fsync() when closing a file will:
1) Cripple performance for the many applications that don't need it (e.g.,
temporary files)
2) Fail to prevent data loss for applications that use the
truncate-and-rewrite paradigm for saving

Consider the following example:

with open('mysavefile', 'w') as f:
    f.write(data)
    f.flush()
    os.fsync(f.fileno())
    f.close()

If the system crashes after the call to open(), but before the call to
fsync(), then both the old and the new mysavefile may be gone.

Since needing to safely replace a file with new data is a moderately common
task, perhaps it would be useful to have a convenience class that looks like
a file, but takes care of the ugly details behind-the-scenes?  Something
vaguely like this flawed and untested class:

class open_for_safe_replacement(file): # needs a better name
    def __init__(self, path, flags):
        if 'w' not in flags:
            raise RuntimeError, 'Writing without writing?'
        self.path = path
        self.tmp_name =
some_function_that_generates_a_safe_temporary_filename() # good luck
        file.__init__(self.tmp_name, flags)

    def close(self):
        self.flush()
        os.fsync(self.fileno())
        self.close()
        os.rename(self.tmp_name, self.path) # won't work on Windows :-(

then we could simply:

with appropriate_module.open_for_safe_replacement('mysavefile', 'w'):
    f.write(data)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090312/9485fa1c/attachment.htm>

From a.badger at gmail.com  Thu Mar 12 22:38:28 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 12 Mar 2009 14:38:28 -0700
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B95F43.3000708@v.loewis.de>
References: <gp6hhq$hss$1@ger.gmane.org>	<49B85D42.9080406@gmail.com>	<loom.20090312T010957-202@post.gmane.org>	<200903122254.21316.steve@pearwood.info>	<loom.20090312T122030-705@post.gmane.org>
	<49B91FAB.2010101@gmail.com> <49B95F43.3000708@v.loewis.de>
Message-ID: <49B980D4.7080100@gmail.com>

Martin v. L?wis wrote:
>> Something that doesn't require deterministicly named tempfiles was Ted
>> T'so's explanation linked to earlier.
>>
>> read data from important file
>> modify data
>> create tempfile
>> write data to tempfile
>> *sync tempfile to disk*
>> mv tempfile to filename of important file
>>
>> The sync is necessary to ensure that the data is written to the disk
>> before the old file overwrites the new filename.
> 
> You still wouldn't use the tempfile module in that case. Instead, you
> would create a regular file, with the name base on the name of the
> important file.
> 
Uhm... why?  The requirements are:

1) lifetime of the temporary file is in control of the app
2) filename is available to the app so it can move it after data is written
3) temporary file can be created on the same filesystem as the important
file.

All of those are doable using the tempfile module.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090312/ddfbb0e5/attachment.pgp>

From cs at zip.com.au  Thu Mar 12 22:44:19 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 13 Mar 2009 08:44:19 +1100
Subject: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss
In-Reply-To: <49B97A12.7080401@v.loewis.de>
Message-ID: <20090312214419.GA29691@cskk.homeip.net>

On 12Mar2009 22:09, Martin v. L?wis <martin at v.loewis.de> wrote:
| > Let me try some examples.
| > Suppose I'm running my applications on a laptop and I don't want the
| > disk to be spinning continually while I work.  I'm willing to take the
| > risk of data loss in order to extend my battery life.
| 
| So when you select "Save" in your application, would you like the data
| to be saved, or would you accept that they get lost?

Often, I will accept that they get lost. Why? Because that will only
happen with and OS/hardware failure, and I expect those to be close to
never.

| If the latter,
| what kind of interaction would you perform with your application to
| indicate that you *do* want the data to appear on disk?

I don't. I type "sync" to a convenient shell prompt. On a UNIX OS, that
will not return until all outstanding data at the time of issuing the
command have been commited to disc.

As you can see, that places the timing in the hands of the user.  Where it
belongs, not impacting the performance of the system except at my own
command.

I speak as one who keeps his bogofilter spam datbase on a RAM disc
because bogofilter, too, is subject to atrocious sync overuse, since it
uses a database library that overuses sync. Testing shows at least one
and possibly more _orders_of_magnitude_ improvement in behaviour. Every
so often I copy the bogofilter db back to real disc.

The wholse point of a good OS on decent hardware is that one can commit
data to the _OS_, and trust that it will reach the disc in due course.
Fsync shows an app that doesn't trust the OS.

I hope you don't believe that handing the data to the disc drive
guarentees it has made it to the magnetic medium. It should do, but
the drive will probably acknowledge the data before the medium has
completed updating.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

Isaac Asimov once remarked that friends had chided him for not patenting the
electronic pocket calculator, since he wrote of similar devices back in the
1940's.  His reply, "Have you ever noticed I only described what it looked
like on the *outside*?" - ijl at mediaone.net

From tino at wildenhain.de  Thu Mar 12 22:35:48 2009
From: tino at wildenhain.de (Tino Wildenhain)
Date: Thu, 12 Mar 2009 22:35:48 +0100
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <fb6fbf560903121401m7d53fdb3v49f1f3c376b5b0f1@mail.gmail.com>
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>	<49B968D3.7060807@v.loewis.de>
	<fb6fbf560903121401m7d53fdb3v49f1f3c376b5b0f1@mail.gmail.com>
Message-ID: <49B98034.4030400@wildenhain.de>

Jim Jewett wrote:
> On 3/12/09, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> It is starting to look as though flush (and close?) should take an
>>> optional wait parameter, to indicate how much re-assurance you're
>>> willing to wait for.
> 
>> Unfortunately, such a thing would be unimplementable on most of today's
>> operating systems.
> 
> What am I missing?
> 
> _file=file
> class file(_file): ...
>     def flush(self, wait=0):
>         super().flush(self)
>         if wait < 0.25:
>             return
>         if wait < 0.5 and os.fdatasync:
>             os.fdatasync(self.fileno())
>             return
>         os.fsync(self.fileno())
>         if wait < 0.75:
>             return
>         if os.ffullsync:
>             os.ffullsync(self.fileno())
> 

What would be wrong with just making the f*sync calls
methods of the file object and that's about it?

alternatively when flush() should get an optional argument,
I'd call it sync and use a set of predefined and meaningful
constants (and no floating point value).

Just my 2ct.

Regards
Tino

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3241 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090312/ed79e208/attachment-0001.bin>

From martin at v.loewis.de  Thu Mar 12 22:54:04 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Mar 2009 22:54:04 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B980D4.7080100@gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org>	<49B85D42.9080406@gmail.com>	<loom.20090312T010957-202@post.gmane.org>	<200903122254.21316.steve@pearwood.info>	<loom.20090312T122030-705@post.gmane.org>	<49B91FAB.2010101@gmail.com>
	<49B95F43.3000708@v.loewis.de> <49B980D4.7080100@gmail.com>
Message-ID: <49B9847C.1030606@v.loewis.de>

>>> The sync is necessary to ensure that the data is written to the disk
>>> before the old file overwrites the new filename.
>> You still wouldn't use the tempfile module in that case. Instead, you
>> would create a regular file, with the name base on the name of the
>> important file.
>>
> Uhm... why?

Because it's much easier not to use the tempfile module, than to use it,
and because the main purpose of the tempfile module is irrelevant to
the specific application; the main purpose being the ability to
auto-delete the file when it gets closed.

Regards,
Martin

From martin at v.loewis.de  Thu Mar 12 22:57:06 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Mar 2009 22:57:06 +0100
Subject: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss
In-Reply-To: <20090312214419.GA29691@cskk.homeip.net>
References: <20090312214419.GA29691@cskk.homeip.net>
Message-ID: <49B98532.3040909@v.loewis.de>

Cameron Simpson wrote:
> On 12Mar2009 22:09, Martin v. L?wis <martin at v.loewis.de> wrote:
> | > Let me try some examples.
> | > Suppose I'm running my applications on a laptop and I don't want the
> | > disk to be spinning continually while I work.  I'm willing to take the
> | > risk of data loss in order to extend my battery life.
> | 
> | So when you select "Save" in your application, would you like the data
> | to be saved, or would you accept that they get lost?
> 
> Often, I will accept that they get lost. Why? Because that will only
> happen with and OS/hardware failure, and I expect those to be close to
> never.

I think you are an atypical user, then. People can accept that data is
lost if the machine crashes at the moment of saving. They get certainly
puzzled if the data is lost if the machine crashes 30 seconds after they
have saved, and not even a backup copy is available anymore.

Regards,
Martin

From a.badger at gmail.com  Thu Mar 12 23:02:19 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 12 Mar 2009 15:02:19 -0700
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B9847C.1030606@v.loewis.de>
References: <gp6hhq$hss$1@ger.gmane.org>	<49B85D42.9080406@gmail.com>	<loom.20090312T010957-202@post.gmane.org>	<200903122254.21316.steve@pearwood.info>	<loom.20090312T122030-705@post.gmane.org>	<49B91FAB.2010101@gmail.com>
	<49B95F43.3000708@v.loewis.de> <49B980D4.7080100@gmail.com>
	<49B9847C.1030606@v.loewis.de>
Message-ID: <49B9866B.2000901@gmail.com>

Martin v. L?wis wrote:
>>>> The sync is necessary to ensure that the data is written to the disk
>>>> before the old file overwrites the new filename.
>>> You still wouldn't use the tempfile module in that case. Instead, you
>>> would create a regular file, with the name base on the name of the
>>> important file.
>>>
>> Uhm... why?
> 
> Because it's much easier not to use the tempfile module, than to use it,
> and because the main purpose of the tempfile module is irrelevant to
> the specific application; the main purpose being the ability to
> auto-delete the file when it gets closed.
> 
auto-delete is one of the nice features of tempfile.  Another feature
which is entirely appropriate to this usage, though, though, is creation
of a non-conflicting filename.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090312/14be757d/attachment.pgp>

From mail at timgolden.me.uk  Thu Mar 12 23:08:06 2009
From: mail at timgolden.me.uk (Tim Golden)
Date: Thu, 12 Mar 2009 22:08:06 +0000
Subject: [Python-Dev] Building Py3K branch docs with Sphinx
In-Reply-To: <49B95FF3.9000407@v.loewis.de>
References: <49B8DFE7.3040403@timgolden.me.uk> <49B95FF3.9000407@v.loewis.de>
Message-ID: <49B987C6.7040801@timgolden.me.uk>

Martin v. L?wis wrote:
>> Can I ask which flavour of Sphinx is being used to build the py3k docs?
> 
> The proper procedure to build the documentation is
> 
> make update
> make htmlhelp #say


I think you misunderstood my question. I can build the docs
for 2.x, say -- have done so, in fact many times. 
I was simply trying to use Python 3.x itself
to build the docs for Python 3.x, not realising at first
that Sphinx (and docutils, Jinja etc.) won't actually
run under 3.x. 

Of course, as I later realised, I can just build them with
an existing 2.x install. I think I was sort of hoping to
have it produce its own dogfood, so to speak :)

TJG

From martin at v.loewis.de  Thu Mar 12 23:22:24 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Mar 2009 23:22:24 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B9866B.2000901@gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org>	<49B85D42.9080406@gmail.com>	<loom.20090312T010957-202@post.gmane.org>	<200903122254.21316.steve@pearwood.info>	<loom.20090312T122030-705@post.gmane.org>	<49B91FAB.2010101@gmail.com>
	<49B95F43.3000708@v.loewis.de> <49B980D4.7080100@gmail.com>
	<49B9847C.1030606@v.loewis.de> <49B9866B.2000901@gmail.com>
Message-ID: <49B98B20.8070000@v.loewis.de>

> auto-delete is one of the nice features of tempfile.  Another feature
> which is entirely appropriate to this usage, though, though, is creation
> of a non-conflicting filename.

Ok. In that use case, however, it is completely irrelevant whether the
tempfile module calls fsync. After it has generated the non-conflicting
filename, it's done.

Regards,
Martin


From leif.walsh at gmail.com  Thu Mar 12 23:35:22 2009
From: leif.walsh at gmail.com (Leif Walsh)
Date: Thu, 12 Mar 2009 18:35:22 -0400
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <loom.20090312T202311-538@post.gmane.org>
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>
	<49B968D3.7060807@v.loewis.de>
	<Pine.LNX.4.64.0903121608590.2596@kimball.webabinitio.net>
	<loom.20090312T202311-538@post.gmane.org>
Message-ID: <1236897322.16437.1.camel@swarley>

On Thu, 2009-03-12 at 20:25 +0000, Antoine Pitrou wrote:
> I disagree. The user usually does not know which kind of flushing is needed in
> order for his data to be safe. Actually, he probably doesn't even know what
> flushing means, and that files are ever "closed".
> 
> However, I also think that any parameter to flush() or close() is a bad idea,
> since it can't be used when flushing and closing is implicit. For example when
> the file is used in a "with" statement.

Perhaps this is an argument that the "synciness" of a file should be
defined when it is opened?  This doesn't give very much control to the
programmer, but it certainly seems easy to use correctly.

-- 
Cheers,
Leif
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090312/8c6ffd64/attachment.pgp>

From martin at v.loewis.de  Thu Mar 12 23:47:09 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Mar 2009 23:47:09 +0100
Subject: [Python-Dev] Building Py3K branch docs with Sphinx
In-Reply-To: <49B987C6.7040801@timgolden.me.uk>
References: <49B8DFE7.3040403@timgolden.me.uk> <49B95FF3.9000407@v.loewis.de>
	<49B987C6.7040801@timgolden.me.uk>
Message-ID: <49B990ED.3050408@v.loewis.de>

Tim Golden wrote:
> Martin v. L?wis wrote:
>>> Can I ask which flavour of Sphinx is being used to build the py3k docs?
>>
>> The proper procedure to build the documentation is
>>
>> make update
>> make htmlhelp #say
> 
> 
> I think you misunderstood my question. I can build the docs
> for 2.x, say -- have done so, in fact many times. I was simply trying to
> use Python 3.x itself
> to build the docs for Python 3.x, not realising at first
> that Sphinx (and docutils, Jinja etc.) won't actually
> run under 3.x.
> Of course, as I later realised, I can just build them with
> an existing 2.x install. I think I was sort of hoping to
> have it produce its own dogfood, so to speak :)

I still think my answer would have helped. It says that
a) you don't need to pull any other magic sphinx version from somewhere;
   the one that the Makefile fetches works just fine, and
b) it uses "python" from the PATH; *not* the ../python which you just
   built.
c) build.bat cannot be trusted to work (as you have found)

Regards,
Martin

From a.badger at gmail.com  Thu Mar 12 23:57:13 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 12 Mar 2009 15:57:13 -0700
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B98B20.8070000@v.loewis.de>
References: <gp6hhq$hss$1@ger.gmane.org>	<49B85D42.9080406@gmail.com>	<loom.20090312T010957-202@post.gmane.org>	<200903122254.21316.steve@pearwood.info>	<loom.20090312T122030-705@post.gmane.org>	<49B91FAB.2010101@gmail.com>
	<49B95F43.3000708@v.loewis.de> <49B980D4.7080100@gmail.com>
	<49B9847C.1030606@v.loewis.de> <49B9866B.2000901@gmail.com>
	<49B98B20.8070000@v.loewis.de>
Message-ID: <49B99349.4060702@gmail.com>

Martin v. L?wis wrote:
>> auto-delete is one of the nice features of tempfile.  Another feature
>> which is entirely appropriate to this usage, though, though, is creation
>> of a non-conflicting filename.
> 
> Ok. In that use case, however, it is completely irrelevant whether the
> tempfile module calls fsync. After it has generated the non-conflicting
> filename, it's done.
>
If you're saying that it shouldn't call fsync automatically I'll agree
to that.  The message thread I was replying to seemed to say that
tempfiles didn't need to support fsync because they will be useless
after a system crash.  I'm just refuting that by showing that it is
useful to call fsync on tempfiles as one of the steps in preserving the
data in another file.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090312/9e4bcbe4/attachment.pgp>

From amk at amk.ca  Fri Mar 13 00:29:00 2009
From: amk at amk.ca (A.M. Kuchling)
Date: Thu, 12 Mar 2009 19:29:00 -0400
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <loom.20090312T202311-538@post.gmane.org>
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>
	<49B968D3.7060807@v.loewis.de>
	<Pine.LNX.4.64.0903121608590.2596@kimball.webabinitio.net>
	<loom.20090312T202311-538@post.gmane.org>
Message-ID: <20090312232900.GA21843@amk.local>

On Thu, Mar 12, 2009 at 08:25:59PM +0000, Antoine Pitrou wrote:
> However, I also think that any parameter to flush() or close() is a bad idea,
> since it can't be used when flushing and closing is implicit. For example when
> the file is used in a "with" statement.

I think the existing os.fsync() and O_SYNC functionality is fine for
new applications and packages to write data securely or not.  We
should just consider whether the stdlib APIs don't make it impossible
to write data securely, e.g. dumbdbm's internal file object, and if
so, is it worth fixing?

--amk

From martin.hellwig at dcuktec.org  Fri Mar 13 00:45:39 2009
From: martin.hellwig at dcuktec.org (Martin P. Hellwig)
Date: Thu, 12 Mar 2009 23:45:39 +0000
Subject: [Python-Dev] =?windows-1252?q?=5BANN=5D_EuroPython_2009_=96_Extra?=
 =?windows-1252?q?_Early_Bird_registration_ends_this_Saturday!?=
Message-ID: <49B99EA3.409@dcuktec.org>

Newsflash!
A large number of Pythoneers has signed up already, for this reason
alone it is worth booking!

If you already know you are joining the conference, why not save some
money in these financially uncertain times and take advantage of the
extra early bird rate!

The extra early bird rate is just 95 GBP for the conference (70 GBP
for the tutorials) and ends this Saturday 14th of March.

You can book your conference and hotel all at once. Register at
http://www.europython.eu/registration/ .

The talks submitted so far promise to be very interesting and practical.
We have room for more though, go to
http://www.europython.eu/talks/cfp/ for this year's themes, and
submissions criteria, the deadline is on 5th April 2009.

Sponsors
A unique opportunity to affiliate with the prestigious EuroPython 
conference!
http://www.europython.eu/sponsors/

Spread the Word
Improve our publicity by distributing this announcement in your corner
of the community, coordinating this with the organisers is highly
appreciated. http://www.europython.eu/contact/

General Information
For more information about the conference, please visit
http://www.europython.eu/ .

Looking forward to seeing you!

From rdmurray at bitdance.com  Fri Mar 13 01:10:10 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 12 Mar 2009 20:10:10 -0400 (EDT)
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <49B97A12.7080401@v.loewis.de>
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>
	<49B968D3.7060807@v.loewis.de>
	<Pine.LNX.4.64.0903121608590.2596@kimball.webabinitio.net>
	<loom.20090312T202311-538@post.gmane.org>
	<Pine.LNX.4.64.0903121638290.2596@kimball.webabinitio.net>
	<49B97A12.7080401@v.loewis.de>
Message-ID: <Pine.LNX.4.64.0903122003290.2596@kimball.webabinitio.net>

On Thu, 12 Mar 2009 at 22:09, "Martin v. L?wis" wrote:
>> Let me try some examples.
>>
>> Suppose I'm running my applications on a laptop and I don't want the
>> disk to be spinning continually while I work.  I'm willing to take the
>> risk of data loss in order to extend my battery life.
>
> So when you select "Save" in your application, would you like the data
> to be saved, or would you accept that they get lost? If the latter,
> what kind of interaction would you perform with your application to
> indicate that you *do* want the data to appear on disk?

I accept that if I have told my laptop to only sync to disk every five
minutes (as I have at times done), and it crashes (eg: the battery runs
out), then anything I did during those last five minutes will be lost.
If the disk then spins up more often than I told it to, I get very
annoyed.

--
R. David Murray           http://www.bitdance.com

From steve at pearwood.info  Fri Mar 13 01:18:32 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 13 Mar 2009 11:18:32 +1100
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <fb6fbf560903121401m7d53fdb3v49f1f3c376b5b0f1@mail.gmail.com>
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>
	<49B968D3.7060807@v.loewis.de>
	<fb6fbf560903121401m7d53fdb3v49f1f3c376b5b0f1@mail.gmail.com>
Message-ID: <200903131118.32338.steve@pearwood.info>

On Fri, 13 Mar 2009 08:01:27 am Jim Jewett wrote:
> On 3/12/09, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> >> It is starting to look as though flush (and close?) should take an
> >> optional wait parameter, to indicate how much re-assurance you're
> >> willing to wait for.
> >
> > Unfortunately, such a thing would be unimplementable on most of
> > today's operating systems.
>
> What am I missing?
>
> _file=file
> class file(_file): ...
>     def flush(self, wait=0):
>         super().flush(self)
>         if wait < 0.25:
>             return
>         if wait < 0.5 and os.fdatasync:
>             os.fdatasync(self.fileno())
>             return

[snip rest of function]

Why are you giving the user the illusion of fine control by making the 
wait parameter a continuous variable and then using it as if it were a 
discrete variable? Your example gives only four distinct behaviours, 
for a (effectively) infinite range of wait. This is bad interface 
design: it misleads people into thinking that wait=0.4 is 33% safer 
than wait=0.3 when in fact they are exactly the same.

So, replace the wait parameter with a discrete variable -- named or 
numeric constants. That's a little better, but I still don't think this 
is the right solution. I believe that we want to leave the foundations 
as they are now, or at least don't rush into making changes to them.

A better approach in my opinion is to leave file as-is (although I 
wouldn't object much to it growing a sync method, for convenience) and 
then providing subclasses with the desired behaviour. That scales much 
better: today we can think of three or four levels of "save 
reliability" (corresponding to your 0.25, 0.5, 0.7 and 1 values for 
wait) but next year we might think of six, or ten. Instead of 
overloading the file type with all these different sorts of behaviour, 
requiring who knows how many arguments and a complicated API, we leave 
file nice and simple and allow the application developer to choose the 
subclass she wants:

from filetools import SyncOnWrite as open
f = open('mydata.txt', 'w')
f.write(data)

The choice of which subclass gets used is up to the application, but 
naturally that might be specified by a user-configurable setting.



-- 
Steven D'Aprano

From steve at pearwood.info  Fri Mar 13 01:21:16 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 13 Mar 2009 11:21:16 +1100
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <loom.20090312T202311-538@post.gmane.org>
References: <fb6fbf560903121247h412069bcr1f5f1704f6fe3886@mail.gmail.com>
	<Pine.LNX.4.64.0903121608590.2596@kimball.webabinitio.net>
	<loom.20090312T202311-538@post.gmane.org>
Message-ID: <200903131121.17024.steve@pearwood.info>

On Fri, 13 Mar 2009 07:25:59 am Antoine Pitrou wrote:
> R. David Murray <rdmurray <at> bitdance.com> writes:
> > By the way, I would not like to see python programmers encouraged
> > to make the same mistake that sqlite3 made. ?The decision about how
> > aggressive to be on flushing data to disk should be in the hands of
> > the _user_, not the application.
>
> I disagree. The user usually does not know which kind of flushing is
> needed in order for his data to be safe. Actually, he probably
> doesn't even know what flushing means, and that files are ever
> "closed".

Surely it depends on what sort of user you're talking about, and that is 
often application or OS specific. As a sweeping generalization, Mac 
users may be more tolerant of slow saves and less tolerant of data loss 
than Windows users, laptop/notebook users will probably expect the app 
to honour whatever setting they put in regarding HDD behaviour, and 
Linux users may expect more fine control over application behaviour and 
be willing to edit config files to get it.



-- 
Steven D'Aprano

From rdmurray at bitdance.com  Fri Mar 13 01:27:54 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 12 Mar 2009 20:27:54 -0400 (EDT)
Subject: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss
In-Reply-To: <49B98532.3040909@v.loewis.de>
References: <20090312214419.GA29691@cskk.homeip.net>
	<49B98532.3040909@v.loewis.de>
Message-ID: <Pine.LNX.4.64.0903122010410.2596@kimball.webabinitio.net>

On Thu, 12 Mar 2009 at 22:57, "Martin v. L?wis" wrote:
> Cameron Simpson wrote:
>> On 12Mar2009 22:09, Martin v. L?wis <martin at v.loewis.de> wrote:
>> | > Let me try some examples.
>> | > Suppose I'm running my applications on a laptop and I don't want the
>> | > disk to be spinning continually while I work.  I'm willing to take the
>> | > risk of data loss in order to extend my battery life.
>> |
>> | So when you select "Save" in your application, would you like the data
>> | to be saved, or would you accept that they get lost?
>>
>> Often, I will accept that they get lost. Why? Because that will only
>> happen with and OS/hardware failure, and I expect those to be close to
>> never.
>
> I think you are an atypical user, then. People can accept that data is
> lost if the machine crashes at the moment of saving. They get certainly
> puzzled if the data is lost if the machine crashes 30 seconds after they
> have saved, and not even a backup copy is available anymore.

The typical user is probably not all that surprised when Windows loses
their data.  They probably figure Windows took more than 30 seconds
to complete the save :) :)

Seriously, though, the point is that IMO an application should not be
calling fsync unless it provides a way for that behavior to be controlled
by the user.

--
R. David Murray           http://www.bitdance.com

From solipsis at pitrou.net  Fri Mar 13 01:35:58 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 13 Mar 2009 00:35:58 +0000 (UTC)
Subject: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss
References: <20090312214419.GA29691@cskk.homeip.net>
	<49B98532.3040909@v.loewis.de>
	<Pine.LNX.4.64.0903122010410.2596@kimball.webabinitio.net>
Message-ID: <loom.20090313T003418-37@post.gmane.org>

R. David Murray <rdmurray <at> bitdance.com> writes:
> 
> Seriously, though, the point is that IMO an application should not be
> calling fsync unless it provides a way for that behavior to be controlled
> by the user.

But whether an application does it or not is none of Python's business, is it?
What is the disagreement exactly?



From tleeuwenburg at gmail.com  Fri Mar 13 02:22:52 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Fri, 13 Mar 2009 12:22:52 +1100
Subject: [Python-Dev] Capability to alter issue metadata
Message-ID: <43c8685c0903121822o21539052l3518ae2086544ab9@mail.gmail.com>

Hi all,

I am continuing to look at issues in the issue tracker. It would be handy to
be able to update some of the metadata fields. For contributions, it's fine
to just be able to upload patches / post messages, but I can see any number
of issues which could use a bit of looking after.

e.g. http://bugs.python.org/issue4535 should probably be set to "pending
feedback"

I'd be happy to do this kind of thing if people are happy for me to do so...

-T

-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090313/0ec63195/attachment.htm>

From steve at pearwood.info  Fri Mar 13 02:29:44 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 13 Mar 2009 12:29:44 +1100
Subject: [Python-Dev] wait time [was: Ext4 data loss]
In-Reply-To: <Pine.LNX.4.64.0903122010410.2596@kimball.webabinitio.net>
References: <20090312214419.GA29691@cskk.homeip.net>
	<49B98532.3040909@v.loewis.de>
	<Pine.LNX.4.64.0903122010410.2596@kimball.webabinitio.net>
Message-ID: <200903131229.44733.steve@pearwood.info>

On Fri, 13 Mar 2009 11:27:54 am R. David Murray wrote:
> Seriously, though, the point is that IMO an application should not be
> calling fsync unless it provides a way for that behavior to be
> controlled by the user.

An admirable approach, but also a sweeping generalisation. Who is your 
expected user-base? Power users, who insist on being given the ability 
to configure every last aspect of the application behaviour? Or regular 
users who will be intimidated if you ask them to make the choice? Every 
configuration choice has a cost: not only does it require more effort 
to code and maintain, but it leads to a combinatorial explosion of test 
paths and greater opportunity for bugs. Why pay that cost if your 
application users won't consider the choice a feature? By all means 
give the user the option to make that choice, if they will consider it 
a feature.

The point is that these are *application* decisions, not *language* 
decisions. Python shouldn't be making those decisions, but should be 
enabling application developers to make them.



-- 
Steven D'Aprano

From ajaksu at gmail.com  Fri Mar 13 02:38:19 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Thu, 12 Mar 2009 22:38:19 -0300
Subject: [Python-Dev] Capability to alter issue metadata
In-Reply-To: <43c8685c0903121822o21539052l3518ae2086544ab9@mail.gmail.com>
References: <43c8685c0903121822o21539052l3518ae2086544ab9@mail.gmail.com>
Message-ID: <2d75d7660903121838x512ca4e6me3397831946dea23@mail.gmail.com>

Tennessee Leeuwenburg wrote:
> I am continuing to look at issues in the issue tracker. It would be handy to
> be able to update some of the metadata fields. For contributions, it's fine
> to just be able to upload patches / post messages, but I can see any number
> of issues which could use a bit of looking after.

I'm +1 to giving you Developer rights (but have no say in that). I'm
available to change any metadata you want until that happens.

BTW, R. David Murray is also interested in helping with the open
issues[1], so we could coordinate efforts at tracker-discuss.

> e.g. http://bugs.python.org/issue4535 should probably be set to "pending
> feedback"

Set to 'pending', 'pending feedback' is pending approval :)

> I'd be happy to do this kind of thing if people are happy for me to do so...

I am, thank you! :)

Regards,
Daniel

[1] http://mail.python.org/pipermail/tracker-discuss/2009-March/001914.html

From brett at python.org  Fri Mar 13 02:47:25 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 12 Mar 2009 18:47:25 -0700
Subject: [Python-Dev] Capability to alter issue metadata
In-Reply-To: <43c8685c0903121822o21539052l3518ae2086544ab9@mail.gmail.com>
References: <43c8685c0903121822o21539052l3518ae2086544ab9@mail.gmail.com>
Message-ID: <bbaeab100903121847k653404ccyff95ae3925874137@mail.gmail.com>

On Thu, Mar 12, 2009 at 18:22, Tennessee Leeuwenburg <tleeuwenburg at gmail.com
> wrote:

> Hi all,
>
> I am continuing to look at issues in the issue tracker. It would be handy
> to be able to update some of the metadata fields. For contributions, it's
> fine to just be able to upload patches / post messages, but I can see any
> number of issues which could use a bit of looking after.
>
> e.g. http://bugs.python.org/issue4535 should probably be set to "pending
> feedback"
>
> I'd be happy to do this kind of thing if people are happy for me to do
> so...


You have the Developer role.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090312/08ef7bb0/attachment.htm>

From cs at zip.com.au  Fri Mar 13 03:00:47 2009
From: cs at zip.com.au (Cameron Simpson)
Date: Fri, 13 Mar 2009 13:00:47 +1100
Subject: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss
In-Reply-To: <loom.20090313T003418-37@post.gmane.org>
Message-ID: <20090313020047.GA8432@cskk.homeip.net>

On 13Mar2009 00:35, Antoine Pitrou <solipsis at pitrou.net> wrote:
| R. David Murray <rdmurray <at> bitdance.com> writes:
| > Seriously, though, the point is that IMO an application should not be
| > calling fsync unless it provides a way for that behavior to be controlled
| > by the user.
| 
| But whether an application does it or not is none of Python's business, is it?
| What is the disagreement exactly?

When the app is written in python, it bears on python's business. The
dispute seems to me to be largely (a) should python libraries call
fsync() and the like on their own, and when (b) whether there should be
class methods to control this. For myself, the answer for (a) is broadly
no and for (b) preferably yes, in which case my answer to (a) becomes
"default to no fsyncness unless asked".

Then the behaviour of the app becomes something to criticise or not
and python can go in its was with a clear conscience.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au> DoD#743
http://www.cskk.ezoshosting.com/cs/

DRM: the functionality of refusing to function. - Richard Stallman

From rdmurray at bitdance.com  Fri Mar 13 03:02:26 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 12 Mar 2009 22:02:26 -0400 (EDT)
Subject: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss
In-Reply-To: <loom.20090313T003418-37@post.gmane.org>
References: <20090312214419.GA29691@cskk.homeip.net>
	<49B98532.3040909@v.loewis.de>
	<Pine.LNX.4.64.0903122010410.2596@kimball.webabinitio.net>
	<loom.20090313T003418-37@post.gmane.org>
Message-ID: <Pine.LNX.4.64.0903122200010.2596@kimball.webabinitio.net>

On Fri, 13 Mar 2009 at 00:35, Antoine Pitrou wrote:
> R. David Murray <rdmurray <at> bitdance.com> writes:
>>
>> Seriously, though, the point is that IMO an application should not be
>> calling fsync unless it provides a way for that behavior to be controlled
>> by the user.
>
> But whether an application does it or not is none of Python's business, is it?
> What is the disagreement exactly?

I'd like to see whatever feature gets added support the application
writer in making this user controllable, or at the very least document
that this to do so is best practice if you use the sync feature.

--
R. David Murray           http://www.bitdance.com

From steve at pearwood.info  Fri Mar 13 04:27:35 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 13 Mar 2009 14:27:35 +1100
Subject: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss
In-Reply-To: <Pine.LNX.4.64.0903122200010.2596@kimball.webabinitio.net>
References: <20090312214419.GA29691@cskk.homeip.net>
	<loom.20090313T003418-37@post.gmane.org>
	<Pine.LNX.4.64.0903122200010.2596@kimball.webabinitio.net>
Message-ID: <200903131427.35688.steve@pearwood.info>

On Fri, 13 Mar 2009 01:02:26 pm R. David Murray wrote:
> On Fri, 13 Mar 2009 at 00:35, Antoine Pitrou wrote:
> > R. David Murray <rdmurray <at> bitdance.com> writes:
> >> Seriously, though, the point is that IMO an application should not
> >> be calling fsync unless it provides a way for that behavior to be
> >> controlled by the user.
> >
> > But whether an application does it or not is none of Python's
> > business, is it? What is the disagreement exactly?
>
> I'd like to see whatever feature gets added support the application
> writer in making this user controllable, or at the very least
> document that this to do so is best practice if you use the sync
> feature.

It's not best practice. It may be best practice for a certain class of 
users and applications, e.g. those who value the ability to control 
low-level behaviour of the app, but it is poor practice for other 
classes of users and applications. Do you really think that having 
Minefield make the file syncing behaviour of the high scores file 
user-configurable is best practice? People care about their high 
scores, but they don't care that much.

It may even lead to more data loss than leaving it out:

* If the application chooses a specific strategy, this strategy might 
(for the sake of the argument) lead to data loss once in ten million 
writes on average.

* If the application makes this a configuration option, the increased 
complexity of writing the code, and the increased number of paths that 
need to be tested, may lead to bugs which cause data loss. This may be 
more risky than the original strategy above (whatever that happens to 
be.)

Complexity is not cost-free, and insisting that the more complex, 
expensive solution is always "best practice" is wrong.


-- 
Steven D'Aprano

From jimjjewett at gmail.com  Fri Mar 13 05:13:09 2009
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 13 Mar 2009 00:13:09 -0400
Subject: [Python-Dev] sure [was: Ext4 data loss]
Message-ID: <fb6fbf560903122113x7556bfbal6ff6e95a8279c5b@mail.gmail.com>

[new name instead of "wait" -- but certainty is too long, patience too
hard to spell, etc...]

>> class file(_file): ...
>>     def flush(self, sure=0):
>         super().flush(self)
>         if sure < 0.25:
>             return
>         if sure < 0.5 and os.fdatasync:
>             os.fdatasync(self.fileno())
> ...

Steven D'Aprano asked
> Why are you giving the user the illusion of fine control by making the
> wait parameter a continuous variable and then using it as if it were a
> discrete variable?

We don't know how many possible values there will be, or whether they
will be affected by environmental settings.  Developers will not
always know what sort of systems users will have, but they can
indicate (with a ratio) where in the range (slow+safe):(fast+risky)
they rate this particular flush.

Before this discussion, I knew about sync, but had not paid attention
even to datasync, let alone fullsync.  I have no idea which additional
options may be relevant in the future, or on smaller devices or other
storage media.

I do expect specific intermediate values (such as 0.3) to be
interpreted differently on a laptop than on a desktop.

-jJ

From rhamph at gmail.com  Fri Mar 13 05:14:41 2009
From: rhamph at gmail.com (Adam Olsen)
Date: Thu, 12 Mar 2009 22:14:41 -0600
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <gp6hhq$hss$1@ger.gmane.org>
References: <gp6hhq$hss$1@ger.gmane.org>
Message-ID: <aac2c7cb0903122114i22d89b9dx88c15176dce7d739@mail.gmail.com>

On Tue, Mar 10, 2009 at 2:11 PM, Christian Heimes <lists at cheimes.de> wrote:
> Multiple blogs and news sites are swamped with a discussion about ext4
> and KDE 4.0. Theodore Ts'o - the developer of ext4 - explains the issue
> at
> https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/317781/comments/54.
>
>
> Python's file type doesn't use fsync() and be the victim of the very
> same issue, too. Should we do anything about it?

It's a kernel defect and we shouldn't touch it.

Traditionally you were hooped regardless of what you did, just with
smaller windows.  Did you want to lose your file 50% of the time or
only 10% of the time?  Heck, 1% of the time you lose the *entire*
filesystem.

Along came journaling file systems.  They guarantee the filesystem
itself stays intact, but not your file.  Still, if you hedge your bets
it's a fairly small window.  In fact if you kill performance you can
eliminate the window: write to a new file, flush all the buffers, then
use the journaling filesystem to rename; few people do that though,
due to the insane performance loss.

What we really want is a simple memory barrier.  We don't need the
file to be saved *now*, just so long as it gets saved before the
rename does.  Unfortunately the filesystem APIs don't touch on this,
as they were designed when losing the entire filesystem was
acceptable.  What we need is a heuristic to make them work in this
scenario.  Lo and behold ext3's data=ordered did just that!

Personally, I consider journaling to be a joke without that.  It has
different justifications, but not this critical one.  Yet the ext4
developers didn't see it that way, so it was sacrificed to new
performance improvements (delayed allocation).

2.6.30 has patches lined up that will fix this use case, making sure
the file is written before the rename.  We don't have to touch it.

Of course if you're planning to use the file without renaming then you
probably do need an explicit fsync and an API for that might help
after all.  That's a different problem though, and has always existed.


-- 
Adam Olsen, aka Rhamphoryncus

From stephen at xemacs.org  Fri Mar 13 07:57:31 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 13 Mar 2009 15:57:31 +0900
Subject: [Python-Dev] Formatting mini-language suggestion
In-Reply-To: <49B96C37.8030801@canterbury.ac.nz>
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
	<49B86015.2030203@gmail.com> <49B96C37.8030801@canterbury.ac.nz>
Message-ID: <8763idlxk4.fsf@xemacs.org>

Greg Ewing writes:
 > Nick Coghlan wrote:
 > 
 > >   [[fill]align][sign][#][0][minimumwidth][,sep][.precision][type]
 > > 
 > > 'sep' is the new field that defines the thousands separator.
 > 
 > Wouldn't it be better to use a locale setting for this,
 > instead of having to specify it in every format string?

Maybe, but the POSIX locale concept is broken (because it's process-
global) in many contexts.  Viz.

 > If an app is using a particular thousands separator in
 > one place, it will probably want to use it everywhere.

Not if that app is internationalized (eg, a webapp that serves both
Americans and Chinese).

From lie.1296 at gmail.com  Fri Mar 13 08:31:59 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Fri, 13 Mar 2009 18:31:59 +1100
Subject: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss
In-Reply-To: <200903131427.35688.steve@pearwood.info>
References: <20090312214419.GA29691@cskk.homeip.net>	<loom.20090313T003418-37@post.gmane.org>	<Pine.LNX.4.64.0903122200010.2596@kimball.webabinitio.net>
	<200903131427.35688.steve@pearwood.info>
Message-ID: <gpd25g$fot$1@ger.gmane.org>

Steven D'Aprano wrote:
> On Fri, 13 Mar 2009 01:02:26 pm R. David Murray wrote:
>> On Fri, 13 Mar 2009 at 00:35, Antoine Pitrou wrote:
>>> R. David Murray <rdmurray <at> bitdance.com> writes:
>>>> Seriously, though, the point is that IMO an application should not
>>>> be calling fsync unless it provides a way for that behavior to be
>>>> controlled by the user.
>>> But whether an application does it or not is none of Python's
>>> business, is it? What is the disagreement exactly?
>> I'd like to see whatever feature gets added support the application
>> writer in making this user controllable, or at the very least
>> document that this to do so is best practice if you use the sync
>> feature.
> 
> It's not best practice. It may be best practice for a certain class of 
> users and applications, e.g. those who value the ability to control 
> low-level behaviour of the app, but it is poor practice for other 
> classes of users and applications. Do you really think that having 
> Minefield make the file syncing behaviour of the high scores file 
> user-configurable is best practice? People care about their high 
> scores, but they don't care that much.
> 
> It may even lead to more data loss than leaving it out:
> 
> * If the application chooses a specific strategy, this strategy might 
> (for the sake of the argument) lead to data loss once in ten million 
> writes on average.
> 
> * If the application makes this a configuration option, the increased 
> complexity of writing the code, and the increased number of paths that 
> need to be tested, may lead to bugs which cause data loss. This may be 
> more risky than the original strategy above (whatever that happens to 
> be.)
> 
> Complexity is not cost-free, and insisting that the more complex, 
> expensive solution is always "best practice" is wrong.

If the pops and moms uses a financial program and lost their only copy 
of 10 years worth of financial data, they'll simply be confused and 
that's it.

Meanwhile if a network administrator needs to squeeze the last bit of 
performance out of his backup script, he definitely would threaten the 
dev-team of the programming language to make manual sync file writing 
the default, since it makes it difficult for him to fine-tune the 
syncing process.


From phd at phd.pp.ru  Fri Mar 13 10:28:07 2009
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Fri, 13 Mar 2009 12:28:07 +0300
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <aac2c7cb0903122114i22d89b9dx88c15176dce7d739@mail.gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org>
	<aac2c7cb0903122114i22d89b9dx88c15176dce7d739@mail.gmail.com>
Message-ID: <20090313092807.GA27105@phd.pp.ru>

On Thu, Mar 12, 2009 at 10:14:41PM -0600, Adam Olsen wrote:
> Yet the ext4
> developers didn't see it that way, so it was sacrificed to new
> performance improvements (delayed allocation).

   Ext4 is not the only FS with delayed allocation. New XFS has it, btrfs
will have it. Don't know about other OS/FS (ZFS? NTFS?)

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From chris at simplistix.co.uk  Fri Mar 13 10:58:57 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 13 Mar 2009 09:58:57 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49B3CD53.5010906@v.loewis.de>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<49B3B2E8.4050605@simplistix.co.uk> <49B3CD53.5010906@v.loewis.de>
Message-ID: <49BA2E61.7070104@simplistix.co.uk>

Martin v. L?wis wrote:
>> In light of this, what I'd love to see (but sadly can't really help
>> with, and am not optimistic about happening) is for:
>>
>> - python to grow a decent, cross platform, package management system
>>
>> - the standard library to actually shrink to a point where only
>> libraries that are not released elsewhere are included
>>
>> I'd be interested to know how many users of python also felt this way ;-)
> 
> I don't like the standard library to shrink. It's good that batteries
> are included.

If a decent package management system *was* included, this wouldn't be 
an issue..

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Fri Mar 13 11:02:00 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 13 Mar 2009 10:02:00 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <gp0n10$lco$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>
	<gp0n10$lco$1@ger.gmane.org>
Message-ID: <49BA2F18.3000601@simplistix.co.uk>

Steve Holden wrote:
> Perhaps we could encourage more "jumbo" distributions, like Enthought's
> and ActiveState's. I suspect many people would rather be able to
> maintain their Python functionality as a single product.

I think you'll find it split..

People who use and love things like zc.buildout do so because they want 
to free package maintainers to do their own release cycles and not have 
individual packages held back by the need to release the "whole project" 
in one go.

However, yes, I'm sure there are just as many people who want to install 
"just one thing" and have it all there. (although they'll be sadly 
disappointed when they realise whatever it is they need (lxml, PIL, 
xlrd,xlwt) isn't there.

That said, a decent package management system in the core *and* the 
jumbo installers you mention would likely keep both camps happy ;-)

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Fri Mar 13 11:05:16 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 13 Mar 2009 10:05:16 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49B3CFFD.7050905@voidspace.org.uk>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>
	<49B3CD53.5010906@v.loewis.de> <49B3CFFD.7050905@voidspace.org.uk>
Message-ID: <49BA2FDC.9060805@simplistix.co.uk>

Michael Foord wrote:
> I have mixed feelings. It is great that the batteries are included, but 
> some batteries are showing their age or not maintained (who maintains 
> IDLE? - does the calendar module really warrant being in the standard 
> library? - imaplib is really not useful and IMAPClient which isn't in 
> the standard library is much better [1]).

Wow, interesting case in point ;-)
I actually stumbled into imaplib, found it not useful and gave up and 
solved the problem a completely different way. Had I not tripped over 
it, I might have found IMAPClient!

Chris
-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From greg.ewing at canterbury.ac.nz  Fri Mar 13 11:07:18 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 13 Mar 2009 23:07:18 +1300
Subject: [Python-Dev] Formatting mini-language suggestion
In-Reply-To: <8763idlxk4.fsf@xemacs.org>
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
	<49B86015.2030203@gmail.com> <49B96C37.8030801@canterbury.ac.nz>
	<8763idlxk4.fsf@xemacs.org>
Message-ID: <49BA3056.40000@canterbury.ac.nz>

Stephen J. Turnbull wrote:
> Greg Ewing writes:
>  > If an app is using a particular thousands separator in
>  > one place, it will probably want to use it everywhere.
> 
> Not if that app is internationalized (eg, a webapp that serves both
> Americans and Chinese).

I don't think you'll want to code the separators into
all your format strings in that case, either. You'll
want some sort of context that you set up for the
page you're about to serve.

-- 
Greg

From chris at simplistix.co.uk  Fri Mar 13 11:11:32 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Fri, 13 Mar 2009 10:11:32 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
Message-ID: <49BA3154.8080408@simplistix.co.uk>

Jim Jewett wrote:
>>> - python to grow a decent, cross platform, package management system
> 
> As stated, this may be impossible, because of the difference in what a
> package should mean on Windows vs Unix.
> 
> If you just mean a way to add python packages from pypi as with
> EasyInstall, then maybe.

I meant package in the python sense, which has a clear definition, 
unless I'm mistaken?

> In some environments, each new component must be approved.  Once
> python is approved, the standard library is OK, but adding 7 packages
> from pypi requires 7 more sets of approvals.

True, but as I mentioend elsewhere, I myself haven't done a python 
project where I only needed python and the standard lib for many years...

> That said, it may make sense to just give greater prominence to
> existing repackagers, such as ActiveState or Enthought.

Right, getting ActivePython or similar approved might be the way to go, 
but I'm betting it depends on the project...

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From p.f.moore at gmail.com  Fri Mar 13 11:47:25 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 13 Mar 2009 10:47:25 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BA2E61.7070104@simplistix.co.uk>
References: <49ABCFC9.1070508@gmail.com> <49AE9698.1020806@simplistix.co.uk>
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
	<gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<49B3B2E8.4050605@simplistix.co.uk> <49B3CD53.5010906@v.loewis.de>
	<49BA2E61.7070104@simplistix.co.uk>
Message-ID: <79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>

2009/3/13 Chris Withers <chris at simplistix.co.uk>:
> If a decent package management system *was* included, this wouldn't be an
> issue..

Remember that a "decent package management system" needs to handle
filling in all the forms and arranging approvals to get authorisation
for packages when you download them.

And no, I'm *not* joking. People in a locked-down corporate
environment really do benefit from just having to get the OK for
"Python", and then knowing that they have all they need.

Even ignoring the above, your "decent package management system" has
to deal with systems with no internet connectivity - just copying the
Python installer onto my pen drive to put on my Mum's laptop so I can
write some admin jobs for her, is a lot easier than having to pick and
choose from PyPI what to download before I start.

-1 on slimming down the stdlib.
(OTOH, I've no problem with seeing an improved package system - just
don't use it as an excuse to cripple the stdlib!)

Paul

From ncoghlan at gmail.com  Fri Mar 13 12:20:21 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 13 Mar 2009 21:20:21 +1000
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B98B20.8070000@v.loewis.de>
References: <gp6hhq$hss$1@ger.gmane.org>	<49B85D42.9080406@gmail.com>	<loom.20090312T010957-202@post.gmane.org>	<200903122254.21316.steve@pearwood.info>	<loom.20090312T122030-705@post.gmane.org>	<49B91FAB.2010101@gmail.com>	<49B95F43.3000708@v.loewis.de>
	<49B980D4.7080100@gmail.com>	<49B9847C.1030606@v.loewis.de>
	<49B9866B.2000901@gmail.com> <49B98B20.8070000@v.loewis.de>
Message-ID: <49BA4175.1070604@gmail.com>

Martin v. L?wis wrote:
>> auto-delete is one of the nice features of tempfile.  Another feature
>> which is entirely appropriate to this usage, though, though, is creation
>> of a non-conflicting filename.
> 
> Ok. In that use case, however, it is completely irrelevant whether the
> tempfile module calls fsync. After it has generated the non-conflicting
> filename, it's done.

I agree, but my comment was that it would be nice if better fsync
support (if added) could be done in such a way that it helped not only
file objects, but also *file-like* objects (such as the wrappers in the
tempfile module) without making the file-like API any fatter.

If that's not possible or practical so be it, but it is still something
to keep in mind when considering options.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From rdmurray at bitdance.com  Fri Mar 13 12:56:20 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 13 Mar 2009 07:56:20 -0400 (EDT)
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BA2E61.7070104@simplistix.co.uk>
References: <49ABCFC9.1070508@gmail.com> <49AE9698.1020806@simplistix.co.uk>
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
	<gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<49B3B2E8.4050605@simplistix.co.uk> <49B3CD53.5010906@v.loewis.de>
	<49BA2E61.7070104@simplistix.co.uk>
Message-ID: <Pine.LNX.4.64.0903130739170.2596@kimball.webabinitio.net>

On Fri, 13 Mar 2009 at 09:58, Chris Withers wrote:
> Martin v. L?wis wrote:
>> >  In light of this, what I'd love to see (but sadly can't really help
>> >  with, and am not optimistic about happening) is for:
>> > 
>> >  - python to grow a decent, cross platform, package management system
>> > 
>> >  - the standard library to actually shrink to a point where only
>> >  libraries that are not released elsewhere are included
>> > 
>> >  I'd be interested to know how many users of python also felt this way 
>> >  ;-)
>>
>>  I don't like the standard library to shrink. It's good that batteries
>>  are included.
>
> If a decent package management system *was* included, this wouldn't be an 
> issue..

I disagree.  One of the jobs I've had is release management for
internal software projects that depend on various external pieces.
Release integration tested against specific versions of those external
packages, and those were the packages that needed to wind up on the system
when the release was installed.  I've done systems depending on both perl
and python, and let me tell you, python is way, _way_ easier to manage.
With python, I have a dependency on a particular python version, and then
maybe one or two add on packages.  With perl, I have perl, and then I
have a gadzillion cpan modules.  I don't know how many a gadzillion is,
because what I wound up doing was making a local copy of the cpan archive,
checking that in to the repository, and writing up some scripts that made
sure I pulled the actual install from my cpan snapshot and supported the
developers in updating that snapshot when we were building a new version.
(Nor was that the only problem with perl....what idiot decided it was
OK to interactively prompt for things during a batch install process?!
And without providing any way to script the answers, at least that I
could find!)

So I'm +1 for keeping the Python stdlib as comprehensive as sensible.
(Please note that last word...I've no objection to pruning things
that are no longer serving a useful purpose, or that are better
managed outside the core.)

--
R. David Murray           http://www.bitdance.com

From ncoghlan at gmail.com  Fri Mar 13 12:57:06 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 13 Mar 2009 21:57:06 +1000
Subject: [Python-Dev] [Python-ideas] Rough draft: Proposed
 format	specifier for a	thousands separator (discussion moved
 from	python-dev)
In-Reply-To: <49B9505C.1050803@trueblade.com>
References: <56EE433347824B66BC1D5FF4E3F47A2C@RaymondLaptop1>	<49B8D702.4040004@trueblade.com>	<1F032CDC26874991B78E47DBDB333917@RaymondLaptop1>
	<49B9505C.1050803@trueblade.com>
Message-ID: <49BA4A12.4040402@gmail.com>

Eric Smith wrote:
> I like approach 2 in general. I'll give some thought to other, similar
> schemes which would allow "8," or "8,d" to work. I think people will
> write "8," and expect "   1,234", not an error.

Given that PEP 3101 deliberately exposed the underlying Formatter class
to support more flexible expansions to formatting, I think trying to
build that level of flexibility into the formatting mini-language itself
would be a mistake.

So step 1 would be to add the "," or "'" to the format string to
indicate the use of separators. While both violate the rule of syntax
not looking like grit on Tim's monitor, the latter has the virtues of
not only being harder to confuse with a period, but also being
consistent with the printf() grouping flag as defined in the Single Unix
Specification v2 (as noted by JYK earlier in the thread and described in
more detail at [1]).

Then step 2 (if deemed appropriate) would be to expose the mini-language
parser in a way that makes it easier to write a custom format_field()
method in a Formatter subclass without having to completely duplicate
the existing parser's functionality.

Step 1 makes sense even if step 2 is never taken. Step 2 also makes
sense even if step 1 is never taken. So I don't think the fact that step
2 is probably a good idea should be used to hold up PEP 378, nor do I
think PEP 378 should be complicated by locale related issues that are
better handled by making it easier to write a custom format_field()
method in a Formatter subclass.

Cheers,
Nick.

[1] http://linux.die.net/man/3/printf

P.S. This post is really just an elaboration of this comment from JYK:

"""I'm not against Raymond's proposal, just against doing a *bad* job of
making it work in multiple locales. Locale conventions can be complex,
and are going to be best represented outside the format string."""


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From rdmurray at bitdance.com  Fri Mar 13 13:17:02 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 13 Mar 2009 08:17:02 -0400 (EDT)
Subject: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss
In-Reply-To: <200903131427.35688.steve@pearwood.info>
References: <20090312214419.GA29691@cskk.homeip.net>
	<loom.20090313T003418-37@post.gmane.org>
	<Pine.LNX.4.64.0903122200010.2596@kimball.webabinitio.net>
	<200903131427.35688.steve@pearwood.info>
Message-ID: <Pine.LNX.4.64.0903130756260.2596@kimball.webabinitio.net>

On Fri, 13 Mar 2009 at 14:27, Steven D'Aprano wrote:
> On Fri, 13 Mar 2009 01:02:26 pm R. David Murray wrote:
>> On Fri, 13 Mar 2009 at 00:35, Antoine Pitrou wrote:
>>> R. David Murray <rdmurray <at> bitdance.com> writes:
>>>> Seriously, though, the point is that IMO an application should not
>>>> be calling fsync unless it provides a way for that behavior to be
>>>> controlled by the user.
>>>
>>> But whether an application does it or not is none of Python's
>>> business, is it? What is the disagreement exactly?
>>
>> I'd like to see whatever feature gets added support the application
>> writer in making this user controllable, or at the very least
>> document that this to do so is best practice if you use the sync
>> feature.
>
> It's not best practice. It may be best practice for a certain class of
> users and applications, e.g. those who value the ability to control
> low-level behaviour of the app, but it is poor practice for other
> classes of users and applications. Do you really think that having
> Minefield make the file syncing behaviour of the high scores file
> user-configurable is best practice? People care about their high
> scores, but they don't care that much.

Why would Minefield bother to use sync/fsync?  You will note that what
I suggested was that applications that _use the sync feature_ make
it user controllable.  And yes, it is Best Practice, because we have
glaring counterexamples (sqlite3, the one mentioned by another poster)
that require people to jump through hoops to compensate for the fact
that it isn't user controllable.

I thought that a significant majority of applications wouldn't have
to care.

If most applications do really have to care (ie: there's no way for them
to recover gracefully from a crash that trashes the new data they were
trying to save unless they call fsync), then my argument is a lot weaker.
And from the post that talked about the problems with rename, I gather
I was misunderstanding the extent of the problem.

Perhaps I can restrict my request: that it be noted that if you use fsync
_aggressively_ (ie: not just on a final save that happens infrequently)
that you make it user controllable in some fashion.  Note that that
user control could be as simple as being able to set the autosave
interval on an autosaving editor if the autosave needs to do an fsync
for safety.

Thinking about it further, I think what I'm really looking for is a
warning to applications developers that using fsync can have significant
performance penalties (and therefore if they use it a lot they should
make it configurable), and can make laptop users hate you if it gets
used any time other than at a user requested save point :)

--
R. David Murray           http://www.bitdance.com

From solipsis at pitrou.net  Fri Mar 13 13:27:12 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 13 Mar 2009 12:27:12 +0000 (UTC)
Subject: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss
References: <20090312214419.GA29691@cskk.homeip.net>
	<loom.20090313T003418-37@post.gmane.org>
	<Pine.LNX.4.64.0903122200010.2596@kimball.webabinitio.net>
	<200903131427.35688.steve@pearwood.info>
	<Pine.LNX.4.64.0903130756260.2596@kimball.webabinitio.net>
Message-ID: <loom.20090313T122552-777@post.gmane.org>

R. David Murray <rdmurray <at> bitdance.com> writes:
> 
> You will note that what
> I suggested was that applications that _use the sync feature_ make
> it user controllable.

I'm sorry, but if it has nothing to do with Python itself, perhaps we could stop
this subthread (or move it to another ML)? There are enough messages already.

Thanks

Antoine.



From gnujohn at gmail.com  Fri Mar 13 12:13:42 2009
From: gnujohn at gmail.com (John Wagner)
Date: Fri, 13 Mar 2009 04:13:42 -0700
Subject: [Python-Dev] Fwd: installation with ubuntu 8.04
In-Reply-To: <mailman.6726.1236915178.11745.python-3000@python.org>
References: <mailman.6726.1236915178.11745.python-3000@python.org>
Message-ID: <3699312a0903130413l28b27f4aof72d763273e66e0b@mail.gmail.com>

---------- Forwarded message ----------
From: <python-3000-owner at python.org>
Date: Thu, Mar 12, 2009 at 8:32 PM
Subject: installation with ubuntu 8.04
To: gnujohn at gmail.com


This mailing list is closed now. Please use python-dev at python.org
instead.



---------- Forwarded message ----------
From: John Wagner <gnujohn at gmail.com>
To: python-3000 at python.org
Date: Thu, 12 Mar 2009 20:32:55 -0700
Subject: installation with ubuntu 8.04
I am sorry to bother about this but I've asked for help from Canonical with
no reply.
They seem reluctant to update Python beyond version 2.5; I'm trying to
install 3.0.1.
I'm a newb, and have tried rtfm--does someone have the incantation required?
Thanks v. much in advance.

-- 
John Wagner
Los Angeles





-- 
John Wagner
Los Angeles
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090313/b7d2f17c/attachment.htm>

From rdmurray at bitdance.com  Fri Mar 13 13:49:25 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 13 Mar 2009 08:49:25 -0400 (EDT)
Subject: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss
In-Reply-To: <loom.20090313T122552-777@post.gmane.org>
References: <20090312214419.GA29691@cskk.homeip.net>
	<loom.20090313T003418-37@post.gmane.org>
	<Pine.LNX.4.64.0903122200010.2596@kimball.webabinitio.net>
	<200903131427.35688.steve@pearwood.info>
	<Pine.LNX.4.64.0903130756260.2596@kimball.webabinitio.net>
	<loom.20090313T122552-777@post.gmane.org>
Message-ID: <Pine.LNX.4.64.0903130846290.2596@kimball.webabinitio.net>

On Fri, 13 Mar 2009 at 12:27, Antoine Pitrou wrote:
> R. David Murray <rdmurray <at> bitdance.com> writes:
>>
>> You will note that what
>> I suggested was that applications that _use the sync feature_ make
>> it user controllable.
>
> I'm sorry, but if it has nothing to do with Python itself, perhaps we could stop
> this subthread (or move it to another ML)? There are enough messages already.

It has to do with python in that it has to do with how this proposed
new feature gets documented.  But I agree, we've gone on too long
about it already.  I'll make a note to myself to bring this up again
(only if warranted) when the feature actually lands and I can see how
it is documented.

--
R. David Murray           http://www.bitdance.com

From ncoghlan at gmail.com  Fri Mar 13 13:58:41 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 13 Mar 2009 22:58:41 +1000
Subject: [Python-Dev] Fwd: installation with ubuntu 8.04
In-Reply-To: <3699312a0903130413l28b27f4aof72d763273e66e0b@mail.gmail.com>
References: <mailman.6726.1236915178.11745.python-3000@python.org>
	<3699312a0903130413l28b27f4aof72d763273e66e0b@mail.gmail.com>
Message-ID: <49BA5881.6030009@gmail.com>

John,

I'm afraid the auto-response to the python-3000 list is a little
misleading. That list was for the initial development of python 3.0, and
hence the redirect was to this list which is to discuss the
*development* of the Python interpreter (both 2.x and 3.x).

For more general questions (such as installation), the appropriate place
to ask is python-list at python.org (also available as the newsgroup
comp.lang.python).

Regards,
Nick.

P.S.

> This mailing list is closed now. Please use python-dev at python.org
> <mailto:python-dev at python.org>
> instead.

The mailman powers-that-be may want to update the python-3000
auto-response to direct people to python-list for normal questions and
python-dev for development specific questions.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From steve at holdenweb.com  Fri Mar 13 14:07:40 2009
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 13 Mar 2009 09:07:40 -0400
Subject: [Python-Dev] Fwd: installation with ubuntu 8.04
In-Reply-To: <3699312a0903130413l28b27f4aof72d763273e66e0b@mail.gmail.com>
References: <mailman.6726.1236915178.11745.python-3000@python.org>
	<3699312a0903130413l28b27f4aof72d763273e66e0b@mail.gmail.com>
Message-ID: <49BA5A9C.6090103@holdenweb.com>

John:

First, you have stumbled on to the wring list. Generally a list whose
name ends in -dev is for the devs of the project. Hence this is the list
where the Python devs live. In future posts like this should be sent to
python-list (comp.lang.python).

Having said that, Canonical's "reluctance" to support Python 3.x is
(IMHO) simply good engineering caution on their part. If you were to
(accidentally) replace the system Python with 3.x you would undoubtedly
break Ubuntu, which is very Python-oriented, so have a care!

Look in the build instructions for "alt-install", and make sure that you
*know* when you install the new Python that it isn't touching the system
Python at all. Always run make -n install and check what the script will
do before running it until you know what you are doing. Then run 3.x
from (somewhere like) /usr/local/bin.

You may find for a while that there's much less support code available
for the 3.x version. The transition to 3.x is really only just beginning.

regards
 Steve

John Wagner wrote:
> 
> 
> ---------- Forwarded message ----------
> From: ** <python-3000-owner at python.org
> <mailto:python-3000-owner at python.org>>
> Date: Thu, Mar 12, 2009 at 8:32 PM
> Subject: installation with ubuntu 8.04
> To: gnujohn at gmail.com <mailto:gnujohn at gmail.com>
> 
> 
> This mailing list is closed now. Please use python-dev at python.org
> <mailto:python-dev at python.org>
> instead.
> 
> 
> 
> ---------- Forwarded message ----------
> From: John Wagner <gnujohn at gmail.com <mailto:gnujohn at gmail.com>>
> To: python-3000 at python.org <mailto:python-3000 at python.org>
> Date: Thu, 12 Mar 2009 20:32:55 -0700
> Subject: installation with ubuntu 8.04
> I am sorry to bother about this but I've asked for help from Canonical
> with no reply.
> They seem reluctant to update Python beyond version 2.5; I'm trying to
> install 3.0.1.
> I'm a newb, and have tried rtfm--does someone have the incantation required?
> Thanks v. much in advance.
> 
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Fri Mar 13 14:07:40 2009
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 13 Mar 2009 09:07:40 -0400
Subject: [Python-Dev] Fwd: installation with ubuntu 8.04
In-Reply-To: <3699312a0903130413l28b27f4aof72d763273e66e0b@mail.gmail.com>
References: <mailman.6726.1236915178.11745.python-3000@python.org>
	<3699312a0903130413l28b27f4aof72d763273e66e0b@mail.gmail.com>
Message-ID: <49BA5A9C.6090103@holdenweb.com>

John:

First, you have stumbled on to the wring list. Generally a list whose
name ends in -dev is for the devs of the project. Hence this is the list
where the Python devs live. In future posts like this should be sent to
python-list (comp.lang.python).

Having said that, Canonical's "reluctance" to support Python 3.x is
(IMHO) simply good engineering caution on their part. If you were to
(accidentally) replace the system Python with 3.x you would undoubtedly
break Ubuntu, which is very Python-oriented, so have a care!

Look in the build instructions for "alt-install", and make sure that you
*know* when you install the new Python that it isn't touching the system
Python at all. Always run make -n install and check what the script will
do before running it until you know what you are doing. Then run 3.x
from (somewhere like) /usr/local/bin.

You may find for a while that there's much less support code available
for the 3.x version. The transition to 3.x is really only just beginning.

regards
 Steve

John Wagner wrote:
> 
> 
> ---------- Forwarded message ----------
> From: ** <python-3000-owner at python.org
> <mailto:python-3000-owner at python.org>>
> Date: Thu, Mar 12, 2009 at 8:32 PM
> Subject: installation with ubuntu 8.04
> To: gnujohn at gmail.com <mailto:gnujohn at gmail.com>
> 
> 
> This mailing list is closed now. Please use python-dev at python.org
> <mailto:python-dev at python.org>
> instead.
> 
> 
> 
> ---------- Forwarded message ----------
> From: John Wagner <gnujohn at gmail.com <mailto:gnujohn at gmail.com>>
> To: python-3000 at python.org <mailto:python-3000 at python.org>
> Date: Thu, 12 Mar 2009 20:32:55 -0700
> Subject: installation with ubuntu 8.04
> I am sorry to bother about this but I've asked for help from Canonical
> with no reply.
> They seem reluctant to update Python beyond version 2.5; I'm trying to
> install 3.0.1.
> I'm a newb, and have tried rtfm--does someone have the incantation required?
> Thanks v. much in advance.
> 
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From stephen at xemacs.org  Fri Mar 13 14:16:41 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 13 Mar 2009 22:16:41 +0900
Subject: [Python-Dev] Formatting mini-language suggestion
In-Reply-To: <49BA3056.40000@canterbury.ac.nz>
References: <D6F1551CA8894C3B95497076B0D74A35@RaymondLaptop1>
	<49B86015.2030203@gmail.com> <49B96C37.8030801@canterbury.ac.nz>
	<8763idlxk4.fsf@xemacs.org> <49BA3056.40000@canterbury.ac.nz>
Message-ID: <87iqmd4l6u.fsf@xemacs.org>

Greg Ewing writes:

 > I don't think you'll want to code the separators into
 > all your format strings in that case, either. You'll
 > want some sort of context that you set up for the
 > page you're about to serve.

Sure.  But the POSIX locale is not a good solution, nor is it a
building block for such a solution.  If this PEP *can't* provide such
a building block, too bad, life is like that.  If it can, but we don't
bother to look for it because of locale, for shame!

From steve at holdenweb.com  Fri Mar 13 14:14:34 2009
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 13 Mar 2009 09:14:34 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <Pine.LNX.4.64.0903130739170.2596@kimball.webabinitio.net>
References: <49ABCFC9.1070508@gmail.com>
	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>
	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>
	<Pine.LNX.4.64.0903130739170.2596@kimball.webabinitio.net>
Message-ID: <49BA5C3A.8030709@holdenweb.com>

R. David Murray wrote:
> On Fri, 13 Mar 2009 at 09:58, Chris Withers wrote:
>> Martin v. L?wis wrote:
>>> >  In light of this, what I'd love to see (but sadly can't really help
>>> >  with, and am not optimistic about happening) is for:
>>> > >  - python to grow a decent, cross platform, package management
>>> system
>>> > >  - the standard library to actually shrink to a point where only
>>> >  libraries that are not released elsewhere are included
>>> > >  I'd be interested to know how many users of python also felt
>>> this way >  ;-)
>>>
>>>  I don't like the standard library to shrink. It's good that batteries
>>>  are included.
>>
>> If a decent package management system *was* included, this wouldn't be
>> an issue..
> 
> I disagree.  One of the jobs I've had is release management for
> internal software projects that depend on various external pieces.
> Release integration tested against specific versions of those external
> packages, and those were the packages that needed to wind up on the system
> when the release was installed.  I've done systems depending on both perl
> and python, and let me tell you, python is way, _way_ easier to manage.
> With python, I have a dependency on a particular python version, and then
> maybe one or two add on packages.  With perl, I have perl, and then I
> have a gadzillion cpan modules.  I don't know how many a gadzillion is,
> because what I wound up doing was making a local copy of the cpan archive,
> checking that in to the repository, and writing up some scripts that made
> sure I pulled the actual install from my cpan snapshot and supported the
> developers in updating that snapshot when we were building a new version.
> (Nor was that the only problem with perl....what idiot decided it was
> OK to interactively prompt for things during a batch install process?!
> And without providing any way to script the answers, at least that I
> could find!)
> 
> So I'm +1 for keeping the Python stdlib as comprehensive as sensible.
> (Please note that last word...I've no objection to pruning things
> that are no longer serving a useful purpose, or that are better
> managed outside the core.)
> 
Just for clarity, when I said a "jumbo distribution" I meant one with
all necessary libraries to run and support a specified set of python
functionalities. The point is *not* to add other libraries (which
invites version creep and undermines stability) but to have everything
you need for a given (core plus) set of functionality.

I believe my original message acknowledged that this wouldn't be
everyone's cup of tea, but if a "good"* set of applications were
analyzed and a distribution built to support just those (presumably
Python) subsystems, you would have a good core that you can augment with
the system-installed Python if you are so minded.

A jumbo shouldn't try to replace the system-installed Python because
hopefully different Python (jumbo) distributions would coexist on the
same system, supporting those Python elements for which their
configuration is acceptable. I would not expect core developers to have
to give the jumbos much mind, except in so far as they might require
support for (slightly?) different versions of Python.

A 1.5.2 process can talk to a 3.1 process without any problems at all as
long as the application protocol is unambiguous. Why this insistence on
trying to do everything with a single interpreter? Sure, it might use
more resources to have three different versions of PIL in use from three
different jumbos, but let's cross that bridge when we come to it.

Naturally, in Python there are already several environments that will
compute the required library subset necessary to support an application,
though at present they do it only across a single Python version and
application. However, writing a simple GUI or command-line program to
call the other Python modules will give you a single analyzable module
tree. You don't even have to distribute the GUI if you don't want ...

So I don't see "jumbo" as replacing "batteries included". More like
"comes with 14v 300AH accumulator and support for domain name and email
services" or "suitable for GeoDjango virtuals under VirtualBox with NAT
addressing". For non-Python stuff (e.g. PostgreSQL, though SQLite is
plenty good enough for experiments) I think the API has to be stable
enough to accommodate some version variations.

regards
 Steve

* This is not a democracy: use your own prejudices to decide.
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Fri Mar 13 14:14:34 2009
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 13 Mar 2009 09:14:34 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <Pine.LNX.4.64.0903130739170.2596@kimball.webabinitio.net>
References: <49ABCFC9.1070508@gmail.com>
	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>
	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>
	<Pine.LNX.4.64.0903130739170.2596@kimball.webabinitio.net>
Message-ID: <49BA5C3A.8030709@holdenweb.com>

R. David Murray wrote:
> On Fri, 13 Mar 2009 at 09:58, Chris Withers wrote:
>> Martin v. L?wis wrote:
>>> >  In light of this, what I'd love to see (but sadly can't really help
>>> >  with, and am not optimistic about happening) is for:
>>> > >  - python to grow a decent, cross platform, package management
>>> system
>>> > >  - the standard library to actually shrink to a point where only
>>> >  libraries that are not released elsewhere are included
>>> > >  I'd be interested to know how many users of python also felt
>>> this way >  ;-)
>>>
>>>  I don't like the standard library to shrink. It's good that batteries
>>>  are included.
>>
>> If a decent package management system *was* included, this wouldn't be
>> an issue..
> 
> I disagree.  One of the jobs I've had is release management for
> internal software projects that depend on various external pieces.
> Release integration tested against specific versions of those external
> packages, and those were the packages that needed to wind up on the system
> when the release was installed.  I've done systems depending on both perl
> and python, and let me tell you, python is way, _way_ easier to manage.
> With python, I have a dependency on a particular python version, and then
> maybe one or two add on packages.  With perl, I have perl, and then I
> have a gadzillion cpan modules.  I don't know how many a gadzillion is,
> because what I wound up doing was making a local copy of the cpan archive,
> checking that in to the repository, and writing up some scripts that made
> sure I pulled the actual install from my cpan snapshot and supported the
> developers in updating that snapshot when we were building a new version.
> (Nor was that the only problem with perl....what idiot decided it was
> OK to interactively prompt for things during a batch install process?!
> And without providing any way to script the answers, at least that I
> could find!)
> 
> So I'm +1 for keeping the Python stdlib as comprehensive as sensible.
> (Please note that last word...I've no objection to pruning things
> that are no longer serving a useful purpose, or that are better
> managed outside the core.)
> 
Just for clarity, when I said a "jumbo distribution" I meant one with
all necessary libraries to run and support a specified set of python
functionalities. The point is *not* to add other libraries (which
invites version creep and undermines stability) but to have everything
you need for a given (core plus) set of functionality.

I believe my original message acknowledged that this wouldn't be
everyone's cup of tea, but if a "good"* set of applications were
analyzed and a distribution built to support just those (presumably
Python) subsystems, you would have a good core that you can augment with
the system-installed Python if you are so minded.

A jumbo shouldn't try to replace the system-installed Python because
hopefully different Python (jumbo) distributions would coexist on the
same system, supporting those Python elements for which their
configuration is acceptable. I would not expect core developers to have
to give the jumbos much mind, except in so far as they might require
support for (slightly?) different versions of Python.

A 1.5.2 process can talk to a 3.1 process without any problems at all as
long as the application protocol is unambiguous. Why this insistence on
trying to do everything with a single interpreter? Sure, it might use
more resources to have three different versions of PIL in use from three
different jumbos, but let's cross that bridge when we come to it.

Naturally, in Python there are already several environments that will
compute the required library subset necessary to support an application,
though at present they do it only across a single Python version and
application. However, writing a simple GUI or command-line program to
call the other Python modules will give you a single analyzable module
tree. You don't even have to distribute the GUI if you don't want ...

So I don't see "jumbo" as replacing "batteries included". More like
"comes with 14v 300AH accumulator and support for domain name and email
services" or "suitable for GeoDjango virtuals under VirtualBox with NAT
addressing". For non-Python stuff (e.g. PostgreSQL, though SQLite is
plenty good enough for experiments) I think the API has to be stable
enough to accommodate some version variations.

regards
 Steve

* This is not a democracy: use your own prejudices to decide.
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From gnujohn at gmail.com  Fri Mar 13 14:16:36 2009
From: gnujohn at gmail.com (John Wagner)
Date: Fri, 13 Mar 2009 06:16:36 -0700
Subject: [Python-Dev] Fwd: installation with ubuntu 8.04
In-Reply-To: <49BA5881.6030009@gmail.com>
References: <mailman.6726.1236915178.11745.python-3000@python.org>
	<3699312a0903130413l28b27f4aof72d763273e66e0b@mail.gmail.com>
	<49BA5881.6030009@gmail.com>
Message-ID: <3699312a0903130616w492a39cx1ea4806a988704b@mail.gmail.com>

Nick:

Thanks, man, for even replying.  I'm ashamed for posting such a stupid
question, and of course, you're right, as I should have known.  I'm used to
answering people's silly questions about c, and am an utter newb in python;
I'm trying to learn Scheme at the same time, so it's kind of hectic around
here.  My question is a newbie question, though, and I may as well face it.
I appreciate the polite tone of your reply.  If I can ever do anything to
help, I'll remember your example.

Regards,
John

On Fri, Mar 13, 2009 at 5:58 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> John,
>
> I'm afraid the auto-response to the python-3000 list is a little
> misleading. That list was for the initial development of python 3.0, and
> hence the redirect was to this list which is to discuss the
> *development* of the Python interpreter (both 2.x and 3.x).
>
> For more general questions (such as installation), the appropriate place
> to ask is python-list at python.org (also available as the newsgroup
> comp.lang.python).
>
> Regards,
> Nick.
>
> P.S.
>
> > This mailing list is closed now. Please use python-dev at python.org
> > <mailto:python-dev at python.org>
> > instead.
>
> The mailman powers-that-be may want to update the python-3000
> auto-response to direct people to python-list for normal questions and
> python-dev for development specific questions.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>



-- 
John Wagner
Los Angeles
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090313/f0ceca92/attachment.htm>

From tseaver at palladion.com  Fri Mar 13 14:30:49 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 13 Mar 2009 09:30:49 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>
References: <49ABCFC9.1070508@gmail.com>
	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>
	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>
	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>
Message-ID: <49BA6009.5070605@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Paul Moore wrote:
> 2009/3/13 Chris Withers <chris at simplistix.co.uk>:
>> If a decent package management system *was* included, this wouldn't be an
>> issue..
> 
> Remember that a "decent package management system" needs to handle
> filling in all the forms and arranging approvals to get authorisation
> for packages when you download them.
> 
> And no, I'm *not* joking. People in a locked-down corporate
> environment really do benefit from just having to get the OK for
> "Python", and then knowing that they have all they need.

You are plainly joking:  nothing in Python should know or care about the
various bureaucratic insanities in some workplaces.  Given the
*existing* stdlib and network connectivity, nothing any corporate
security blackshirt can do will prevent an even moderately-motivated
person from executing arbitrary code downloaded from elsewhere.  In that
case, what is the point in trying to help those who impose such craziness?

> Even ignoring the above,

Which the language and library should do.

> your "decent package management system" has
> to deal with systems with no internet connectivity - just copying the
> Python installer onto my pen drive to put on my Mum's laptop so I can
> write some admin jobs for her, is a lot easier than having to pick and
> choose from PyPI what to download before I start.

Nobody is arguing that there should be *no* batteries in the stdlib:  we
are talking about the ones which are leaking acid already, or which
might get there soon due to lack of maintenance.

> -1 on slimming down the stdlib.
> (OTOH, I've no problem with seeing an improved package system - just
> don't use it as an excuse to cripple the stdlib!)

Part of this discussion is about not *expanding* the stdlib:  give a
reasonable packaging story, leaving a given component out of the library
is a defensible choice.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJumAJ+gerLs4ltQ4RApC7AKDVsIxfBlw6CWWLa+VhaASyDz+LFQCfQp5I
yzrdYPo1FbXGAB90Ucf/Le8=
=bCTx
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Fri Mar 13 14:30:49 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 13 Mar 2009 09:30:49 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>
References: <49ABCFC9.1070508@gmail.com>
	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>
	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>
	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>
Message-ID: <49BA6009.5070605@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Paul Moore wrote:
> 2009/3/13 Chris Withers <chris at simplistix.co.uk>:
>> If a decent package management system *was* included, this wouldn't be an
>> issue..
> 
> Remember that a "decent package management system" needs to handle
> filling in all the forms and arranging approvals to get authorisation
> for packages when you download them.
> 
> And no, I'm *not* joking. People in a locked-down corporate
> environment really do benefit from just having to get the OK for
> "Python", and then knowing that they have all they need.

You are plainly joking:  nothing in Python should know or care about the
various bureaucratic insanities in some workplaces.  Given the
*existing* stdlib and network connectivity, nothing any corporate
security blackshirt can do will prevent an even moderately-motivated
person from executing arbitrary code downloaded from elsewhere.  In that
case, what is the point in trying to help those who impose such craziness?

> Even ignoring the above,

Which the language and library should do.

> your "decent package management system" has
> to deal with systems with no internet connectivity - just copying the
> Python installer onto my pen drive to put on my Mum's laptop so I can
> write some admin jobs for her, is a lot easier than having to pick and
> choose from PyPI what to download before I start.

Nobody is arguing that there should be *no* batteries in the stdlib:  we
are talking about the ones which are leaking acid already, or which
might get there soon due to lack of maintenance.

> -1 on slimming down the stdlib.
> (OTOH, I've no problem with seeing an improved package system - just
> don't use it as an excuse to cripple the stdlib!)

Part of this discussion is about not *expanding* the stdlib:  give a
reasonable packaging story, leaving a given component out of the library
is a defensible choice.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJumAJ+gerLs4ltQ4RApC7AKDVsIxfBlw6CWWLa+VhaASyDz+LFQCfQp5I
yzrdYPo1FbXGAB90Ucf/Le8=
=bCTx
-----END PGP SIGNATURE-----


From zvezdan at zope.com  Fri Mar 13 15:09:27 2009
From: zvezdan at zope.com (Zvezdan Petkovic)
Date: Fri, 13 Mar 2009 10:09:27 -0400
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49B95F43.3000708@v.loewis.de>
References: <gp6hhq$hss$1@ger.gmane.org>	<49B85D42.9080406@gmail.com>	<loom.20090312T010957-202@post.gmane.org>	<200903122254.21316.steve@pearwood.info>	<loom.20090312T122030-705@post.gmane.org>
	<49B91FAB.2010101@gmail.com> <49B95F43.3000708@v.loewis.de>
Message-ID: <873EFBC3-E729-4C02-B941-38D0C5314035@zope.com>


On Mar 12, 2009, at 3:15 PM, Martin v. L?wis wrote:
> You still wouldn't use the tempfile module in that case. Instead, you
> would create a regular file, with the name base on the name of the
> important file.

If the file is *really* important, you actually want to use a  
temporary, randomly chosen, *unpredictable* name.

Think about the security implications of a file name that is in  
advance known to an attacker as well as the fact that the said file  
will replace an *important* system file.

See the details in any man page on mkstemp() and why it was introduced  
to replace a predictable mktemp().  Also notice that even mktemp() is  
better then what you proposed above.

Of course, the above are C functions.  I don't think that Python  
programming is immune from such security considerations either.

	Zvezdan


From mishok13 at gmail.com  Fri Mar 13 15:23:11 2009
From: mishok13 at gmail.com (Andrii V. Mishkovskyi)
Date: Fri, 13 Mar 2009 16:23:11 +0200
Subject: [Python-Dev] py: urls, new bazaar plugin available
In-Reply-To: <FFBA9E3F-0392-4D00-848D-1184E85A04BC@python.org>
References: <FFBA9E3F-0392-4D00-848D-1184E85A04BC@python.org>
Message-ID: <192840a00903130723y52de4f34rd3dbadbd34873b15@mail.gmail.com>

On Thu, Mar 12, 2009 at 6:26 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hello Bazaar users!
>
> There's a new Bazaar plugin you can use to more easily access read-only or
> read-write branches on code.python.org. ?This plugin provides the 'py:' url
> prefix. ?For example, to get the trunk branch with the plugin installed, you
> can now do:
>
> ? ?bzr branch py:trunk
>
> or to get the 2.6 branch you can do:
>
> ? ?bzr branch py:2.6
>
> You can also use this to get user branches, e.g. my email rewrite branch:
>
> ? ?bzr branch py:~barry/30email
>
> If you have write access to branches on code.python.org, you can either set
> the environment variable $PYDEV_USER or the Bazaar configuration option
> pydev_user (the value doesn't matter) to get bzr+ssh access instead of the
> standard http access. ?py: works both for branching and pushing.
>
> Thanks to Karl Fogel for the implementation. ?You'll need Karl's pydev
> plugin branch, and instructions on installing this are available here:
>
> ? ?http://tinyurl.com/aq55oc
>
> I've updated the wiki page with additional details:
>
> ? ?http://wiki.python.org/moin/Bazaar

And here is anouncement of similar Mercurial extension (slightly more
general, though):
http://selenic.com/pipermail/mercurial/2009-March/024547.html
Should this go to http://wiki.python.org/moin/Mercurial (this page
doesn't exist yet) or somewhere else?

>
> Enjoy!
> Barry
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Darwin)
>
> iQCVAwUBSbk3xHEjvBPtnXfVAQIiVgQAt3GwmDSkFjog/mp4PxVKn/F6MQoEDa/A
> 0nNysiU2oEvUXDBWPlab2V53tqpZ/uvOS17hl7ZhlDe+Z2jUUYiCkH+Hfvpz5F9n
> u0Gf+5dMA7GQkLhvOezu7r6ngu2mmBB84ZwAfX4tJM+bBuQEn+U3BuRspkDiCb0S
> iZONBdHyk5g=
> =Pb2v
> -----END PGP SIGNATURE-----
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/mishok13%40gmail.com
>



-- 
--
Wbr, Andrii V. Mishkovskyi.

He's got a heart of a little child, and he keeps it in a jar on his desk.

From aahz at pythoncraft.com  Fri Mar 13 15:31:33 2009
From: aahz at pythoncraft.com (Aahz)
Date: Fri, 13 Mar 2009 07:31:33 -0700
Subject: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss
In-Reply-To: <loom.20090313T122552-777@post.gmane.org>
References: <20090312214419.GA29691@cskk.homeip.net>
	<loom.20090313T003418-37@post.gmane.org>
	<Pine.LNX.4.64.0903122200010.2596@kimball.webabinitio.net>
	<200903131427.35688.steve@pearwood.info>
	<Pine.LNX.4.64.0903130756260.2596@kimball.webabinitio.net>
	<loom.20090313T122552-777@post.gmane.org>
Message-ID: <20090313143133.GA27930@panix.com>

On Fri, Mar 13, 2009, Antoine Pitrou wrote:
> R. David Murray <rdmurray <at> bitdance.com> writes:
>> 
>> You will note that what
>> I suggested was that applications that _use the sync feature_ make
>> it user controllable.
> 
> I'm sorry, but if it has nothing to do with Python itself, perhaps we
> could stop this subthread (or move it to another ML)? There are enough
> messages already.

<blink>  Yes, this *is* about Python: how should Python support David's
use-case?  This discussion started because Python currently doesn't have
good mechansisms for fine-grained synching, and people are discussing how
Python should support various use-cases.  Please don't be too aggressive
about labeling discussion off-topic.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"All problems in computer science can be solved by another level of     
indirection."  --Butler Lampson

From lie.1296 at gmail.com  Fri Mar 13 15:38:12 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Sat, 14 Mar 2009 01:38:12 +1100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BA6009.5070605@palladion.com>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>
	<49BA6009.5070605@palladion.com>
Message-ID: <gpdr4k$sbl$1@ger.gmane.org>

Tres Seaver wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Paul Moore wrote:
>> 2009/3/13 Chris Withers <chris at simplistix.co.uk>:
>>> If a decent package management system *was* included, this wouldn't be an
>>> issue..
>> Remember that a "decent package management system" needs to handle
>> filling in all the forms and arranging approvals to get authorisation
>> for packages when you download them.
>>
>> And no, I'm *not* joking. People in a locked-down corporate
>> environment really do benefit from just having to get the OK for
>> "Python", and then knowing that they have all they need.
> 
> You are plainly joking:  nothing in Python should know or care about the
> various bureaucratic insanities in some workplaces.  Given the
> *existing* stdlib and network connectivity, nothing any corporate
> security blackshirt can do will prevent an even moderately-motivated
> person from executing arbitrary code downloaded from elsewhere.  In that
> case, what is the point in trying to help those who impose such craziness?

I (and most people, I presume) would not run arbitrary program 
downloaded from somewhere else on a corporate server that holds many 
important customer data even when there is no technical or even 
bureaucratic restriction, maybe I will sneak around on a workstation but 
definitely not on the server especially if I love my job and want to 
keep it (I'm a student though so that applies to me in the future).


From tseaver at palladion.com  Fri Mar 13 16:01:05 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 13 Mar 2009 11:01:05 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <gpdr4k$sbl$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>	<49BA6009.5070605@palladion.com>
	<gpdr4k$sbl$1@ger.gmane.org>
Message-ID: <49BA7531.8000806@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Lie Ryan wrote:
> Tres Seaver wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Paul Moore wrote:
>>> 2009/3/13 Chris Withers <chris at simplistix.co.uk>:
>>>> If a decent package management system *was* included, this wouldn't be an
>>>> issue..
>>> Remember that a "decent package management system" needs to handle
>>> filling in all the forms and arranging approvals to get authorisation
>>> for packages when you download them.
>>>
>>> And no, I'm *not* joking. People in a locked-down corporate
>>> environment really do benefit from just having to get the OK for
>>> "Python", and then knowing that they have all they need.
>> You are plainly joking:  nothing in Python should know or care about the
>> various bureaucratic insanities in some workplaces.  Given the
>> *existing* stdlib and network connectivity, nothing any corporate
>> security blackshirt can do will prevent an even moderately-motivated
>> person from executing arbitrary code downloaded from elsewhere.  In that
>> case, what is the point in trying to help those who impose such craziness?
> 
> I (and most people, I presume) would not run arbitrary program 
> downloaded from somewhere else on a corporate server that holds many 
> important customer data even when there is no technical or even 
> bureaucratic restriction, maybe I will sneak around on a workstation but 
> definitely not on the server especially if I love my job and want to 
> keep it (I'm a student though so that applies to me in the future).

I'm not arguing that employees should violate their employers' policies:
 I'm arguing that Python itself shouldn't try to cater to such policies.
 Note that I'm not talking about running code pushed on me by malware
authors, either:  I'm talking about "ordinary" software development
activities like using a script from a cookbook, or using a well-tested
and supported library, rather than NIH.

Given that the out-of-the-box Python install already has facilities for
retrieving text over the net and executing that text, the notion of
"locking down" a machine to include only the bits installed in the stock
Python install is just "security theatre;"  such a machine shouldn't
have Python installed at all (nor a C compiler, etc.)



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJunUx+gerLs4ltQ4RAojAAKCdoliiVDoGoKzfGXNuQUZVmoPrhgCfXeSa
pGCKI3wLt9W1A4ccnINSdLs=
=3H9u
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Fri Mar 13 16:01:05 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 13 Mar 2009 11:01:05 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <gpdr4k$sbl$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>	<49BA6009.5070605@palladion.com>
	<gpdr4k$sbl$1@ger.gmane.org>
Message-ID: <49BA7531.8000806@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Lie Ryan wrote:
> Tres Seaver wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Paul Moore wrote:
>>> 2009/3/13 Chris Withers <chris at simplistix.co.uk>:
>>>> If a decent package management system *was* included, this wouldn't be an
>>>> issue..
>>> Remember that a "decent package management system" needs to handle
>>> filling in all the forms and arranging approvals to get authorisation
>>> for packages when you download them.
>>>
>>> And no, I'm *not* joking. People in a locked-down corporate
>>> environment really do benefit from just having to get the OK for
>>> "Python", and then knowing that they have all they need.
>> You are plainly joking:  nothing in Python should know or care about the
>> various bureaucratic insanities in some workplaces.  Given the
>> *existing* stdlib and network connectivity, nothing any corporate
>> security blackshirt can do will prevent an even moderately-motivated
>> person from executing arbitrary code downloaded from elsewhere.  In that
>> case, what is the point in trying to help those who impose such craziness?
> 
> I (and most people, I presume) would not run arbitrary program 
> downloaded from somewhere else on a corporate server that holds many 
> important customer data even when there is no technical or even 
> bureaucratic restriction, maybe I will sneak around on a workstation but 
> definitely not on the server especially if I love my job and want to 
> keep it (I'm a student though so that applies to me in the future).

I'm not arguing that employees should violate their employers' policies:
 I'm arguing that Python itself shouldn't try to cater to such policies.
 Note that I'm not talking about running code pushed on me by malware
authors, either:  I'm talking about "ordinary" software development
activities like using a script from a cookbook, or using a well-tested
and supported library, rather than NIH.

Given that the out-of-the-box Python install already has facilities for
retrieving text over the net and executing that text, the notion of
"locking down" a machine to include only the bits installed in the stock
Python install is just "security theatre;"  such a machine shouldn't
have Python installed at all (nor a C compiler, etc.)



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJunUx+gerLs4ltQ4RAojAAKCdoliiVDoGoKzfGXNuQUZVmoPrhgCfXeSa
pGCKI3wLt9W1A4ccnINSdLs=
=3H9u
-----END PGP SIGNATURE-----


From phd at phd.pp.ru  Fri Mar 13 16:06:44 2009
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Fri, 13 Mar 2009 18:06:44 +0300
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <20090313092807.GA27105@phd.pp.ru>
References: <gp6hhq$hss$1@ger.gmane.org>
	<aac2c7cb0903122114i22d89b9dx88c15176dce7d739@mail.gmail.com>
	<20090313092807.GA27105@phd.pp.ru>
Message-ID: <20090313150644.GB8643@phd.pp.ru>

On Fri, Mar 13, 2009 at 12:28:07PM +0300, Oleg Broytmann wrote:
> On Thu, Mar 12, 2009 at 10:14:41PM -0600, Adam Olsen wrote:
> > Yet the ext4
> > developers didn't see it that way, so it was sacrificed to new
> > performance improvements (delayed allocation).
> 
>    Ext4 is not the only FS with delayed allocation. New XFS has it, btrfs
> will have it. Don't know about other OS/FS (ZFS? NTFS?)

http://thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/

   Ted Tso said HFS+ and ZFS have the property as well. So no, it is not
a deficiency in the Linux kernel or in Ext4 FS - it is a mainstream path in
modern filesystem design.

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From solipsis at pitrou.net  Fri Mar 13 16:06:27 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 13 Mar 2009 15:06:27 +0000 (UTC)
Subject: [Python-Dev] Python-Dev] wait time [was: Ext4 data loss
References: <20090312214419.GA29691@cskk.homeip.net>
	<loom.20090313T003418-37@post.gmane.org>
	<Pine.LNX.4.64.0903122200010.2596@kimball.webabinitio.net>
	<200903131427.35688.steve@pearwood.info>
	<Pine.LNX.4.64.0903130756260.2596@kimball.webabinitio.net>
	<loom.20090313T122552-777@post.gmane.org>
	<20090313143133.GA27930@panix.com>
Message-ID: <loom.20090313T145643-371@post.gmane.org>

Aahz <aahz <at> pythoncraft.com> writes:
> 
> On Fri, Mar 13, 2009, Antoine Pitrou wrote:
> > R. David Murray <rdmurray <at> bitdance.com> writes:
> >> 
> >> You will note that what
> >> I suggested was that applications that _use the sync feature_ make
> >> it user controllable.
> > 
> > I'm sorry, but if it has nothing to do with Python itself, perhaps we
> > could stop this subthread (or move it to another ML)? There are enough
> > messages already.
> 
> <blink>  Yes, this *is* about Python: how should Python support David's
> use-case?

Python already has plenty of mechanisms for enabling configuration choices, so I
don't see your point: the use case is /already/ supported assuming there is a
sync() method at all.

The discussion is (or should be) about whether/how Python should expose a sync()
facility in its file input/output stack (or somewhere else in the stdlib).
Configuration policies are not for us to decide.



From lie.1296 at gmail.com  Fri Mar 13 16:28:50 2009
From: lie.1296 at gmail.com (Lie Ryan)
Date: Sat, 14 Mar 2009 02:28:50 +1100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BA7531.8000806@palladion.com>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>	<49BA6009.5070605@palladion.com>	<gpdr4k$sbl$1@ger.gmane.org>
	<49BA7531.8000806@palladion.com>
Message-ID: <gpdu3h$8ct$1@ger.gmane.org>

Tres Seaver wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Lie Ryan wrote:
>> Tres Seaver wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> Paul Moore wrote:
>>>> 2009/3/13 Chris Withers <chris at simplistix.co.uk>:
>>>>> If a decent package management system *was* included, this wouldn't be an
>>>>> issue..
>>>> Remember that a "decent package management system" needs to handle
>>>> filling in all the forms and arranging approvals to get authorisation
>>>> for packages when you download them.
>>>>
>>>> And no, I'm *not* joking. People in a locked-down corporate
>>>> environment really do benefit from just having to get the OK for
>>>> "Python", and then knowing that they have all they need.
>>> You are plainly joking:  nothing in Python should know or care about the
>>> various bureaucratic insanities in some workplaces.  Given the
>>> *existing* stdlib and network connectivity, nothing any corporate
>>> security blackshirt can do will prevent an even moderately-motivated
>>> person from executing arbitrary code downloaded from elsewhere.  In that
>>> case, what is the point in trying to help those who impose such craziness?
>> I (and most people, I presume) would not run arbitrary program 
>> downloaded from somewhere else on a corporate server that holds many 
>> important customer data even when there is no technical or even 
>> bureaucratic restriction, maybe I will sneak around on a workstation but 
>> definitely not on the server especially if I love my job and want to 
>> keep it (I'm a student though so that applies to me in the future).
> 
> I'm not arguing that employees should violate their employers' policies:
>  I'm arguing that Python itself shouldn't try to cater to such policies.

Basically you're saying: Python is designed not to work on such environment.

>  Note that I'm not talking about running code pushed on me by malware
> authors, either:  I'm talking about "ordinary" software development
> activities like using a script from a cookbook, or using a well-tested
> and supported library, rather than NIH.

Some companies have /very/ strict policies on running anything on live 
server, including scripts you write yourself. The problem is if the 
script goes awry, it might disturb the stability or even security of the 
server.

> Given that the out-of-the-box Python install already has facilities for
> retrieving text over the net and executing that text, the notion of
> "locking down" a machine to include only the bits installed in the stock
> Python install is just "security theatre;"  such a machine shouldn't
> have Python installed at all (nor a C compiler, etc.)

When the server administrator is already freaked out about adding an 
script developed by in-house employee, what about adding an external module?

Of course all of this does not (usually) apply to regular workstation. A 
messed up workstation only means a reinstall, a messed up server may 
mean company reputation.


From lists at cheimes.de  Fri Mar 13 16:35:24 2009
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 13 Mar 2009 16:35:24 +0100
Subject: [Python-Dev] Fwd: installation with ubuntu 8.04
In-Reply-To: <49BA5A9C.6090103@holdenweb.com>
References: <mailman.6726.1236915178.11745.python-3000@python.org>	<3699312a0903130413l28b27f4aof72d763273e66e0b@mail.gmail.com>
	<49BA5A9C.6090103@holdenweb.com>
Message-ID: <gpdufs$8lb$1@ger.gmane.org>

Steve Holden wrote:
> Look in the build instructions for "alt-install", and make sure that you
> *know* when you install the new Python that it isn't touching the system
> Python at all. Always run make -n install and check what the script will
> do before running it until you know what you are doing. Then run 3.x
> from (somewhere like) /usr/local/bin.

We anticipated the issue during the development cycle of py3k. The 3.x
series doesn't install a "python" command when it's installed with "make
install".

$ make install
[...]
* Note: not installed as 'python'.
* Use 'make fullinstall' to install as 'python'.
* However, 'make fullinstall' is discouraged,
* as it will clobber your Python 2.x installation.

Christian


From tseaver at palladion.com  Fri Mar 13 17:30:12 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 13 Mar 2009 12:30:12 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <gpdu3h$8ct$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>	<49BA6009.5070605@palladion.com>	<gpdr4k$sbl$1@ger.gmane.org>	<49BA7531.8000806@palladion.com>
	<gpdu3h$8ct$1@ger.gmane.org>
Message-ID: <49BA8A14.7090607@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Lie Ryan wrote:

> Tres Seaver wrote:
>> I'm not arguing that employees should violate their employers' policies:
>>  I'm arguing that Python itself shouldn't try to cater to such policies.
> 
> Basically you're saying: Python is designed not to work on such environment.

No, I'm saying that it isn't Python's responibility to enable that kind
of policy.  If it happens to be good *for Python* to have a a package
installation / upgrade machinery (the real point of the discussion),
then it will be up to the paranoid admin to figure out how to disable
that feature:  it isn't the problem of the Python developers.

There are real costs to "batteries included," especially for modules
which don't get used much.  One such cost is that an unused module tends
to bitrot over time;  another is that the presence of a module in the
stdlib may "shadow" other, better modules / packages which are not in
the stdlib.  Those costs need to be balanced against the undoubted
benefits, when making the choice to add or remove a module from the stdlib.

>>  Note that I'm not talking about running code pushed on me by malware
>> authors, either:  I'm talking about "ordinary" software development
>> activities like using a script from a cookbook, or using a well-tested
>> and supported library, rather than NIH.
> 
> Some companies have /very/ strict policies on running anything on live 
> server, including scripts you write yourself. The problem is if the 
> script goes awry, it might disturb the stability or even security of the 
> server.

I understand that such policies exist, and why.  However, I don't think
their existence is a relevant constraint on the design or implementation
of Python.

>> Given that the out-of-the-box Python install already has facilities for
>> retrieving text over the net and executing that text, the notion of
>> "locking down" a machine to include only the bits installed in the stock
>> Python install is just "security theatre;"  such a machine shouldn't
>> have Python installed at all (nor a C compiler, etc.)
> 
> When the server administrator is already freaked out about adding an 
> script developed by in-house employee, what about adding an external module?

An admin who is that paranoid shouldn't be giving people shell access,
either:  given shell acccess, network connectivity, and the existing
stdlib the admin's policy is unenforceable as a technical measure.  Even
if the user can't create a file anywhere on the filesystem, the
interpreter prompt is enough to allow the user to evade the policy.
Heck, the *bash* prompt is enough to wreck it, e.g. using "here
documents," even without network connectivity.

As an aisde:  anybody who is installing packages from PyPI on a
production box (rather than using an index under their own control)
isn't sufficiently paranoid:  it can and does happen that people
re-upload changed packages to PyPI without changing the version, for
instance, not to mention removing older releases.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJuooU+gerLs4ltQ4RAsdnAKCSkKc94bHvHBIrILampl9+Mksz8wCeJSBe
+Yl5HVmwQ6StGTcWNmDiSjE=
=qGID
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Fri Mar 13 17:30:12 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 13 Mar 2009 12:30:12 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <gpdu3h$8ct$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>	<49BA6009.5070605@palladion.com>	<gpdr4k$sbl$1@ger.gmane.org>	<49BA7531.8000806@palladion.com>
	<gpdu3h$8ct$1@ger.gmane.org>
Message-ID: <49BA8A14.7090607@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Lie Ryan wrote:

> Tres Seaver wrote:
>> I'm not arguing that employees should violate their employers' policies:
>>  I'm arguing that Python itself shouldn't try to cater to such policies.
> 
> Basically you're saying: Python is designed not to work on such environment.

No, I'm saying that it isn't Python's responibility to enable that kind
of policy.  If it happens to be good *for Python* to have a a package
installation / upgrade machinery (the real point of the discussion),
then it will be up to the paranoid admin to figure out how to disable
that feature:  it isn't the problem of the Python developers.

There are real costs to "batteries included," especially for modules
which don't get used much.  One such cost is that an unused module tends
to bitrot over time;  another is that the presence of a module in the
stdlib may "shadow" other, better modules / packages which are not in
the stdlib.  Those costs need to be balanced against the undoubted
benefits, when making the choice to add or remove a module from the stdlib.

>>  Note that I'm not talking about running code pushed on me by malware
>> authors, either:  I'm talking about "ordinary" software development
>> activities like using a script from a cookbook, or using a well-tested
>> and supported library, rather than NIH.
> 
> Some companies have /very/ strict policies on running anything on live 
> server, including scripts you write yourself. The problem is if the 
> script goes awry, it might disturb the stability or even security of the 
> server.

I understand that such policies exist, and why.  However, I don't think
their existence is a relevant constraint on the design or implementation
of Python.

>> Given that the out-of-the-box Python install already has facilities for
>> retrieving text over the net and executing that text, the notion of
>> "locking down" a machine to include only the bits installed in the stock
>> Python install is just "security theatre;"  such a machine shouldn't
>> have Python installed at all (nor a C compiler, etc.)
> 
> When the server administrator is already freaked out about adding an 
> script developed by in-house employee, what about adding an external module?

An admin who is that paranoid shouldn't be giving people shell access,
either:  given shell acccess, network connectivity, and the existing
stdlib the admin's policy is unenforceable as a technical measure.  Even
if the user can't create a file anywhere on the filesystem, the
interpreter prompt is enough to allow the user to evade the policy.
Heck, the *bash* prompt is enough to wreck it, e.g. using "here
documents," even without network connectivity.

As an aisde:  anybody who is installing packages from PyPI on a
production box (rather than using an index under their own control)
isn't sufficiently paranoid:  it can and does happen that people
re-upload changed packages to PyPI without changing the version, for
instance, not to mention removing older releases.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJuooU+gerLs4ltQ4RAsdnAKCSkKc94bHvHBIrILampl9+Mksz8wCeJSBe
+Yl5HVmwQ6StGTcWNmDiSjE=
=qGID
-----END PGP SIGNATURE-----


From status at bugs.python.org  Fri Mar 13 18:06:51 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 13 Mar 2009 18:06:51 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090313170651.308AE781ED@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (03/06/09 - 03/13/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2388 open (+38) / 14920 closed (+16) / 17308 total (+54)

Open issues with patches:   837

Average duration of open issues: 658 days.
Median duration of open issues: 399 days.

Open Issues Breakdown
   open  2359 (+37)
pending    29 ( +1)

Issues Created Or Reopened (56)
_______________________________

2to3 wrong for types.StringTypes                                 03/08/09
       http://bugs.python.org/issue5425    reopened hagen                         
       patch                                                                   

cmpfunc in Python 3.0.1 windows installer                        03/10/09
       http://bugs.python.org/issue5431    reopened Nigel Galloway                
                                                                               

plistlib contains unescaped hex sequence in doc string           03/06/09
       http://bugs.python.org/issue5432    created  jim.correia                   
                                                                               

Excessive optimization in IncrementalNewlineDecoder              03/06/09
CLOSED http://bugs.python.org/issue5433    created  pitrou                        
       patch                                                                   

datetime.MonthDelta                                              03/07/09
       http://bugs.python.org/issue5434    created  jess.austin                   
       patch                                                                   

test_httpservers on Debian Testing                               03/07/09
       http://bugs.python.org/issue5435    created  zamotcr                       
                                                                               

test_distutils fails with official Mac OS X Installer Disk Image 03/07/09
       http://bugs.python.org/issue5436    created  oefe                          
                                                                               

Singleton MemoryError can hold traceback data and locals indefin 03/08/09
       http://bugs.python.org/issue5437    created  pitrou                        
       patch                                                                   

test_bigmem.test_from_2G_generator uses more memory than expecte 03/08/09
       http://bugs.python.org/issue5438    created  pitrou                        
                                                                               

string.strip behaves strangly                                    03/08/09
CLOSED http://bugs.python.org/issue5439    created  dwjang                        
                                                                               

string.strip behaves strangly                                    03/08/09
CLOSED http://bugs.python.org/issue5440    created  dwjang                        
                                                                               

Convenience API for timeit.main                                  03/08/09
       http://bugs.python.org/issue5441    created  ncoghlan                      
       easy                                                                    

[3.1alpha1] test_importlib fails on Mac OSX 10.5.6               03/08/09
       http://bugs.python.org/issue5442    created  cartman                       
                                                                               

trivial typo in itertools documentation                          03/10/09
       http://bugs.python.org/issue5443    reopened donlorenzo                    
       patch                                                                   

.chm build process on Windows doesn't use the right filename     03/08/09
       http://bugs.python.org/issue5444    created  gagenellina                   
       patch                                                                   

codecs.StreamWriter.writelines problem when passed generator     03/08/09
       http://bugs.python.org/issue5445    created  dlesco                        
                                                                               

doc building progress on Windows doesn't show any color          03/08/09
CLOSED http://bugs.python.org/issue5446    created  gagenellina                   
       patch                                                                   

future unicode literals and r'\u'                                03/08/09
       http://bugs.python.org/issue5447    created  pooryorick                    
       patch                                                                   

Add precision property to decimal.Decimal                        03/08/09
CLOSED http://bugs.python.org/issue5448    created  dlesco                        
                                                                               

bug fix to prevent io.BytesIO from accepting arbitrary keyword a 03/08/09
       http://bugs.python.org/issue5449    created  erickt                        
       patch                                                                   

test_tcl testLoadTk fails if DISPLAY defined but connect fails,  03/09/09
       http://bugs.python.org/issue5450    created  bitdancer                     
       patch                                                                   

patch to make io.StringIO consistent with open with respect to n 03/09/09
CLOSED http://bugs.python.org/issue5451    created  erickt                        
       patch                                                                   

Documentation for io.StringIO has wrong info for arguments       03/09/09
CLOSED http://bugs.python.org/issue5452    created  erickt                        
                                                                               

pydoc -k fails (release30-maint patch)                           03/09/09
       http://bugs.python.org/issue5453    created  dariusp                       
       patch                                                                   

pydoc -k fails (py3k patch)                                      03/09/09
CLOSED http://bugs.python.org/issue5454    created  dariusp                       
       patch                                                                   

csv module no longer works as expected when file opened in binar 03/09/09
       http://bugs.python.org/issue5455    created  skip.montanaro                
                                                                               

io.StringIO's universal newlines support is broken in 3.0.1      03/09/09
CLOSED http://bugs.python.org/issue5456    created  erickt                        
                                                                               

ZipFile writes incorrect modification time (second is off-by-one 03/09/09
CLOSED http://bugs.python.org/issue5457    created  ctheune                       
                                                                               

threading.Thread.join() documentation: missing 'from version' fo 03/09/09
CLOSED http://bugs.python.org/issue5458    created  eyal.gordon                   
                                                                               

msiexec not creating msvcr90.dll with python -2.6.1.msi          03/09/09
       http://bugs.python.org/issue5459    created  cjlesh                        
                                                                               

Python 3.0 grammar is ambiguous with the addition of star_expr   03/09/09
       http://bugs.python.org/issue5460    created  fabioz                        
                                                                               

python3 symlink                                                  03/09/09
       http://bugs.python.org/issue5461    created  dripton                       
                                                                               

Pythonwin Vista Compatibility                                    03/09/09
CLOSED http://bugs.python.org/issue5462    created  jearls                        
                                                                               

Compiler warning get_ulong is never used 3.x                     03/09/09
       http://bugs.python.org/issue5463    created  andreas.schawo                
       patch                                                                   

msgfmt.py does not work with plural form                         03/09/09
       http://bugs.python.org/issue5464    created  steve_geo                     
                                                                               

No edit in IDLE in right click context menu                      03/10/09
CLOSED http://bugs.python.org/issue5465    created  ESlim                         
                                                                               

Tix.Balloon causes TCLError: unknown color name "{#ffff60}" in P 03/10/09
       http://bugs.python.org/issue5466    created  andrewp22                     
                                                                               

tools\msi\merge.py is sensitive to lack of config.py             03/10/09
       http://bugs.python.org/issue5467    created  tim.golden                    
       patch                                                                   

urlencode does not handle "bytes", and could easily handle alter 03/10/09
       http://bugs.python.org/issue5468    created  dmahn                         
                                                                               

Reference paragraph about the constructs that bind names needs u 03/10/09
       http://bugs.python.org/issue5469    created  MLModel                       
                                                                               

MSI installer misses zipdir.zip file in Lib\test                 03/10/09
       http://bugs.python.org/issue5470    created  tim.golden                    
       patch                                                                   

os.path.expanduser('~') doesnt works correctly when HOME is '/'  03/10/09
       http://bugs.python.org/issue5471    created  pancake                       
       patch                                                                   

distutils.test_util fails to restore os.uname; causes test_platf 03/10/09
CLOSED http://bugs.python.org/issue5472    created  tim.golden                    
       patch                                                                   

round(float, ndigits<0) sometimes rounds to odd                  03/10/09
       http://bugs.python.org/issue5473    created  dingo                         
                                                                               

distutils produces invalid RPM packages of prerelease python pac 03/11/09
       http://bugs.python.org/issue5474    created  Rudd-O                        
       patch                                                                   

urllib2.getproxies not documented                                03/11/09
       http://bugs.python.org/issue5475    created  timmie                        
                                                                               

datetime: timedelta(minutes = i) silently fails with numpy.int32 03/11/09
       http://bugs.python.org/issue5476    created  barronh                       
                                                                               

Typo in itertools documentation                                  03/11/09
CLOSED http://bugs.python.org/issue5477    created  thomasguest                   
                                                                               

document mistake xml.dom.minidom.Element                         03/12/09
       http://bugs.python.org/issue5478    created  hsmtkk                        
                                                                               

Add an easy way to provide total ordering now that __cmp__ is de 03/12/09
       http://bugs.python.org/issue5479    created  mrts                          
                                                                               

".egg-info" => ".pkg-info"                                       03/12/09
CLOSED http://bugs.python.org/issue5480    created  zooko                         
                                                                               

Expand Decimal.__format__() support to include "n"               03/12/09
       http://bugs.python.org/issue5481    created  rhettinger                    
                                                                               

RFC: improve distutils bdist_rpm so it builds pure python module 03/13/09
       http://bugs.python.org/issue5482    created  Rudd-O                        
                                                                               

[PATCH]Add FastDbfilenameShelf: shelf nerver sync cache even whe 03/13/09
       http://bugs.python.org/issue5483    created  zhigang                       
       patch                                                                   

subprocess.call() fails for .bat files on Windows, if executable 03/13/09
       http://bugs.python.org/issue5484    created  sandberg                      
                                                                               

pyexpat has no unit tests for UseForeignDTD functionality        03/13/09
       http://bugs.python.org/issue5485    created  exarkun                       
       patch                                                                   



Issues Now Closed (36)
______________________

Update PEP 3135 (super())                                         360 days
       http://bugs.python.org/issue2312    gvanrossum                    
       patch                                                                   

string representation of range and dictionary views               335 days
       http://bugs.python.org/issue2610    benjamin.peterson             
       patch                                                                   

tkSimpleDialog Window Flashing                                    325 days
       http://bugs.python.org/issue2638    gpolo                         
       patch                                                                   

test issue                                                        309 days
       http://bugs.python.org/issue2771    ajaksu2                       
                                                                               

Remove DeprecationWarning in _warnings about 'line'               200 days
       http://bugs.python.org/issue3652    brett.cannon                  
       patch                                                                   

test_bigmem broken                                                193 days
       http://bugs.python.org/issue3700    pitrou                        
       patch                                                                   

2.6rc1: test_threading hangs on FreeBSD 6.3 i386                  179 days
       http://bugs.python.org/issue3863    bms                           
       patch                                                                   

Issue with RotatingFileHandler logging handler on Windows          77 days
       http://bugs.python.org/issue4749    neyro                         
                                                                               

PythonCmd in Modules/_tkinter.c should use the given "interp" pa   65 days
       http://bugs.python.org/issue4792    gpolo                         
       patch                                                                   

Guarantee that Tkinter.Text.search returns a string                25 days
       http://bugs.python.org/issue5193    gpolo                         
       patch                                                                   

Enhance 3.1 IDLE to exec IDLESTARTUP/PYTHONSTARTUP on restart      25 days
       http://bugs.python.org/issue5234    gpolo                         
       patch                                                                   

Distutils in trunk does not work with old Python (2.3 - 2.5)        6 days
       http://bugs.python.org/issue5394    tarek                         
       patch                                                                   

Core dumps on the Solaris buildbot                                  1 days
       http://bugs.python.org/issue5429    pitrou                        
       patch                                                                   

Excessive optimization in IncrementalNewlineDecoder                 0 days
       http://bugs.python.org/issue5433    pitrou                        
       patch                                                                   

string.strip behaves strangly                                       0 days
       http://bugs.python.org/issue5439    loewis                        
                                                                               

string.strip behaves strangly                                       0 days
       http://bugs.python.org/issue5440    loewis                        
                                                                               

doc building progress on Windows doesn't show any color             0 days
       http://bugs.python.org/issue5446    benjamin.peterson             
       patch                                                                   

Add precision property to decimal.Decimal                           1 days
       http://bugs.python.org/issue5448    dlesco                        
                                                                               

patch to make io.StringIO consistent with open with respect to n    0 days
       http://bugs.python.org/issue5451    erickt                        
       patch                                                                   

Documentation for io.StringIO has wrong info for arguments          0 days
       http://bugs.python.org/issue5452    benjamin.peterson             
                                                                               

pydoc -k fails (py3k patch)                                         0 days
       http://bugs.python.org/issue5454    loewis                        
       patch                                                                   

io.StringIO's universal newlines support is broken in 3.0.1         1 days
       http://bugs.python.org/issue5456    benjamin.peterson             
                                                                               

ZipFile writes incorrect modification time (second is off-by-one    1 days
       http://bugs.python.org/issue5457    loewis                        
                                                                               

threading.Thread.join() documentation: missing 'from version' fo    0 days
       http://bugs.python.org/issue5458    georg.brandl                  
                                                                               

Pythonwin Vista Compatibility                                       0 days
       http://bugs.python.org/issue5462    benjamin.peterson             
                                                                               

No edit in IDLE in right click context menu                         0 days
       http://bugs.python.org/issue5465    loewis                        
                                                                               

distutils.test_util fails to restore os.uname; causes test_platf    1 days
       http://bugs.python.org/issue5472    tarek                         
       patch                                                                   

Typo in itertools documentation                                     0 days
       http://bugs.python.org/issue5477    rhettinger                    
                                                                               

".egg-info" => ".pkg-info"                                          0 days
       http://bugs.python.org/issue5480    zooko                         
                                                                               

configure argument --libdir is ignored                           1859 days
       http://bugs.python.org/issue891930  matejcik                      
                                                                               

__name__ available during class dictionary build                 1244 days
       http://bugs.python.org/issue1324261 amaury.forgeotdarc            
                                                                               

ScrolledText hungs up in some conditions                         1169 days
       http://bugs.python.org/issue1390086 amaury.forgeotdarc            
                                                                               

Proper locking with asynchronous callbacks.                      1140 days
       http://bugs.python.org/issue1412632 amaury.forgeotdarc            
       patch                                                                   

Unicode IOError: execfile(u'\u043a\u043a\u043a/x.py')            1130 days
       http://bugs.python.org/issue1422398 amaury.forgeotdarc            
                                                                               

filterwarnings('error') has no effect                             925 days
       http://bugs.python.org/issue1548371 brett.cannon                  
                                                                               

symbol conflict for 'readahead'                                   692 days
       http://bugs.python.org/issue1701192 amaury.forgeotdarc            
                                                                               



Top Issues Most Discussed (10)
______________________________

 13 csv fails when file is opened in binary mode                      67 days
open    http://bugs.python.org/issue4847   

 11 Add named tuple reader to CSV module                             425 days
open    http://bugs.python.org/issue1818   

  8 Allow auto-numbered replacement fields in str.format() strings    29 days
open    http://bugs.python.org/issue5237   

  7 Regexp 2.7 (modifications to current re 2.2.2)                   332 days
open    http://bugs.python.org/issue2636   

  6 test_tcl testLoadTk fails if DISPLAY defined but connect fails,    5 days
open    http://bugs.python.org/issue5450   

  6 codecs.StreamWriter.writelines problem when passed generator       5 days
open    http://bugs.python.org/issue5445   

  5 Add an easy way to provide total ordering now that __cmp__ is d    1 days
open    http://bugs.python.org/issue5479   

  5 distutils produces invalid RPM packages of prerelease python pa    3 days
open    http://bugs.python.org/issue5474   

  5 csv module no longer works as expected when file opened in bina    5 days
open    http://bugs.python.org/issue5455   

  5 test_httpservers on Debian Testing                                 6 days
open    http://bugs.python.org/issue5435   




From martin at v.loewis.de  Fri Mar 13 18:36:22 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Mar 2009 18:36:22 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BA2E61.7070104@simplistix.co.uk>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<49B3B2E8.4050605@simplistix.co.uk> <49B3CD53.5010906@v.loewis.de>
	<49BA2E61.7070104@simplistix.co.uk>
Message-ID: <49BA9996.4010507@v.loewis.de>

>> I don't like the standard library to shrink. It's good that batteries
>> are included.
> 
> If a decent package management system *was* included, this wouldn't be
> an issue..

You can prove anything with a false premise... I believe that a package
management system that is decent cannot possibly be included in Python
(IOW, any packaging system included in Python cannot be decent enough
to allow removal of things from the standard library)

Regards,
Martin

From martin at v.loewis.de  Fri Mar 13 19:27:34 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Mar 2009 19:27:34 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BA3154.8080408@simplistix.co.uk>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BA3154.8080408@simplistix.co.uk>
Message-ID: <49BAA596.5020106@v.loewis.de>

Chris Withers wrote:
> Jim Jewett wrote:
>>>> - python to grow a decent, cross platform, package management system
>>
>> As stated, this may be impossible, because of the difference in what a
>> package should mean on Windows vs Unix.
>>
>> If you just mean a way to add python packages from pypi as with
>> EasyInstall, then maybe.
> 
> I meant package in the python sense, which has a clear definition,
> unless I'm mistaken?

Unfortunately, you are mistaken: the term "package" is highly confusing
and ambiguous in the python sense. It could be a
package-as-import-sees-it, or it could be a package-as-pypi-sees-it.
For the latter, the distutils inventors tried to coin the term
"distribution", but that didn't quite catch on. It *is* the Python
*Package* index, and people do often refer to the things it indexes
as packages.

>> In some environments, each new component must be approved.  Once
>> python is approved, the standard library is OK, but adding 7 packages
>> from pypi requires 7 more sets of approvals.
> 
> True, but as I mentioend elsewhere, I myself haven't done a python
> project where I only needed python and the standard lib for many years...

I was always able to get what I need through aptitude.

Regards,
Martin


From trentm at activestate.com  Fri Mar 13 19:31:47 2009
From: trentm at activestate.com (Trent Mick)
Date: Fri, 13 Mar 2009 11:31:47 -0700
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BA3154.8080408@simplistix.co.uk>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BA3154.8080408@simplistix.co.uk>
Message-ID: <49BAA693.3080709@activestate.com>

Chris Withers wrote:
>> That said, it may make sense to just give greater prominence to
>> existing repackagers, such as ActiveState or Enthought.
> 
> Right, getting ActivePython or similar approved might be the way to go, 
> but I'm betting it depends on the project...

Apologies for jumping in mid-thread here. FYI: We're (where "we" == 
ActiveState here) looking at spending more effort on Python of late. 
Some of our thoughts are on add modules: whether added to the 
ActivePython core or easily addable via an equivalent to ActivePerl's 
ppm (package manager) is still be batted around.

I'm curious as to people's thoughts. I'll also be at PyCon in Chicago 
getting thoughts.

Cheers,
Trent

-- 
Trent Mick
trentm at activestate.com

From martin at v.loewis.de  Fri Mar 13 19:31:21 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Mar 2009 19:31:21 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <873EFBC3-E729-4C02-B941-38D0C5314035@zope.com>
References: <gp6hhq$hss$1@ger.gmane.org>	<49B85D42.9080406@gmail.com>	<loom.20090312T010957-202@post.gmane.org>	<200903122254.21316.steve@pearwood.info>	<loom.20090312T122030-705@post.gmane.org>	<49B91FAB.2010101@gmail.com>
	<49B95F43.3000708@v.loewis.de>
	<873EFBC3-E729-4C02-B941-38D0C5314035@zope.com>
Message-ID: <49BAA679.3010307@v.loewis.de>

> Think about the security implications of a file name that is in advance
> known to an attacker as well as the fact that the said file will replace
> an *important* system file.

You should always use O_EXCL in that case. Relying on random name will
be a severe security threat to the application.

Regards,
Martin

From zvezdan at zope.com  Fri Mar 13 20:01:03 2009
From: zvezdan at zope.com (Zvezdan Petkovic)
Date: Fri, 13 Mar 2009 15:01:03 -0400
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49BAA679.3010307@v.loewis.de>
References: <gp6hhq$hss$1@ger.gmane.org>	<49B85D42.9080406@gmail.com>	<loom.20090312T010957-202@post.gmane.org>	<200903122254.21316.steve@pearwood.info>	<loom.20090312T122030-705@post.gmane.org>	<49B91FAB.2010101@gmail.com>
	<49B95F43.3000708@v.loewis.de>
	<873EFBC3-E729-4C02-B941-38D0C5314035@zope.com>
	<49BAA679.3010307@v.loewis.de>
Message-ID: <4B617AD7-DA61-4946-A4D9-EE1810213185@zope.com>

On Mar 13, 2009, at 2:31 PM, Martin v. L?wis wrote:

>> Think about the security implications of a file name that is in  
>> advance known to an attacker as well as the fact that the said file  
>> will replace an *important* system file.
>
> You should always use O_EXCL in that case. Relying on random name will
> be a severe security threat to the application.

If you read an implementation of mkstemp() function, you'll see that  
it does exactly that:

	if ((*doopen = open(path, O_CREAT|O_EXCL|O_RDWR, 0600)) >= 0)
		return(1);
	if (errno != EEXIST)
		return(0);

That's why I mentioned mkstemp() in the OP.

	Zvezdan


From amcnabb at mcnabbs.org  Fri Mar 13 19:42:30 2009
From: amcnabb at mcnabbs.org (Andrew McNabb)
Date: Fri, 13 Mar 2009 12:42:30 -0600
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49BAA679.3010307@v.loewis.de>
References: <gp6hhq$hss$1@ger.gmane.org> <49B85D42.9080406@gmail.com>
	<loom.20090312T010957-202@post.gmane.org>
	<200903122254.21316.steve@pearwood.info>
	<loom.20090312T122030-705@post.gmane.org>
	<49B91FAB.2010101@gmail.com> <49B95F43.3000708@v.loewis.de>
	<873EFBC3-E729-4C02-B941-38D0C5314035@zope.com>
	<49BAA679.3010307@v.loewis.de>
Message-ID: <20090313184230.GJ19975@mcnabbs.org>

On Fri, Mar 13, 2009 at 07:31:21PM +0100, "Martin v. L?wis" wrote:
> > Think about the security implications of a file name that is in advance
> > known to an attacker as well as the fact that the said file will replace
> > an *important* system file.
> 
> You should always use O_EXCL in that case. Relying on random name will
> be a severe security threat to the application.

But mkstemp does open files with O_EXCL, so the two approaches really
aren't that different.  Using tempfile can be a little simpler because
it will eventually succeed.

-- 
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55  8012 AB4D 6098 8826 6868

From p.f.moore at gmail.com  Fri Mar 13 23:42:28 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 13 Mar 2009 22:42:28 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BA6009.5070605@palladion.com>
References: <49ABCFC9.1070508@gmail.com>
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>
	<gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<49B3B2E8.4050605@simplistix.co.uk> <49B3CD53.5010906@v.loewis.de>
	<49BA2E61.7070104@simplistix.co.uk>
	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>
	<49BA6009.5070605@palladion.com>
Message-ID: <79990c6b0903131542hdfad7fbi1964cba097d497b7@mail.gmail.com>

2009/3/13 Tres Seaver <tseaver at palladion.com>:
> Paul Moore wrote:
>> 2009/3/13 Chris Withers <chris at simplistix.co.uk>:
>>> If a decent package management system *was* included, this wouldn't be an
>>> issue..
>>
>> Remember that a "decent package management system" needs to handle
>> filling in all the forms and arranging approvals to get authorisation
>> for packages when you download them.
>>
>> And no, I'm *not* joking. People in a locked-down corporate
>> environment really do benefit from just having to get the OK for
>> "Python", and then knowing that they have all they need.
>
> You are plainly joking: ?nothing in Python should know or care about the
> various bureaucratic insanities in some workplaces.

I am not. What I *am* doing is saying (obliquely, I admit) is that for
a package management system to be "decent enough" for stripping down
the stdlib to not be an issue, it has to address these points (which
clearly it can't). In other words, the problems inherent in
restricting the stdlib aren't ones which are solvable by a package
management system.

Paul.

From ncoghlan at gmail.com  Sat Mar 14 04:04:22 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 14 Mar 2009 13:04:22 +1000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BA6009.5070605@palladion.com>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>
	<49BA6009.5070605@palladion.com>
Message-ID: <49BB1EB6.90702@gmail.com>

Tres Seaver wrote:
> You are plainly joking:  nothing in Python should know or care about the
> various bureaucratic insanities in some workplaces.  Given the
> *existing* stdlib and network connectivity, nothing any corporate
> security blackshirt can do will prevent an even moderately-motivated
> person from executing arbitrary code downloaded from elsewhere.  In that
> case, what is the point in trying to help those who impose such craziness?

Network connectivity isn't a given, even today. So yes, there are
environments that are secure (i.e. no network connectivity), and there
are environments where developers are trusted (shock, horror) to
actually follow company policy and get all licenses vetted by their
Contracts group before installing downloaded software on company machines.

Given that even some of the core developers work in environments like
that, then yes, I believe Python can and should take reasonable steps to
enable its use in such situations.

And the most reasonably step Python can take on that front is to
continue to provide a relatively powerful standard library *even if* a
flexible and otherwise useful package management approach is added at
some stage.

If someone else decides to create a MinimalPython which consists solely
of something like easy_install and whatever is needed to run it (i.e.
the opposite of the "fat" bundles from folks like ActiveState and
Enthought), then more power to them. But I don't believe the official
releases from python.org should go that way.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Sat Mar 14 04:09:51 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 14 Mar 2009 13:09:51 +1000
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <873EFBC3-E729-4C02-B941-38D0C5314035@zope.com>
References: <gp6hhq$hss$1@ger.gmane.org>	<49B85D42.9080406@gmail.com>	<loom.20090312T010957-202@post.gmane.org>	<200903122254.21316.steve@pearwood.info>	<loom.20090312T122030-705@post.gmane.org>	<49B91FAB.2010101@gmail.com>
	<49B95F43.3000708@v.loewis.de>
	<873EFBC3-E729-4C02-B941-38D0C5314035@zope.com>
Message-ID: <49BB1FFF.4080105@gmail.com>

Zvezdan Petkovic wrote:
> Of course, the above are C functions.  I don't think that Python
> programming is immune from such security considerations either.

The tempfile module exposes the same functionality (and uses mkstemp()
to create its filenames). It has also had features added over the years
to prevent automatic deletion of the temporary files, precisely so you
*can* grab them and rename them afterwards.

It actually wouldn't be a bad place to put a "create a temporary file
and rename it to <name> when closing it" helper class. Such a utility
could also include a way to request "fsync() before rename" behaviour
(off by default of course).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From lists at janc.be  Sat Mar 14 05:08:48 2009
From: lists at janc.be (Jan Claeys)
Date: Sat, 14 Mar 2009 05:08:48 +0100
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <20090313092807.GA27105@phd.pp.ru>
References: <gp6hhq$hss$1@ger.gmane.org>
	<aac2c7cb0903122114i22d89b9dx88c15176dce7d739@mail.gmail.com>
	<20090313092807.GA27105@phd.pp.ru>
Message-ID: <1237003728.16233.581.camel@saeko.local>

Op vrijdag 13-03-2009 om 12:28 uur [tijdzone +0300], schreef Oleg
Broytmann:
>    Ext4 is not the only FS with delayed allocation.

Of course not, even ext3 has delayed allocation (even if 5 sec vs. 2 min
makes the disaster window a bit smaller).

The real problem seems to be that ext4 re-orders the rename (which it
does almost instantly) before the write (which waits for 2-15 minutes or
so).

There are other modern filesystems that take care such reordering
doesn't happen...


-- 
Jan Claeys


From lists at janc.be  Sat Mar 14 05:00:06 2009
From: lists at janc.be (Jan Claeys)
Date: Sat, 14 Mar 2009 05:00:06 +0100
Subject: [Python-Dev] Fwd: installation with ubuntu 8.04
In-Reply-To: <3699312a0903130413l28b27f4aof72d763273e66e0b@mail.gmail.com>
References: <mailman.6726.1236915178.11745.python-3000@python.org>
	<3699312a0903130413l28b27f4aof72d763273e66e0b@mail.gmail.com>
Message-ID: <1237003206.16233.575.camel@saeko.local>

Op vrijdag 13-03-2009 om 04:13 uur [tijdzone -0700], schreef John
Wagner:

> I am sorry to bother about this but I've asked for help from Canonical
> with no reply.

> They seem reluctant to update Python beyond version 2.5; I'm trying to
> install 3.0.1.

If you can wait until April 23rd then you will be able to use python 2.6
(as the default) or python 3.0.1 (as an option for your apps) on the
next release of Ubuntu (9.04).

Maybe Ubuntu will provide a package of python 3.0.1 in the 8.04
backports too, if there is enough demand for it.  And if not, somebody
can make a backport for Ubuntu 8.04 in a PPA on launchpad.


-- 
Jan Claeys


From tseaver at palladion.com  Sat Mar 14 05:15:20 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 14 Mar 2009 00:15:20 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BB1EB6.90702@gmail.com>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>	<49BA6009.5070605@palladion.com>
	<49BB1EB6.90702@gmail.com>
Message-ID: <49BB2F58.60203@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nick Coghlan wrote:
> Tres Seaver wrote:
>> You are plainly joking:  nothing in Python should know or care about the
>> various bureaucratic insanities in some workplaces.  Given the
>> *existing* stdlib and network connectivity, nothing any corporate
>> security blackshirt can do will prevent an even moderately-motivated
>> person from executing arbitrary code downloaded from elsewhere.  In that
>> case, what is the point in trying to help those who impose such craziness?
> 
> Network connectivity isn't a given, even today. So yes, there are
> environments that are secure (i.e. no network connectivity), and there
> are environments where developers are trusted (shock, horror) to
> actually follow company policy and get all licenses vetted by their
> Contracts group before installing downloaded software on company machines.
> 
> Given that even some of the core developers work in environments like
> that, then yes, I believe Python can and should take reasonable steps to
> enable its use in such situations.
> 
> And the most reasonably step Python can take on that front is to
> continue to provide a relatively powerful standard library *even if* a
> flexible and otherwise useful package management approach is added at
> some stage.

My inclination would be to leave the stdlib largely as is, except that
occostonally I would argue for ripping out a particular obsolete /
bitrotted module.

A couple of other points:

- - Absent a sufficiently powerful package management system, the pressure
  to add modules to the stdlib (or keep them) is higher because it is
  harder for *all* Python users to add them, or replace them if dropped.

- - The choice to add or remove a module to / from the stdlib should be
  made on the merits of the module, without regard to the kind of
  specialized deployment policies you outline.

- - Changing the stdlib in a new release of Python is probably irrelevant
  for the kind of environments you allude to, as there is likely as much
  review involved to approve a new version of Python as there was in
  approving it in the first place:  of the few I know of today, all are
  still running Python 2.1.x and / or 2.2.x for this reason.

> If someone else decides to create a MinimalPython which consists solely
> of something like easy_install and whatever is needed to run it (i.e.
> the opposite of the "fat" bundles from folks like ActiveState and
> Enthought), then more power to them. But I don't believe the official
> releases from python.org should go that way.

Note that I am *not* advocating scrubbing / exploding the stdlib.



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJuy9Y+gerLs4ltQ4RAranAJ4rCXgq0opHPki6OmlABbaqE3D1sQCeJ7Zt
Em6VMK1u+6+xYsoqixwfoJ4=
=YzN7
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Sat Mar 14 05:15:20 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 14 Mar 2009 00:15:20 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BB1EB6.90702@gmail.com>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>	<49BA6009.5070605@palladion.com>
	<49BB1EB6.90702@gmail.com>
Message-ID: <49BB2F58.60203@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nick Coghlan wrote:
> Tres Seaver wrote:
>> You are plainly joking:  nothing in Python should know or care about the
>> various bureaucratic insanities in some workplaces.  Given the
>> *existing* stdlib and network connectivity, nothing any corporate
>> security blackshirt can do will prevent an even moderately-motivated
>> person from executing arbitrary code downloaded from elsewhere.  In that
>> case, what is the point in trying to help those who impose such craziness?
> 
> Network connectivity isn't a given, even today. So yes, there are
> environments that are secure (i.e. no network connectivity), and there
> are environments where developers are trusted (shock, horror) to
> actually follow company policy and get all licenses vetted by their
> Contracts group before installing downloaded software on company machines.
> 
> Given that even some of the core developers work in environments like
> that, then yes, I believe Python can and should take reasonable steps to
> enable its use in such situations.
> 
> And the most reasonably step Python can take on that front is to
> continue to provide a relatively powerful standard library *even if* a
> flexible and otherwise useful package management approach is added at
> some stage.

My inclination would be to leave the stdlib largely as is, except that
occostonally I would argue for ripping out a particular obsolete /
bitrotted module.

A couple of other points:

- - Absent a sufficiently powerful package management system, the pressure
  to add modules to the stdlib (or keep them) is higher because it is
  harder for *all* Python users to add them, or replace them if dropped.

- - The choice to add or remove a module to / from the stdlib should be
  made on the merits of the module, without regard to the kind of
  specialized deployment policies you outline.

- - Changing the stdlib in a new release of Python is probably irrelevant
  for the kind of environments you allude to, as there is likely as much
  review involved to approve a new version of Python as there was in
  approving it in the first place:  of the few I know of today, all are
  still running Python 2.1.x and / or 2.2.x for this reason.

> If someone else decides to create a MinimalPython which consists solely
> of something like easy_install and whatever is needed to run it (i.e.
> the opposite of the "fat" bundles from folks like ActiveState and
> Enthought), then more power to them. But I don't believe the official
> releases from python.org should go that way.

Note that I am *not* advocating scrubbing / exploding the stdlib.



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJuy9Y+gerLs4ltQ4RAranAJ4rCXgq0opHPki6OmlABbaqE3D1sQCeJ7Zt
Em6VMK1u+6+xYsoqixwfoJ4=
=YzN7
-----END PGP SIGNATURE-----


From steve at pearwood.info  Sat Mar 14 05:27:52 2009
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 14 Mar 2009 15:27:52 +1100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BB2F58.60203@palladion.com>
References: <49ABCFC9.1070508@gmail.com> <49BB1EB6.90702@gmail.com>
	<49BB2F58.60203@palladion.com>
Message-ID: <200903141527.52458.steve@pearwood.info>

Tres, for some reason every time you reply to the list, you send TWO 
copies instead of one:

To: python-dev at python.org
CC: Python Dev <python-dev at python.org>

Could you please fix that?


-- 
Steven D'Aprano

From tseaver at palladion.com  Sat Mar 14 05:41:42 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 14 Mar 2009 00:41:42 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <200903141527.52458.steve@pearwood.info>
References: <49ABCFC9.1070508@gmail.com>
	<49BB1EB6.90702@gmail.com>	<49BB2F58.60203@palladion.com>
	<200903141527.52458.steve@pearwood.info>
Message-ID: <gpfcii$aon$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Steven D'Aprano wrote:
> Tres, for some reason every time you reply to the list, you send TWO 
> copies instead of one:
> 
> To: python-dev at python.org
> CC: Python Dev <python-dev at python.org>
> 
> Could you please fix that?

I  can try:  I normally post via gmane, and leave python-dev CC'ed so
that folks who read via the list don't have their replies to me fall off
list (which happens on some lists, anyway).

I will trim the CC in the future, and hope for the best.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJuzWG+gerLs4ltQ4RAsQfAKCBGfeI6FEP8cNbOdh0zFhLjj65CgCgiLZb
725QgMYFCyhdM6OP5+SC04U=
=yRwI
-----END PGP SIGNATURE-----


From stefan_ml at behnel.de  Sat Mar 14 07:42:41 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 14 Mar 2009 07:42:41 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <gpfcii$aon$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com>	<49BB1EB6.90702@gmail.com>	<49BB2F58.60203@palladion.com>	<200903141527.52458.steve@pearwood.info>
	<gpfcii$aon$1@ger.gmane.org>
Message-ID: <gpfjl1$ljp$1@ger.gmane.org>

Tres Seaver wrote:
> Steven D'Aprano wrote:
>> Tres, for some reason every time you reply to the list, you send TWO 
>> copies instead of one:
> 
>> To: python-dev at python.org
>> CC: Python Dev <python-dev at python.org>
> 
>> Could you please fix that?
> 
> I  can try:  I normally post via gmane, and leave python-dev CC'ed so
> that folks who read via the list don't have their replies to me fall off
> list (which happens on some lists, anyway).
> 
> I will trim the CC in the future, and hope for the best.

That's what works best for me, anyway.

Stefan


From jaraco at jaraco.com  Sat Mar 14 15:30:19 2009
From: jaraco at jaraco.com (Jason R. Coombs)
Date: Sat, 14 Mar 2009 10:30:19 -0400
Subject: [Python-Dev] Python 2.6.2 and 3.0.2
Message-ID: <8B473FAE8A08C34C9F5666FD4B0A87B640AFF09987@hornigold.jaraco.com>

I'm still holding my breath for Python 2.6.2, which fixes a Windows DLL linking issue that was already fixed in 3.0.1.  Obviously, the proposed schedule has passed, but I would prefer a release sooner than later.

Of course, that's just my preference.

Regards,
Jason

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Thinking again about 3.0.2.
>
> If we'd like to do bug fix releases before Pycon, I suggest Monday March
> 9th for code freeze and tagging.  That would mean a Tuesday March 10th
> release.
>
> What do you think?


From martin at v.loewis.de  Sat Mar 14 21:39:21 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Mar 2009 21:39:21 +0100
Subject: [Python-Dev] Python 2.6.2 and 3.0.2
In-Reply-To: <8B473FAE8A08C34C9F5666FD4B0A87B640AFF09987@hornigold.jaraco.com>
References: <8B473FAE8A08C34C9F5666FD4B0A87B640AFF09987@hornigold.jaraco.com>
Message-ID: <49BC15F9.4020100@v.loewis.de>

Jason R. Coombs wrote:
> I'm still holding my breath for Python 2.6.2, which fixes a Windows
> DLL linking issue that was already fixed in 3.0.1.  Obviously, the
> proposed schedule has passed, but I would prefer a release sooner
> than later.

I suggest you start breathing again.

Martin

From ncoghlan at gmail.com  Sun Mar 15 13:56:14 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 15 Mar 2009 22:56:14 +1000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
Message-ID: <49BCFAEE.2070603@gmail.com>

PEP 377 is a proposal to allow context manager __enter__() methods to
skip the body of the with statement by raising a specific (new) flow
control exception.

Since there is a working reference implementation now, I thought it was
time to open it up for broader discussion.

Full PEP attached, or you can find it in the usual place at
http://www.python.org/dev/peps/pep-0377

Cheers,
Nick.

P.S. I expect a rationale for the StatementSkipped value binding is
probably going to be pretty high on the list of questions that aren't
currently covered by the PEP. I hope to write more on that some time
this week.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pep-0377.txt
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090315/ba1df1e1/attachment.txt>

From brett at python.org  Sun Mar 15 18:40:03 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 15 Mar 2009 10:40:03 -0700
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
In-Reply-To: <49BCFAEE.2070603@gmail.com>
References: <49BCFAEE.2070603@gmail.com>
Message-ID: <bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>

Without knowing what StatementSkipped is (just some singleton? If so why not
just used SkipStatement instance that was raised?) and wondering if we are
just going to continue to adding control flow exceptions that directly
inherit from BaseException or some ControlFlowException base class, the
basic idea seems fine by me.

On Sun, Mar 15, 2009 at 05:56, Nick Coghlan <ncoghlan at gmail.com> wrote:

> PEP 377 is a proposal to allow context manager __enter__() methods to
> skip the body of the with statement by raising a specific (new) flow
> control exception.
>
> Since there is a working reference implementation now, I thought it was
> time to open it up for broader discussion.
>
> Full PEP attached, or you can find it in the usual place at
> http://www.python.org/dev/peps/pep-0377
>
> Cheers,
> Nick.
>
> P.S. I expect a rationale for the StatementSkipped value binding is
> probably going to be pretty high on the list of questions that aren't
> currently covered by the PEP. I hope to write more on that some time
> this week.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>
> PEP: 377
> Title: Allow __enter__() methods to skip the statement body
> Version: $Revision: 70384 $
> Last-Modified: $Date: 2009-03-15 22:48:49 +1000 (Sun, 15 Mar 2009) $
> Author: Nick Coghlan <ncoghlan at gmail.com>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 8-Mar-2009
> Python-Version: 2.7, 3.1
> Post-History: 8-Mar-2009
>
>
> Abstract
> ========
>
> This PEP proposes a backwards compatible mechanism that allows
> ``__enter__()``
> methods to skip the body of the associated ``with`` statement. The lack of
> this ability currently means the ``contextlib.contextmanager`` decorator
> is unable to fulfil its specification of being able to turn arbitrary
> code into a context manager by moving it into a generator function
> with a yield in the appropriate location. One symptom of this is that
> ``contextlib.nested`` will currently raise ``RuntimeError`` in
> situations where writing out the corresponding nested ``with``
> statements would not [1].
>
> The proposed change is to introduce a new flow control exception
> ``SkipStatement``, and skip the execution of the ``with``
> statement body if ``__enter__()`` raises this exception.
>
>
> Proposed Change
> ===============
>
> The semantics of the ``with`` statement will be changed to include a
> new ``try``/``except``/``else`` block around the call to ``__enter__()``.
> If ``SkipStatement`` is raised by the ``__enter__()`` method, then
> the main section of the ``with`` statement (now located in the ``else``
> clause) will not be executed. To avoid leaving the names in any ``as``
> clause unbound in this case, a new ``StatementSkipped`` singleton
> (similar to the existing ``NotImplemented`` singleton) will be
> assigned to all names that appear in the ``as`` clause.
>
> The components of the ``with`` statement remain as described in PEP 343
> [2]::
>
>    with EXPR as VAR:
>        BLOCK
>
> After the modification, the ``with`` statement semantics would
> be as follows::
>
>    mgr = (EXPR)
>    exit = mgr.__exit__  # Not calling it yet
>    try:
>        value = mgr.__enter__()
>    except SkipStatement:
>        VAR = StatementSkipped
>        # Only if "as VAR" is present and
>        # VAR is a single name
>        # If VAR is a tuple of names, then StatementSkipped
>        # will be assigned to each name in the tuple
>    else:
>        exc = True
>        try:
>            try:
>                VAR = value  # Only if "as VAR" is present
>                BLOCK
>            except:
>                # The exceptional case is handled here
>                exc = False
>                if not exit(*sys.exc_info()):
>                    raise
>                # The exception is swallowed if exit() returns true
>        finally:
>            # The normal and non-local-goto cases are handled here
>            if exc:
>                exit(None, None, None)
>
> With the above change in place for the ``with`` statement semantics,
> ``contextlib.contextmanager()`` will then be modified to raise
> ``SkipStatement`` instead of ``RuntimeError`` when the underlying
> generator doesn't yield.
>
>
> Rationale for Change
> ====================
>
> Currently, some apparently innocuous context managers may raise
> ``RuntimeError`` when executed. This occurs when the context
> manager's ``__enter__()`` method encounters a situation where
> the written out version of the code corresponding to the
> context manager would skip the code that is now the body
> of the ``with`` statement. Since the ``__enter__()`` method
> has no mechanism available to signal this to the interpreter,
> it is instead forced to raise an exception that not only
> skips the body of the ``with`` statement, but also jumps over
> all code until the nearest exception handler. This goes against
> one of the design goals of the ``with`` statement, which was to
> be able to factor out arbitrary common exception handling code
> into a single context manager by putting into a generator
> function and replacing the variant part of the code with a
> ``yield`` statement.
>
> Specifically, the following examples behave differently if
> ``cmB().__enter__()`` raises an exception which ``cmA().__exit__()``
> then handles and suppresses::
>
>  with cmA():
>    with cmB():
>      do_stuff()
>  # This will resume here without executing "do_stuff()"
>
>  @contextlib.contextmanager
>  def combined():
>    with cmA():
>      with cmB():
>        yield
>
>  with combined():
>    do_stuff()
>  # This will raise a RuntimeError complaining that the context
>  # manager's underlying generator didn't yield
>
>  with contextlib.nested(cmA(), cmB()):
>    do_stuff()
>  # This will raise the same RuntimeError as the contextmanager()
>  # example (unsurprising, given that the nested() implementation
>  # uses contextmanager())
>
>  # The following class based version shows that the issue isn't
>  # specific to contextlib.contextmanager() (it also shows how
>  # much simpler it is to write context managers as generators
>  # instead of as classes!)
>  class CM(object):
>    def __init__(self):
>      self.cmA = None
>      self.cmB = None
>
>    def __enter__(self):
>      if self.cmA is not None:
>        raise RuntimeError("Can't re-use this CM")
>      self.cmA = cmA()
>      self.cmA.__enter__()
>      try:
>        self.cmB = cmB()
>        self.cmB.__enter__()
>      except:
>        self.cmA.__exit__(*sys.exc_info())
>        # Can't suppress in __enter__(), so must raise
>        raise
>
>    def __exit__(self, *args):
>      suppress = False
>      try:
>        if self.cmB is not None:
>          suppress = self.cmB.__exit__(*args)
>      except:
>        suppress = self.cmA.__exit__(*sys.exc_info()):
>        if not suppress:
>          # Exception has changed, so reraise explicitly
>          raise
>      else:
>        if suppress:
>          # cmB already suppressed the exception,
>          # so don't pass it to cmA
>          suppress = self.cmA.__exit__(None, None, None):
>        else:
>          suppress = self.cmA.__exit__(*args):
>      return suppress
>
> With the proposed semantic change in place, the contextlib based examples
> above would then "just work", but the class based version would need
> a small adjustment to take advantage of the new semantics::
>
>  class CM(object):
>    def __init__(self):
>      self.cmA = None
>      self.cmB = None
>
>    def __enter__(self):
>      if self.cmA is not None:
>        raise RuntimeError("Can't re-use this CM")
>      self.cmA = cmA()
>      self.cmA.__enter__()
>      try:
>        self.cmB = cmB()
>        self.cmB.__enter__()
>      except:
>        if self.cmA.__exit__(*sys.exc_info()):
>          # Suppress the exception, but don't run
>          # the body of the with statement either
>          raise SkipStatement
>        raise
>
>    def __exit__(self, *args):
>      suppress = False
>      try:
>        if self.cmB is not None:
>          suppress = self.cmB.__exit__(*args)
>      except:
>        suppress = self.cmA.__exit__(*sys.exc_info()):
>        if not suppress:
>          # Exception has changed, so reraise explicitly
>          raise
>      else:
>        if suppress:
>          # cmB already suppressed the exception,
>          # so don't pass it to cmA
>          suppress = self.cmA.__exit__(None, None, None):
>        else:
>          suppress = self.cmA.__exit__(*args):
>      return suppress
>
> There is currently a tentative suggestion [3] to add import-style syntax to
> the ``with`` statement to allow multiple context managers to be included in
> a single ``with`` statement without needing to use ``contextlib.nested``.
> In
> that case the compiler has the option of simply emitting multiple ``with``
> statements at the AST level, thus allowing the semantics of actual nested
> ``with`` statements to be reproduced accurately. However, such a change
> would highlight rather than alleviate the problem the current PEP aims to
> address: it would not be possible to use ``contextlib.contextmanager`` to
> reliably factor out such ``with`` statements, as they would exhibit exactly
> the same semantic differences as are seen with the ``combined()`` context
> manager in the above example.
>
>
> Performance Impact
> ==================
>
> Implementing the new semantics makes it necessary to store the references
> to the ``__enter__`` and ``__exit__`` methods in temporary variables
> instead
> of on the stack. This results in a slight regression in ``with`` statement
> speed relative to Python 2.6/3.1. However, implementing a custom
> ``SETUP_WITH`` opcode would negate any differences between the two
> approaches (as well as dramatically improving speed by eliminating more
> than a dozen unnecessary trips around the eval loop).
>
>
> Reference Implementation
> ========================
>
> Patch attached to Issue 5251 [1]. That patch uses only existing opcodes
> (i.e. no ``SETUP_WITH``).
>
>
> Acknowledgements
> ================
>
> James William Pye both raised the issue and suggested the basic outline of
> the solution described in this PEP.
>
>
> References
> ==========
>
> .. [1] Issue 5251: contextlib.nested inconsistent with nested with
> statements
>   (http://bugs.python.org/issue5251)
>
> .. [2] PEP 343: The "with" Statement
>   (http://www.python.org/dev/peps/pep-0343/)
>
> .. [3] Import-style syntax to reduce indentation of nested with statements
>   (http://mail.python.org/pipermail/python-ideas/2009-March/003188.html)
>
>
> Copyright
> =========
>
> This document has been placed in the public domain.
>
> ..
>   Local Variables:
>   mode: indented-text
>   indent-tabs-mode: nil
>   sentence-end-double-space: t
>   fill-column: 70
>   End:
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090315/431dcbfb/attachment-0001.htm>

From fuzzyman at voidspace.org.uk  Sun Mar 15 18:50:15 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 15 Mar 2009 17:50:15 +0000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
References: <49BCFAEE.2070603@gmail.com>
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
Message-ID: <49BD3FD7.3080309@voidspace.org.uk>

Brett Cannon wrote:
> Without knowing what StatementSkipped is (just some singleton? If so 
> why not just used SkipStatement instance that was raised?) and 
> wondering if we are just going to continue to adding control flow 
> exceptions that directly inherit from BaseException or some 
> ControlFlowException base class, the basic idea seems fine by me.
>

Note that using exceptions for control flow can  be bad for other 
implementations of Python. For example exceptions on the .NET framework 
are very expensive. (Although there are workarounds such as not really 
raising the exception - but they're ugly).

Isn't it better practise for exceptions to be used for exceptional 
circumstances rather than for control flow?

Michael

> On Sun, Mar 15, 2009 at 05:56, Nick Coghlan <ncoghlan at gmail.com 
> <mailto:ncoghlan at gmail.com>> wrote:
>
>     PEP 377 is a proposal to allow context manager __enter__() methods to
>     skip the body of the with statement by raising a specific (new) flow
>     control exception.
>
>     Since there is a working reference implementation now, I thought
>     it was
>     time to open it up for broader discussion.
>
>     Full PEP attached, or you can find it in the usual place at
>     http://www.python.org/dev/peps/pep-0377
>
>     Cheers,
>     Nick.
>
>     P.S. I expect a rationale for the StatementSkipped value binding is
>     probably going to be pretty high on the list of questions that aren't
>     currently covered by the PEP. I hope to write more on that some time
>     this week.
>
>     --
>     Nick Coghlan   |   ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>
>       |   Brisbane, Australia
>     ---------------------------------------------------------------
>
>     PEP: 377
>     Title: Allow __enter__() methods to skip the statement body
>     Version: $Revision: 70384 $
>     Last-Modified: $Date: 2009-03-15 22:48:49 +1000 (Sun, 15 Mar 2009) $
>     Author: Nick Coghlan <ncoghlan at gmail.com <mailto:ncoghlan at gmail.com>>
>     Status: Draft
>     Type: Standards Track
>     Content-Type: text/x-rst
>     Created: 8-Mar-2009
>     Python-Version: 2.7, 3.1
>     Post-History: 8-Mar-2009
>
>
>     Abstract
>     ========
>
>     This PEP proposes a backwards compatible mechanism that allows
>     ``__enter__()``
>     methods to skip the body of the associated ``with`` statement. The
>     lack of
>     this ability currently means the ``contextlib.contextmanager``
>     decorator
>     is unable to fulfil its specification of being able to turn arbitrary
>     code into a context manager by moving it into a generator function
>     with a yield in the appropriate location. One symptom of this is that
>     ``contextlib.nested`` will currently raise ``RuntimeError`` in
>     situations where writing out the corresponding nested ``with``
>     statements would not [1].
>
>     The proposed change is to introduce a new flow control exception
>     ``SkipStatement``, and skip the execution of the ``with``
>     statement body if ``__enter__()`` raises this exception.
>
>
>     Proposed Change
>     ===============
>
>     The semantics of the ``with`` statement will be changed to include a
>     new ``try``/``except``/``else`` block around the call to
>     ``__enter__()``.
>     If ``SkipStatement`` is raised by the ``__enter__()`` method, then
>     the main section of the ``with`` statement (now located in the
>     ``else``
>     clause) will not be executed. To avoid leaving the names in any ``as``
>     clause unbound in this case, a new ``StatementSkipped`` singleton
>     (similar to the existing ``NotImplemented`` singleton) will be
>     assigned to all names that appear in the ``as`` clause.
>
>     The components of the ``with`` statement remain as described in
>     PEP 343 [2]::
>
>        with EXPR as VAR:
>            BLOCK
>
>     After the modification, the ``with`` statement semantics would
>     be as follows::
>
>        mgr = (EXPR)
>        exit = mgr.__exit__  # Not calling it yet
>        try:
>            value = mgr.__enter__()
>        except SkipStatement:
>            VAR = StatementSkipped
>            # Only if "as VAR" is present and
>            # VAR is a single name
>            # If VAR is a tuple of names, then StatementSkipped
>            # will be assigned to each name in the tuple
>        else:
>            exc = True
>            try:
>                try:
>                    VAR = value  # Only if "as VAR" is present
>                    BLOCK
>                except:
>                    # The exceptional case is handled here
>                    exc = False
>                    if not exit(*sys.exc_info()):
>                        raise
>                    # The exception is swallowed if exit() returns true
>            finally:
>                # The normal and non-local-goto cases are handled here
>                if exc:
>                    exit(None, None, None)
>
>     With the above change in place for the ``with`` statement semantics,
>     ``contextlib.contextmanager()`` will then be modified to raise
>     ``SkipStatement`` instead of ``RuntimeError`` when the underlying
>     generator doesn't yield.
>
>
>     Rationale for Change
>     ====================
>
>     Currently, some apparently innocuous context managers may raise
>     ``RuntimeError`` when executed. This occurs when the context
>     manager's ``__enter__()`` method encounters a situation where
>     the written out version of the code corresponding to the
>     context manager would skip the code that is now the body
>     of the ``with`` statement. Since the ``__enter__()`` method
>     has no mechanism available to signal this to the interpreter,
>     it is instead forced to raise an exception that not only
>     skips the body of the ``with`` statement, but also jumps over
>     all code until the nearest exception handler. This goes against
>     one of the design goals of the ``with`` statement, which was to
>     be able to factor out arbitrary common exception handling code
>     into a single context manager by putting into a generator
>     function and replacing the variant part of the code with a
>     ``yield`` statement.
>
>     Specifically, the following examples behave differently if
>     ``cmB().__enter__()`` raises an exception which ``cmA().__exit__()``
>     then handles and suppresses::
>
>      with cmA():
>        with cmB():
>          do_stuff()
>      # This will resume here without executing "do_stuff()"
>
>      @contextlib.contextmanager
>      def combined():
>        with cmA():
>          with cmB():
>            yield
>
>      with combined():
>        do_stuff()
>      # This will raise a RuntimeError complaining that the context
>      # manager's underlying generator didn't yield
>
>      with contextlib.nested(cmA(), cmB()):
>        do_stuff()
>      # This will raise the same RuntimeError as the contextmanager()
>      # example (unsurprising, given that the nested() implementation
>      # uses contextmanager())
>
>      # The following class based version shows that the issue isn't
>      # specific to contextlib.contextmanager() (it also shows how
>      # much simpler it is to write context managers as generators
>      # instead of as classes!)
>      class CM(object):
>        def __init__(self):
>          self.cmA = None
>          self.cmB = None
>
>        def __enter__(self):
>          if self.cmA is not None:
>            raise RuntimeError("Can't re-use this CM")
>          self.cmA = cmA()
>          self.cmA.__enter__()
>          try:
>            self.cmB = cmB()
>            self.cmB.__enter__()
>          except:
>            self.cmA.__exit__(*sys.exc_info())
>            # Can't suppress in __enter__(), so must raise
>            raise
>
>        def __exit__(self, *args):
>          suppress = False
>          try:
>            if self.cmB is not None:
>              suppress = self.cmB.__exit__(*args)
>          except:
>            suppress = self.cmA.__exit__(*sys.exc_info()):
>            if not suppress:
>              # Exception has changed, so reraise explicitly
>              raise
>          else:
>            if suppress:
>              # cmB already suppressed the exception,
>              # so don't pass it to cmA
>              suppress = self.cmA.__exit__(None, None, None):
>            else:
>              suppress = self.cmA.__exit__(*args):
>          return suppress
>
>     With the proposed semantic change in place, the contextlib based
>     examples
>     above would then "just work", but the class based version would need
>     a small adjustment to take advantage of the new semantics::
>
>      class CM(object):
>        def __init__(self):
>          self.cmA = None
>          self.cmB = None
>
>        def __enter__(self):
>          if self.cmA is not None:
>            raise RuntimeError("Can't re-use this CM")
>          self.cmA = cmA()
>          self.cmA.__enter__()
>          try:
>            self.cmB = cmB()
>            self.cmB.__enter__()
>          except:
>            if self.cmA.__exit__(*sys.exc_info()):
>              # Suppress the exception, but don't run
>              # the body of the with statement either
>              raise SkipStatement
>            raise
>
>        def __exit__(self, *args):
>          suppress = False
>          try:
>            if self.cmB is not None:
>              suppress = self.cmB.__exit__(*args)
>          except:
>            suppress = self.cmA.__exit__(*sys.exc_info()):
>            if not suppress:
>              # Exception has changed, so reraise explicitly
>              raise
>          else:
>            if suppress:
>              # cmB already suppressed the exception,
>              # so don't pass it to cmA
>              suppress = self.cmA.__exit__(None, None, None):
>            else:
>              suppress = self.cmA.__exit__(*args):
>          return suppress
>
>     There is currently a tentative suggestion [3] to add import-style
>     syntax to
>     the ``with`` statement to allow multiple context managers to be
>     included in
>     a single ``with`` statement without needing to use
>     ``contextlib.nested``. In
>     that case the compiler has the option of simply emitting multiple
>     ``with``
>     statements at the AST level, thus allowing the semantics of actual
>     nested
>     ``with`` statements to be reproduced accurately. However, such a
>     change
>     would highlight rather than alleviate the problem the current PEP
>     aims to
>     address: it would not be possible to use
>     ``contextlib.contextmanager`` to
>     reliably factor out such ``with`` statements, as they would
>     exhibit exactly
>     the same semantic differences as are seen with the ``combined()``
>     context
>     manager in the above example.
>
>
>     Performance Impact
>     ==================
>
>     Implementing the new semantics makes it necessary to store the
>     references
>     to the ``__enter__`` and ``__exit__`` methods in temporary
>     variables instead
>     of on the stack. This results in a slight regression in ``with``
>     statement
>     speed relative to Python 2.6/3.1. However, implementing a custom
>     ``SETUP_WITH`` opcode would negate any differences between the two
>     approaches (as well as dramatically improving speed by eliminating
>     more
>     than a dozen unnecessary trips around the eval loop).
>
>
>     Reference Implementation
>     ========================
>
>     Patch attached to Issue 5251 [1]. That patch uses only existing
>     opcodes
>     (i.e. no ``SETUP_WITH``).
>
>
>     Acknowledgements
>     ================
>
>     James William Pye both raised the issue and suggested the basic
>     outline of
>     the solution described in this PEP.
>
>
>     References
>     ==========
>
>     .. [1] Issue 5251: contextlib.nested inconsistent with nested with
>     statements
>       (http://bugs.python.org/issue5251)
>
>     .. [2] PEP 343: The "with" Statement
>       (http://www.python.org/dev/peps/pep-0343/)
>
>     .. [3] Import-style syntax to reduce indentation of nested with
>     statements
>      
>     (http://mail.python.org/pipermail/python-ideas/2009-March/003188.html)
>
>
>     Copyright
>     =========
>
>     This document has been placed in the public domain.
>
>     ..
>       Local Variables:
>       mode: indented-text
>       indent-tabs-mode: nil
>       sentence-end-double-space: t
>       fill-column: 70
>       End:
>
>     _______________________________________________
>     Python-Dev mailing list
>     Python-Dev at python.org <mailto:Python-Dev at python.org>
>     http://mail.python.org/mailman/listinfo/python-dev
>     Unsubscribe:
>     http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From steven.bethard at gmail.com  Sun Mar 15 19:09:57 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 15 Mar 2009 11:09:57 -0700
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
In-Reply-To: <49BD3FD7.3080309@voidspace.org.uk>
References: <49BCFAEE.2070603@gmail.com>
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk>
Message-ID: <d11dcfba0903151109q1a16408bl7e2b21c5888e04f0@mail.gmail.com>

On Sun, Mar 15, 2009 at 10:50 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> Brett Cannon wrote:
>>
>> Without knowing what StatementSkipped is (just some singleton? If so why
>> not just used SkipStatement instance that was raised?) and wondering if we
>> are just going to continue to adding control flow exceptions that directly
>> inherit from BaseException or some ControlFlowException base class, the
>> basic idea seems fine by me.
>>
>
> Note that using exceptions for control flow can ?be bad for other
> implementations of Python. For example exceptions on the .NET framework are
> very expensive. (Although there are workarounds such as not really raising
> the exception - but they're ugly).
>
> Isn't it better practise for exceptions to be used for exceptional
> circumstances rather than for control flow?

If my understanding is correct, the primary use case for this is when
an exception is raised by an __enter__() method and caught by an
enclosing __exit__() method. So at least in that case, you've already
incurred the cost of an exception.

It might be nice to see an example of this being used with only a
single context manager. Is that possible?

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From glyph at divmod.com  Sun Mar 15 19:37:52 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Sun, 15 Mar 2009 18:37:52 -0000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip
	the	statement body
In-Reply-To: <49BCFAEE.2070603@gmail.com>
References: <49BCFAEE.2070603@gmail.com>
Message-ID: <20090315183752.12555.868958475.divmod.xquotient.6154@weber.divmod.com>


On 12:56 pm, ncoghlan at gmail.com wrote:
>PEP 377 is a proposal to allow context manager __enter__() methods to
>skip the body of the with statement by raising a specific (new) flow
>control exception.
>
>Since there is a working reference implementation now, I thought it was
>time to open it up for broader discussion.

Why not allow a context manager to implement some other method, for the 
sake of argument let's say "__start__", which was invoked with a 
callable object and could choose to evaluate or not evaluate the 
statement body by simply not calling that object (or perhaps iterable, 
in the case of a generator)?

This PEP proposes that we have two ways to deal with the body of a 
'with' statement: either the body is run or not.  I have always wanted 
to have another option: run the body later.

Passing around an object representing the body of the with statement 
would allow for this use-case, as well as removing the ugly protrusion 
of yet another control-flow exception (which, as has already been noted, 
creates difficulties for some other python implementations).

From martin at v.loewis.de  Sun Mar 15 20:38:57 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Mar 2009 20:38:57 +0100
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <49BD3FD7.3080309@voidspace.org.uk>
References: <49BCFAEE.2070603@gmail.com>	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk>
Message-ID: <49BD5951.3040303@v.loewis.de>

> Note that using exceptions for control flow can  be bad for other
> implementations of Python. For example exceptions on the .NET framework
> are very expensive.

Why do you say that? What specific implementation of .NET are you
referring to? What do you mean by "very"?

> Isn't it better practise for exceptions to be used for exceptional
> circumstances rather than for control flow?

This is an ongoing debate (in Python, and outside). I'm in the camp
that says that exceptions are a control flow mechanism just like
loops, conditionals, and recursion. With exceptions, you get essentially
multiple alternative outcomes of a function call, rather than just a
single result. In principle, it would be possible to eliminate the
return statement altogether, but it is useful syntactic sugar.

Regards,
Martin

From jbaker at zyasoft.com  Sun Mar 15 20:43:18 2009
From: jbaker at zyasoft.com (Jim Baker)
Date: Sun, 15 Mar 2009 13:43:18 -0600
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
In-Reply-To: <20090315183752.12555.868958475.divmod.xquotient.6154@weber.divmod.com>
References: <49BCFAEE.2070603@gmail.com>
	<20090315183752.12555.868958475.divmod.xquotient.6154@weber.divmod.com>
Message-ID: <d03bb4010903151243l6f2e1b92mdac66de2241b304d@mail.gmail.com>

For Jython, this proposal would not present any problems. Exceptions are in
any event of lower cost than for CPython.

Given that we have now adopted Python bytecode for various scenarios where
we cannot compile to Java bytecode, it would be nice to track any changes in
the VM such as the proposed SETUP_WITH opcode. But I'm sure we'll continue
to diff ceval.c, etc. (Consider this request perhaps fodder for the language
summit?)

- Jim

On Sun, Mar 15, 2009 at 12:37 PM, <glyph at divmod.com> wrote:

>
> On 12:56 pm, ncoghlan at gmail.com wrote:
>
>> PEP 377 is a proposal to allow context manager __enter__() methods to
>> skip the body of the with statement by raising a specific (new) flow
>> control exception.
>>
>> Since there is a working reference implementation now, I thought it was
>> time to open it up for broader discussion.
>>
>
> Why not allow a context manager to implement some other method, for the
> sake of argument let's say "__start__", which was invoked with a callable
> object and could choose to evaluate or not evaluate the statement body by
> simply not calling that object (or perhaps iterable, in the case of a
> generator)?
>
> This PEP proposes that we have two ways to deal with the body of a 'with'
> statement: either the body is run or not.  I have always wanted to have
> another option: run the body later.
>
> Passing around an object representing the body of the with statement would
> allow for this use-case, as well as removing the ugly protrusion of yet
> another control-flow exception (which, as has already been noted, creates
> difficulties for some other python implementations).
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/jbaker%40zyasoft.com
>



-- 
Jim Baker
jbaker at zyasoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090315/b28ad53b/attachment.htm>

From fuzzyman at voidspace.org.uk  Sun Mar 15 21:01:21 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 15 Mar 2009 20:01:21 +0000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <49BD5951.3040303@v.loewis.de>
References: <49BCFAEE.2070603@gmail.com>	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk> <49BD5951.3040303@v.loewis.de>
Message-ID: <49BD5E91.4060804@voidspace.org.uk>

Martin v. L?wis wrote:
>> Note that using exceptions for control flow can  be bad for other
>> implementations of Python. For example exceptions on the .NET framework
>> are very expensive.
>>     
>
> Why do you say that? What specific implementation of .NET are you
> referring to? What do you mean by "very"?
>   
I'm talking about IronPython on the Microsoft .NET framework - although 
it is likely that the same is true of IronPython on Mono.

On the .NET framework the setup for exception handling is virtually free 
until an exception is raised. Once an exception is raised it takes a lot 
longer (expensive in time). This means that in IronPython exception 
handling code (try... except and try... finally blocks) are much faster 
than CPython if no exception is raised - but much more expensive if an 
exception is raised.

You can see this in a comparison of IronPython 2 and Python 2.5 running 
PyBench:

http://ironpython.codeplex.com/Wiki/View.aspx?title=IP201VsCPy25Perf

TryExcept:      26ms      888ms      -97.1%      63ms      890ms      -92.9%
TryRaiseExcept:     58234ms     1286ms     +4428.6%     58474ms     
1298ms     +4404.6%

>> Isn't it better practise for exceptions to be used for exceptional
>> circumstances rather than for control flow?
>>     
>
> This is an ongoing debate (in Python, and outside). I'm in the camp
> that says that exceptions are a control flow mechanism just like
> loops, conditionals, and recursion. With exceptions, you get essentially
> multiple alternative outcomes of a function call, rather than just a
> single result. In principle, it would be possible to eliminate the
> return statement altogether, but it is useful syntactic sugar.
>   

Using exceptions for control flow is akin to goto. Sometimes useful but 
a dubious practise. :-)

Michael


> Regards,
> Martin
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From ncoghlan at gmail.com  Sun Mar 15 21:14:23 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 16 Mar 2009 06:14:23 +1000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <49BD3FD7.3080309@voidspace.org.uk>
References: <49BCFAEE.2070603@gmail.com>
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk>
Message-ID: <49BD619F.8070109@gmail.com>

Michael Foord wrote:
> Brett Cannon wrote:
>> Without knowing what StatementSkipped is (just some singleton? If so
>> why not just used SkipStatement instance that was raised?)

It does get described in the full PEP - it is indeed just a singleton
like NotImplemented. That whole aspect of the PEP is something I'm not
entirely sure of at this stage - it may make more sense to just leave
the variables unbound. If a particular program cares to tell whether or
not the statement was skipped, that would still be easy enough to do:

  x = skipped = object()
  with cm() as x:
    do_stuff()

  if x is skipped:
    print "CM aborted with statement!"

Actually, now that I see how easy it is to do something equivalent for
yourself, I'm definitely going to drop the StatementSkipped assignment
part of the PEP in favour of just skipping over the variable assignment
part as well.

>> and
>> wondering if we are just going to continue to adding control flow
>> exceptions that directly inherit from BaseException or some
>> ControlFlowException base class, the basic idea seems fine by me.

Given how different the control flow exceptions are from each other, I
don't think it really makes sense to ever treat them as a cohesive
group. There's also the fact that StopIteration is a control flow
exception that doesn't inherit directly from BaseException.

> Note that using exceptions for control flow can  be bad for other
> implementations of Python. For example exceptions on the .NET framework
> are very expensive. (Although there are workarounds such as not really
> raising the exception - but they're ugly).

Is it that exceptions are expensive, or setting up a try/except block is
expensive? The reason the SkipStatement idea is tenable at all (even in
CPython) is that try/except is fairly cheap when no exception is raised.

(In this specific case, my initial patch does slow things down a bit,
since one side effect of the extra try/except block is to disallow a
couple of stack based optimisations that are used in the current CPython
implementation of the with statement)

> Isn't it better practise for exceptions to be used for exceptional
> circumstances rather than for control flow?

This *is* an exceptional circumstance: a typical __enter__ method will
just return or raise some other exception. I suppose you could use some
kind of dedicated thread-local state instead of an exception to indicate
that the underlying generator didn't yield, but a new control flow
exception seemed like the most straightforward option.

I'm somewhat intrigued by Glyph's idea though - if I can figure out a
way to make it practical, it does offer very some interesting
possibilities (and would, in effect, bring reusable embedded code blocks
to Python...).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From fuzzyman at voidspace.org.uk  Sun Mar 15 21:16:53 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 15 Mar 2009 20:16:53 +0000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <49BD619F.8070109@gmail.com>
References: <49BCFAEE.2070603@gmail.com>
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk> <49BD619F.8070109@gmail.com>
Message-ID: <49BD6235.9050301@voidspace.org.uk>

Nick Coghlan wrote:
>> Note that using exceptions for control flow can  be bad for other
>> implementations of Python. For example exceptions on the .NET framework
>> are very expensive. (Although there are workarounds such as not really
>> raising the exception - but they're ugly).
>>     
>
> Is it that exceptions are expensive, or setting up a try/except block is
> expensive? The reason the SkipStatement idea is tenable at all (even in
> CPython) is that try/except is fairly cheap when no exception is raised.
>   

It is the raising of the exception that is expensive.

Michael
> (In this specific case, my initial patch does slow things down a bit,
> since one side effect of the extra try/except block is to disallow a
> couple of stack based optimisations that are used in the current CPython
> implementation of the with statement)
>
>   
>> Isn't it better practise for exceptions to be used for exceptional
>> circumstances rather than for control flow?
>>     
>
> This *is* an exceptional circumstance: a typical __enter__ method will
> just return or raise some other exception. I suppose you could use some
> kind of dedicated thread-local state instead of an exception to indicate
> that the underlying generator didn't yield, but a new control flow
> exception seemed like the most straightforward option.
>
> I'm somewhat intrigued by Glyph's idea though - if I can figure out a
> way to make it practical, it does offer very some interesting
> possibilities (and would, in effect, bring reusable embedded code blocks
> to Python...).
>
> Cheers,
> Nick.
>
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From ncoghlan at gmail.com  Sun Mar 15 21:28:37 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 16 Mar 2009 06:28:37 +1000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip	the
 statement body
In-Reply-To: <20090315183752.12555.868958475.divmod.xquotient.6154@weber.divmod.com>
References: <49BCFAEE.2070603@gmail.com>
	<20090315183752.12555.868958475.divmod.xquotient.6154@weber.divmod.com>
Message-ID: <49BD64F5.104@gmail.com>

glyph at divmod.com wrote:
> 
> On 12:56 pm, ncoghlan at gmail.com wrote:
>> PEP 377 is a proposal to allow context manager __enter__() methods to
>> skip the body of the with statement by raising a specific (new) flow
>> control exception.
>>
>> Since there is a working reference implementation now, I thought it was
>> time to open it up for broader discussion.
> 
> Why not allow a context manager to implement some other method, for the
> sake of argument let's say "__start__", which was invoked with a
> callable object and could choose to evaluate or not evaluate the
> statement body by simply not calling that object (or perhaps iterable,
> in the case of a generator)?

So the with statement would in effect create a separate code object for
the statement body that still shared the scope of the containing
function, and then pass a zero-argument callable in to the new method to
allow it to execute that code?

There are some practical hurdles to that idea (specifically, creating a
callable which uses its parent's namespace rather than having its own),
but the basic concept seems sound.

Rough spec for the concept:

Implementing __enter__/__exit__ on a CM would work as per PEP 343.

Implementing __with__ instead would give the CM complete control over
whether or not to execute the block.

The implementation of contextlib.GeneratorContextManager would then
change so that instead of providing __enter__/__exit__ as it does now it
would instead provide __with__ as follows:

  def __with__(self, exec_block):
    try:
      return self.gen.next()
    except StopIteration:
      pass
    else:
      try:
        exec_block()
      except:
        exc_type, value, traceback = sys.exc_info()
        try:
          self.gen.throw(type, value, traceback)
          raise RuntimeError("generator didn't stop after throw()")
        except StopIteration, exc:
          # Suppress the exception *unless* it's the same exception that
          # was passed to throw().  This prevents a StopIteration
          # raised inside the "with" statement from being suppressed
          return exc is not value
        except:
          # only re-raise if it's *not* the exception that was
          # passed to throw(), because __exit__() must not raise
          # an exception unless __exit__() itself failed.  But throw()
          # has to raise the exception to signal propagation, so this
          # fixes the impedance mismatch between the throw() protocol
          # and the __exit__() protocol.
          if sys.exc_info()[1] is not value:
            raise
      else:
        try:
          self.gen.next()
        except StopIteration:
          return
        else:
          raise RuntimeError("generator didn't stop")

More radical in some ways that what I was suggesting, but also cleaner
and more powerful.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From martin at v.loewis.de  Sun Mar 15 21:30:54 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Mar 2009 21:30:54 +0100
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <49BD5E91.4060804@voidspace.org.uk>
References: <49BCFAEE.2070603@gmail.com>	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk> <49BD5951.3040303@v.loewis.de>
	<49BD5E91.4060804@voidspace.org.uk>
Message-ID: <49BD657E.3020108@v.loewis.de>

> I'm talking about IronPython on the Microsoft .NET framework - although
> it is likely that the same is true of IronPython on Mono.

I see. It would be interesting to find out why this is so much slower -
I cannot believe that it is inherent in the commercial .NET framework,
but rather expect that it is some issue in IronPython (*). Also, the
test case measured doesn't entirely reflect what is proposed, since it
catches the exception in the same function - for a realistic comparison,
the raise should occur in a function call (so to measure the overhead
of stack unwinding also).

Regards,
Martin

(*) My wild guess is that IronPython feels obliged to provide traceback
objects, and that this a costly operation - I just can't believe that
exceptions are themselves costly on .NET, in the Microsoft
implementation. In the specific case, it would be possible to suppress
traceback generation.

From ncoghlan at gmail.com  Sun Mar 15 21:34:09 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 16 Mar 2009 06:34:09 +1000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <49BD6235.9050301@voidspace.org.uk>
References: <49BCFAEE.2070603@gmail.com>
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk> <49BD619F.8070109@gmail.com>
	<49BD6235.9050301@voidspace.org.uk>
Message-ID: <49BD6641.1070200@gmail.com>

Michael Foord wrote:
> Nick Coghlan wrote:
>>> Note that using exceptions for control flow can  be bad for other
>>> implementations of Python. For example exceptions on the .NET framework
>>> are very expensive. (Although there are workarounds such as not really
>>> raising the exception - but they're ugly).
>>>     
>>
>> Is it that exceptions are expensive, or setting up a try/except block is
>> expensive? The reason the SkipStatement idea is tenable at all (even in
>> CPython) is that try/except is fairly cheap when no exception is raised.
>>   
> 
> It is the raising of the exception that is expensive.

Then that isn't a huge drawback in this case - the SkipStatement
exception is only used in situations which would currently probably be
handled by raising an exception anyway (e.g. the change to
contextlib.contextmanager.__enter__() in the patch is to raise
SkipStatement where it currently raises RuntimeError).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Sun Mar 15 21:40:00 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 16 Mar 2009 06:40:00 +1000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip	the
 statement body
In-Reply-To: <49BD64F5.104@gmail.com>
References: <49BCFAEE.2070603@gmail.com>
	<20090315183752.12555.868958475.divmod.xquotient.6154@weber.divmod.com>
	<49BD64F5.104@gmail.com>
Message-ID: <49BD67A0.2030906@gmail.com>

Nick Coghlan wrote:
> Rough spec for the concept:
> 
> Implementing __enter__/__exit__ on a CM would work as per PEP 343.
> 
> Implementing __with__ instead would give the CM complete control over
> whether or not to execute the block.
> 
> The implementation of contextlib.GeneratorContextManager would then
> change so that instead of providing __enter__/__exit__ as it does now it
> would instead provide __with__ as follows:

Expansion in the previous message wasn't quite right since it didn't
give the executed block access to the result of __enter__().

Mark II:

  def __with__(self, exec_block):
    try:
      enter_result = self.gen.next()
    except StopIteration:
      pass
    else:
      try:
        exec_block(enter_result)
      except:
        exc_type, value, traceback = sys.exc_info()
        try:
          self.gen.throw(type, value, traceback)
          raise RuntimeError("generator didn't stop after throw()")
        except StopIteration, exc:
          # Suppress the exception *unless* it's the same exception that
          # was passed to throw().  This prevents a StopIteration
          # raised inside the "with" statement from being suppressed
          return exc is not value
        except:
          # only re-raise if it's *not* the exception that was
          # passed to throw(), because __exit__() must not raise
          # an exception unless __exit__() itself failed.  But throw()
          # has to raise the exception to signal propagation, so this
          # fixes the impedance mismatch between the throw() protocol
          # and the __exit__() protocol.
          if sys.exc_info()[1] is not value:
            raise
      else:
        try:
          self.gen.next()
        except StopIteration:
          return
        else:
          raise RuntimeError("generator didn't stop")


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From fuzzyman at voidspace.org.uk  Sun Mar 15 21:40:08 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 15 Mar 2009 20:40:08 +0000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <49BD657E.3020108@v.loewis.de>
References: <49BCFAEE.2070603@gmail.com>	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk> <49BD5951.3040303@v.loewis.de>
	<49BD5E91.4060804@voidspace.org.uk> <49BD657E.3020108@v.loewis.de>
Message-ID: <49BD67A8.3010907@voidspace.org.uk>

Martin v. L?wis wrote:
>> I'm talking about IronPython on the Microsoft .NET framework - although
>> it is likely that the same is true of IronPython on Mono.
>>     
>
> I see. It would be interesting to find out why this is so much slower -
> I cannot believe that it is inherent in the commercial .NET framework,
> but rather expect that it is some issue in IronPython (*). Also, the
> test case measured doesn't entirely reflect what is proposed, since it
> catches the exception in the same function - for a realistic comparison,
> the raise should occur in a function call (so to measure the overhead
> of stack unwinding also).
>
> Regards,
> Martin
>
> (*) My wild guess is that IronPython feels obliged to provide traceback
> objects, and that this a costly operation - I just can't believe that
> exceptions are themselves costly on .NET, in the Microsoft
> implementation. In the specific case, it would be possible to suppress
> traceback generation.
>   
I have discussed this issue with the IronPython team several times. They 
say that it is a deliberate design decision in .NET - to minimize the 
cost of exception handling code in the absence of exceptions at the 
expense of slower performance when exceptions are raised.

Googling for ".NET exceptions performance" would seem to confirm that.

Apparently this article on the managed exception model was written by 
one of the core developers:

http://blogs.msdn.com/cbrumme/archive/2003/10/01/51524.aspx

"This is light years away from returning a -1 from your function call. 
Exceptions are inherently non-local, and if there?s an obvious and 
enduring trend for today?s architectures, it?s that you must remain 
local for good performance.

Relative to straight-line local execution, exception performance will 
keep getting worse. Sure, we might dig into our current behavior and 
speed it up a little. But the trend will relentlessly make exceptions 
perform worse.

How do I reconcile the trend to worse performance with our 
recommendation that managed code should use exceptions to communicate 
errors? By ensuring that error cases are exceedingly rare. We used to 
say that exceptions should be used for exceptional cases, but folks 
pushed back on that as tautological."

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From pje at telecommunity.com  Sun Mar 15 22:23:31 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 15 Mar 2009 17:23:31 -0400
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip	the
 statement body
In-Reply-To: <49BD64F5.104@gmail.com>
References: <49BCFAEE.2070603@gmail.com>
	<20090315183752.12555.868958475.divmod.xquotient.6154@weber.divmod.com>
	<49BD64F5.104@gmail.com>
Message-ID: <20090315212115.482AE3A40A7@sparrow.telecommunity.com>

At 06:28 AM 3/16/2009 +1000, Nick Coghlan wrote:
>There are some practical hurdles to that idea (specifically, creating a
>callable which uses its parent's namespace rather than having its own),
>but the basic concept seems sound.

Actually, that bit's pretty simple -- they're just "nonlocal" 
variables.  But the cost of creating that function, and the cost of 
having cell variables in the surrounding function is potentially 
high.  (In CPython at least, function calls are optimized in certain 
ways when the function only has "fast" locals, and no "cell" locals.)

The cost of creating the function (though not the code object) could 
be held till runtime, since the interpreter could wait until it's 
sure there's a __with__ method before doing a MAKE_CLOSURE on the code object.

Of course, at that point, what's the difference between:

     with foo() as bar:
         baz

and...

     @foo
     def bar():
        baz

except for being slightly less verbose? (due to missing nonlocal 
statements, etc.)


From greg.ewing at canterbury.ac.nz  Sun Mar 15 23:25:04 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 16 Mar 2009 10:25:04 +1200
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49BB1FFF.4080105@gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org> <49B85D42.9080406@gmail.com>
	<loom.20090312T010957-202@post.gmane.org>
	<200903122254.21316.steve@pearwood.info>
	<loom.20090312T122030-705@post.gmane.org> <49B91FAB.2010101@gmail.com>
	<49B95F43.3000708@v.loewis.de>
	<873EFBC3-E729-4C02-B941-38D0C5314035@zope.com>
	<49BB1FFF.4080105@gmail.com>
Message-ID: <49BD8040.7010903@canterbury.ac.nz>

Nick Coghlan wrote:

> It actually wouldn't be a bad place to put a "create a temporary file
> and rename it to <name> when closing it" helper class.

I'm not sure it would be a good idea to make that
behaviour automatic on closing. If anything goes
wrong while writing the file, you *don't* want the
rename to happen, otherwise it defeats the purpose.

It would be okay to have an explicit close_and_rename()
method, although there wouldn't be much gained over
just calling os.rename() afterwards.

What might make more sense is a context manager,
e.g.

   with renaming_file("blarg.txt", "w") as f:
     ...

-- 
Greg

From aahz at pythoncraft.com  Sun Mar 15 23:30:16 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 15 Mar 2009 15:30:16 -0700
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip
	the	statement body
In-Reply-To: <49BD3FD7.3080309@voidspace.org.uk>
References: <49BCFAEE.2070603@gmail.com>
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk>
Message-ID: <20090315223016.GA27106@panix.com>

On Sun, Mar 15, 2009, Michael Foord wrote:
>
> Note that using exceptions for control flow can  be bad for other  
> implementations of Python. For example exceptions on the .NET framework  
> are very expensive. (Although there are workarounds such as not really  
> raising the exception - but they're ugly).
>
> Isn't it better practise for exceptions to be used for exceptional  
> circumstances rather than for control flow?

It seems to me that we as a development community already made a decision
when we switched to StopIteration as the primary mechanism for halting
``for`` loops.  (Not that it was really a new decision because parts of
the Python community have always advocated using exceptions for control
flow, but the ``for`` loop enshrines it.)  I doubt that using exceptions
for control flow in ``with`` blocks will cause anywhere near so much a
performance degradation.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Adopt A Process -- stop killing all your children!

From fuzzyman at voidspace.org.uk  Sun Mar 15 23:37:11 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 15 Mar 2009 22:37:11 +0000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip	the
 statement body
In-Reply-To: <20090315223016.GA27106@panix.com>
References: <49BCFAEE.2070603@gmail.com>	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>	<49BD3FD7.3080309@voidspace.org.uk>
	<20090315223016.GA27106@panix.com>
Message-ID: <49BD8317.3090704@voidspace.org.uk>

Aahz wrote:
> On Sun, Mar 15, 2009, Michael Foord wrote:
>   
>> Note that using exceptions for control flow can  be bad for other  
>> implementations of Python. For example exceptions on the .NET framework  
>> are very expensive. (Although there are workarounds such as not really  
>> raising the exception - but they're ugly).
>>
>> Isn't it better practise for exceptions to be used for exceptional  
>> circumstances rather than for control flow?
>>     
>
> It seems to me that we as a development community already made a decision
> when we switched to StopIteration as the primary mechanism for halting
> ``for`` loops.  (Not that it was really a new decision because parts of
> the Python community have always advocated using exceptions for control
> flow, but the ``for`` loop enshrines it.)  I doubt that using exceptions
> for control flow in ``with`` blocks will cause anywhere near so much a
> performance degradation.
>   
Well, StopIteration is still an implementation detail that only 
occasionally bleeds through to actual programming. It says nothing about 
whether using exceptions for non-exceptional circumstances (control 
flow) is good practise. Personally I think it makes the intent of code 
less easy to understand - in effect the exceptions *are* being used as a 
goto.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From mikko+python at redinnovation.com  Sun Mar 15 23:50:25 2009
From: mikko+python at redinnovation.com (Mikko Ohtamaa)
Date: Mon, 16 Mar 2009 00:50:25 +0200
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49BA4175.1070604@gmail.com>
References: <gp6hhq$hss$1@ger.gmane.org>
	<200903122254.21316.steve@pearwood.info>
	<loom.20090312T122030-705@post.gmane.org> <49B91FAB.2010101@gmail.com>
	<49B95F43.3000708@v.loewis.de> <49B980D4.7080100@gmail.com>
	<49B9847C.1030606@v.loewis.de> <49B9866B.2000901@gmail.com>
	<49B98B20.8070000@v.loewis.de> <49BA4175.1070604@gmail.com>
Message-ID: <7b5b293c0903151550u597db34fm9c1de990bbf28e3f@mail.gmail.com>

>
>
> > Ok. In that use case, however, it is completely irrelevant whether the
> > tempfile module calls fsync. After it has generated the non-conflicting
> > filename, it's done.
>
> I agree, but my comment was that it would be nice if better fsync
> support (if added) could be done in such a way that it helped not only
> file objects, but also *file-like* objects (such as the wrappers in the
> tempfile module) without making the file-like API any fatter.


fsync() might not be "the answer".

I found this blog post very entertaining to read:
http://www.advogato.org/person/mjg59/diary.html?start=195

"So, on the one hand, we're trying to use things like relatime to batch data
to reduce the amount of time a disk has to be spun up. And on the other
hand, we're moving to filesystems that require us to generate *more* io in
order to guarantee that our data hits disk, which is a guarantee we often
don't want anyway! Users will be fine with losing their most recent changes
to preferences if a machine crashes. They will not be fine with losing the
entirity of their preferences. Arguing that applications need to use *fsync*()
and are otherwise broken is ignoring the important difference between these
use cases. It's no longer going to be possible to spin down a disk when any
software is running at all, since otherwise it's probably going to write
something and then have to *fsync* it out of sheer paranoia that something
bad will happen. And then probably *fsync* the directory as well, because
what if someone writes an even more pathological filesystem. And the disks
sit there spinning gently and chitter away as they write tiny files[4] and
never spin down and the polar bears all drown in the bitter tears of
application developers who are forced to drink so much to forget that they
all die of acute liver failure by the age of 35 and where are we then oh yes
we're fucked."

-M
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090316/a6bfe87a/attachment.htm>

From aahz at pythoncraft.com  Sun Mar 15 23:55:01 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 15 Mar 2009 15:55:01 -0700
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip
	the	statement body
In-Reply-To: <49BD8317.3090704@voidspace.org.uk>
References: <49BCFAEE.2070603@gmail.com>
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk>
	<20090315223016.GA27106@panix.com>
	<49BD8317.3090704@voidspace.org.uk>
Message-ID: <20090315225501.GA19638@panix.com>

On Sun, Mar 15, 2009, Michael Foord wrote:
> Aahz wrote:
>> On Sun, Mar 15, 2009, Michael Foord wrote:
>>>   
>>> Note that using exceptions for control flow can  be bad for other   
>>> implementations of Python. For example exceptions on the .NET 
>>> framework  are very expensive. (Although there are workarounds such 
>>> as not really  raising the exception - but they're ugly).
>>>
>>> Isn't it better practise for exceptions to be used for exceptional   
>>> circumstances rather than for control flow?
>>
>> It seems to me that we as a development community already made a decision
>> when we switched to StopIteration as the primary mechanism for halting
>> ``for`` loops.  (Not that it was really a new decision because parts of
>> the Python community have always advocated using exceptions for control
>> flow, but the ``for`` loop enshrines it.)  I doubt that using exceptions
>> for control flow in ``with`` blocks will cause anywhere near so much a
>> performance degradation.
>   
> Well, StopIteration is still an implementation detail that only
> occasionally bleeds through to actual programming. It says nothing
> about whether using exceptions for non-exceptional circumstances
> (control flow) is good practise. Personally I think it makes the
> intent of code less easy to understand - in effect the exceptions
> *are* being used as a goto.

Let me know how you'd rewrite this more clearly without a control-flow
exception:

        try:
            for field in curr_fields:
                for item in record[field]:
                    item = item.lower()
                    for filter in excludes:
                        if match(item, filter):
                            raise Excluded
        except Excluded: 
            continue 

This is pretty much the canonical example showing why control-flow
exceptions are a Good Thing.  They're a *structured* goto.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Adopt A Process -- stop killing all your children!

From fuzzyman at voidspace.org.uk  Mon Mar 16 00:00:32 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 15 Mar 2009 23:00:32 +0000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip	the
 statement body
In-Reply-To: <20090315225501.GA19638@panix.com>
References: <49BCFAEE.2070603@gmail.com>	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>	<49BD3FD7.3080309@voidspace.org.uk>	<20090315223016.GA27106@panix.com>	<49BD8317.3090704@voidspace.org.uk>
	<20090315225501.GA19638@panix.com>
Message-ID: <49BD8890.6070404@voidspace.org.uk>

Aahz wrote:
> On Sun, Mar 15, 2009, Michael Foord wrote:
>   
>> Aahz wrote:
>>     
>>> On Sun, Mar 15, 2009, Michael Foord wrote:
>>>       
>>>>   
>>>> Note that using exceptions for control flow can  be bad for other   
>>>> implementations of Python. For example exceptions on the .NET 
>>>> framework  are very expensive. (Although there are workarounds such 
>>>> as not really  raising the exception - but they're ugly).
>>>>
>>>> Isn't it better practise for exceptions to be used for exceptional   
>>>> circumstances rather than for control flow?
>>>>         
>>> It seems to me that we as a development community already made a decision
>>> when we switched to StopIteration as the primary mechanism for halting
>>> ``for`` loops.  (Not that it was really a new decision because parts of
>>> the Python community have always advocated using exceptions for control
>>> flow, but the ``for`` loop enshrines it.)  I doubt that using exceptions
>>> for control flow in ``with`` blocks will cause anywhere near so much a
>>> performance degradation.
>>>       
>>   
>> Well, StopIteration is still an implementation detail that only
>> occasionally bleeds through to actual programming. It says nothing
>> about whether using exceptions for non-exceptional circumstances
>> (control flow) is good practise. Personally I think it makes the
>> intent of code less easy to understand - in effect the exceptions
>> *are* being used as a goto.
>>     
>
> Let me know how you'd rewrite this more clearly without a control-flow
> exception:
>
>         try:
>             for field in curr_fields:
>                 for item in record[field]:
>                     item = item.lower()
>                     for filter in excludes:
>                         if match(item, filter):
>                             raise Excluded
>         except Excluded: 
>             continue 
>
> This is pretty much the canonical example showing why control-flow
> exceptions are a Good Thing.  They're a *structured* goto.
>   

You didn't include all the code - so impossible to match the exact 
semantics. Breaking out of multiple loops with a return is a cleaner way 
to handle it IMHO.


def find_excludes():
    for field in curr_fields:
        for item in record[field]:
            item = item.lower()
            for filter in excludes:
                if match(item, filter):
                    return
                   
while something:
    find_excludes()

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From tjreedy at udel.edu  Mon Mar 16 00:04:11 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 15 Mar 2009 19:04:11 -0400
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
In-Reply-To: <20090315223016.GA27106@panix.com>
References: <49BCFAEE.2070603@gmail.com>	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>	<49BD3FD7.3080309@voidspace.org.uk>
	<20090315223016.GA27106@panix.com>
Message-ID: <gpk1h7$sui$1@ger.gmane.org>

Aahz wrote:
> On Sun, Mar 15, 2009, Michael Foord wrote:
>
> It seems to me that we as a development community already made a decision
> when we switched to StopIteration as the primary mechanism for halting
> ``for`` loops. 

If was previously IndexError that stopped for loops, so that was not new 
;-).



From mithrandi at mithrandi.net  Mon Mar 16 00:05:48 2009
From: mithrandi at mithrandi.net (Tristan Seligmann)
Date: Mon, 16 Mar 2009 01:05:48 +0200
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
In-Reply-To: <49BD8890.6070404@voidspace.org.uk>
References: <49BCFAEE.2070603@gmail.com>
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk> <20090315223016.GA27106@panix.com>
	<49BD8317.3090704@voidspace.org.uk> <20090315225501.GA19638@panix.com>
	<49BD8890.6070404@voidspace.org.uk>
Message-ID: <f5eea9170903151605s482f860cj2d79fd619d4379b8@mail.gmail.com>

On Mon, Mar 16, 2009 at 1:00 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> You didn't include all the code - so impossible to match the exact
> semantics. Breaking out of multiple loops with a return is a cleaner way to
> handle it IMHO.

I don't really see why this is cleaner; they're both just structured
gotos. Heck, the code even looks practically the same.
-- 
mithrandi, i Ainil en-Balandor, a faer Ambar

From greg.ewing at canterbury.ac.nz  Mon Mar 16 00:40:16 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 16 Mar 2009 11:40:16 +1200
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip	the
 statement body
In-Reply-To: <20090315225501.GA19638@panix.com>
References: <49BCFAEE.2070603@gmail.com>
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk> <20090315223016.GA27106@panix.com>
	<49BD8317.3090704@voidspace.org.uk> <20090315225501.GA19638@panix.com>
Message-ID: <49BD91E0.30707@canterbury.ac.nz>

Aahz wrote:

> This is pretty much the canonical example showing why control-flow
> exceptions are a Good Thing.  They're a *structured* goto.

I'm wondering whether what we really want is something
that actually *is* a structured goto. Or something like
a very light-weight exception that doesn't carry all the
expensive baggage of tracebacks, isinstance() tests for
matching, etc.

Ruby seems to have something like this. It has a
try/rescue/raise mechanism that works like Python's
try/except/raise, but it also has try/catch/throw as
a less expensive alternative for flow control.

A Python version might look like

   Foo = object() # a token to throw

   try:
     do_something()
   catch Foo:
     someone_threw_a_foo()

   def do_something():
     throw Foo

Nothing is instantiated -- the token value itself is
thrown -- and the catch clauses compare it by identity
with candidate values.

There is also no traceback carried by the thrown token.

(Although if something is thrown but not caught, an
exception should be raised at the point of the throw
with an appropriate traceback -- implementation of that
is left as an exercise for the reader.)

-- 
Greg

From cmjohnson.mailinglist at gmail.com  Mon Mar 16 00:38:34 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Sun, 15 Mar 2009 13:38:34 -1000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
In-Reply-To: <20090315212115.482AE3A40A7@sparrow.telecommunity.com>
References: <49BCFAEE.2070603@gmail.com>
	<20090315183752.12555.868958475.divmod.xquotient.6154@weber.divmod.com>
	<49BD64F5.104@gmail.com>
	<20090315212115.482AE3A40A7@sparrow.telecommunity.com>
Message-ID: <3bdda690903151638l312c8241xeb48f1d35d73ca53@mail.gmail.com>

P.J. Eby wrote:

> Of course, at that point, what's the difference between:
>
> ? ?with foo() as bar:
> ? ? ? ?baz
>
> and...
>
> ? ?@foo
> ? ?def bar():
> ? ? ? baz
>
> except for being slightly less verbose? (due to missing nonlocal statements,
> etc.)

That's not quite direct translation. Closer would be:

    @foo()
    def _(bar):
       baz
    del _

since the "bar" of the with statement is injected into the namespace
of the block. (Obviously, the foo would have be implemented
differently in the two cases in order to have it operate the same
way.)

I have thought about suggesting adding control flow to the with
statement before, since I think if done properly, it might be able to
negate the desire people have for multi-line lambdas/Ruby-style
blocks, but I do wonder if it would be too confusing to have "with"
sometimes mean "abstracting error handling code" and other times mean
"abstracted control flow." But now it looks like there's no way around
using the with-statement for control flow, since sometimes the block
needs to be skipped anyway. So, I'm +.5 on the idea.

-- Carl

From martin at v.loewis.de  Mon Mar 16 01:37:53 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Mar 2009 01:37:53 +0100
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip	the
 statement body
In-Reply-To: <49BD8317.3090704@voidspace.org.uk>
References: <49BCFAEE.2070603@gmail.com>	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>	<49BD3FD7.3080309@voidspace.org.uk>	<20090315223016.GA27106@panix.com>
	<49BD8317.3090704@voidspace.org.uk>
Message-ID: <49BD9F61.40003@v.loewis.de>

> Well, StopIteration is still an implementation detail that only
> occasionally bleeds through to actual programming. It says nothing about
> whether using exceptions for non-exceptional circumstances (control
> flow) is good practise. Personally I think it makes the intent of code
> less easy to understand - in effect the exceptions *are* being used as a
> goto.

The same can be said about if statements and while loops - they are also
being used "as gotos". The bad thing about the goto statement is that it
allows arbitrary, unstructured control flow. This is unlike if
statements, while loops, and - yes - exceptions. They all provide for
structured control flow. raise, in particular, is no more evil than
break, continue, return, or yield.

Regards,
Martin

From ncoghlan at gmail.com  Mon Mar 16 12:15:53 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 16 Mar 2009 21:15:53 +1000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip	the
 statement body
In-Reply-To: <20090315212115.482AE3A40A7@sparrow.telecommunity.com>
References: <49BCFAEE.2070603@gmail.com>
	<20090315183752.12555.868958475.divmod.xquotient.6154@weber.divmod.com>
	<49BD64F5.104@gmail.com>
	<20090315212115.482AE3A40A7@sparrow.telecommunity.com>
Message-ID: <49BE34E9.2050801@gmail.com>

P.J. Eby wrote:
> At 06:28 AM 3/16/2009 +1000, Nick Coghlan wrote:
>> There are some practical hurdles to that idea (specifically, creating a
>> callable which uses its parent's namespace rather than having its own),
>> but the basic concept seems sound.
> 
> Actually, that bit's pretty simple -- they're just "nonlocal"
> variables.  But the cost of creating that function, and the cost of
> having cell variables in the surrounding function is potentially high. 
> (In CPython at least, function calls are optimized in certain ways when
> the function only has "fast" locals, and no "cell" locals.)

Some additional complexities occurred to me today - they go by the names
"return", "break" and "continue". With the current design those are
perfectly well defined inside a with statement, but they pose a pretty
serious hurdle for a practical implementation of glyph's idea.

So I'm going to stick with the more modest approach of a new control
flow exception for PEP 377.

I still find the callable block idea somewhat intriguing, but if it ever
happens it should be a new construct rather than overloading the meaning
of the with statement.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Mar 16 12:43:11 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 16 Mar 2009 21:43:11 +1000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip	the
 statement body
In-Reply-To: <49BD8317.3090704@voidspace.org.uk>
References: <49BCFAEE.2070603@gmail.com>	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>	<49BD3FD7.3080309@voidspace.org.uk>	<20090315223016.GA27106@panix.com>
	<49BD8317.3090704@voidspace.org.uk>
Message-ID: <49BE3B4F.3060300@gmail.com>

Michael Foord wrote:
> Well, StopIteration is still an implementation detail that only
> occasionally bleeds through to actual programming. It says nothing about
> whether using exceptions for non-exceptional circumstances (control
> flow) is good practise. Personally I think it makes the intent of code
> less easy to understand - in effect the exceptions *are* being used as a
> goto.

Note that raising SkipStatement manually is likely to be even rarer than
raising StopIteration. Catching it should almost never happen other than
implicitly inside a with statement (that's the reason I made it a peer
of SystemExit and GeneratorExit rather than a peer of StopIteration).

It is primarily proposed as a way for contextlib.contextmanager to tell
the interpreter that the underlying generator didn't yield, so the body
of the with statement should be skipped completely. It just so happens
that manually implemented context managers will also be free to use it
if they need to for some reason.

An alternative approach worth considering may be to use NotImplemented
as a model instead of StopIteration. With that approach, instead of
having SkipStatement be an exception, have it be a singleton that can be
returned from __enter__ to indicate that the with statement body would
be skipped.

That has a big advantage over using an exception when it comes to
execution speed. The statement semantics in that case would become:

    mgr = (EXPR)
    exit = mgr.__exit__  # Not calling it yet
    value = mgr.__enter__()
    if value is not SkipStatement:
        exc = True
        try:
            try:
                VAR = value  # Only if "as VAR" is present
                BLOCK
            except:
                # The exceptional case is handled here
                exc = False
                if not exit(*sys.exc_info()):
                    raise
                # The exception is swallowed if exit() returns true
        finally:
            # The normal and non-local-goto cases are handled here
            if exc:
                exit(None, None, None)

(keeping in mind that I already plan to change PEP 377 to drop the idea
of assigning anything to VAR when the statement body is skipped)

The major drawback of that approach is that it becomes a little trickier
to write a context manager like nested() correctly - it would need to
check all of the __enter__() return values and start unwinding the
context manager stack if it encountered SkipStatement. The fix isn't
particularly complicated*, but it does contrast with the fact that
having SkipStatement as an exception means that the current
implementation of nested() will "just work" with the new semantics.

Cheers,
Nick.

* For reference, to support a "SkipStatement as return value" approach
the main loop in nested() would have to change from this:

    for mgr in managers:
        exit = mgr.__exit__
        enter = mgr.__enter__
        vars.append(enter())
        exits.append(exit)
    yield vars

To this:

    for mgr in managers:
        exit = mgr.__exit__
        enter = mgr.__enter__
        var = enter()
        if var is SkipStatement:
            break
        vars.append(var)
        exits.append(exit)
    else:
        yield vars

As mentioned above, if SkipStatement is an exception then nested() works
correctly without any changes.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Mar 16 12:46:35 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 16 Mar 2009 21:46:35 +1000
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49BD8040.7010903@canterbury.ac.nz>
References: <gp6hhq$hss$1@ger.gmane.org>
	<49B85D42.9080406@gmail.com>	<loom.20090312T010957-202@post.gmane.org>	<200903122254.21316.steve@pearwood.info>	<loom.20090312T122030-705@post.gmane.org>
	<49B91FAB.2010101@gmail.com>	<49B95F43.3000708@v.loewis.de>	<873EFBC3-E729-4C02-B941-38D0C5314035@zope.com>	<49BB1FFF.4080105@gmail.com>
	<49BD8040.7010903@canterbury.ac.nz>
Message-ID: <49BE3C1B.3070206@gmail.com>

Greg Ewing wrote:
> What might make more sense is a context manager,
> e.g.
> 
>   with renaming_file("blarg.txt", "w") as f:
>     ...

As you were describing the problems with "rename on close", I actually
immediately thought of the oft-repeated db transaction commit/rollback
example from PEP 343 :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From guido at python.org  Mon Mar 16 18:01:09 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Mar 2009 10:01:09 -0700
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
In-Reply-To: <49BE3B4F.3060300@gmail.com>
References: <49BCFAEE.2070603@gmail.com>
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk> <20090315223016.GA27106@panix.com>
	<49BD8317.3090704@voidspace.org.uk> <49BE3B4F.3060300@gmail.com>
Message-ID: <ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>

I have no right to speak because I haven't read through all the
details of the proposal, but reading this I am very sad that we have
to introduce a whole new exception (and one with special status as
well) in order to fix such a niggly corner case of the context manager
protocol.

Since IIUC the original context manager design was intended to have
exactly one yield in the body of the context manager -- can't we just
declare fewer (or more) yields an error and rase an appropriate
TypeError or something?

--Guido

On Mon, Mar 16, 2009 at 4:43 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Michael Foord wrote:
>> Well, StopIteration is still an implementation detail that only
>> occasionally bleeds through to actual programming. It says nothing about
>> whether using exceptions for non-exceptional circumstances (control
>> flow) is good practise. Personally I think it makes the intent of code
>> less easy to understand - in effect the exceptions *are* being used as a
>> goto.
>
> Note that raising SkipStatement manually is likely to be even rarer than
> raising StopIteration. Catching it should almost never happen other than
> implicitly inside a with statement (that's the reason I made it a peer
> of SystemExit and GeneratorExit rather than a peer of StopIteration).
>
> It is primarily proposed as a way for contextlib.contextmanager to tell
> the interpreter that the underlying generator didn't yield, so the body
> of the with statement should be skipped completely. It just so happens
> that manually implemented context managers will also be free to use it
> if they need to for some reason.
>
> An alternative approach worth considering may be to use NotImplemented
> as a model instead of StopIteration. With that approach, instead of
> having SkipStatement be an exception, have it be a singleton that can be
> returned from __enter__ to indicate that the with statement body would
> be skipped.
>
> That has a big advantage over using an exception when it comes to
> execution speed. The statement semantics in that case would become:
>
> ? ?mgr = (EXPR)
> ? ?exit = mgr.__exit__ ?# Not calling it yet
> ? ?value = mgr.__enter__()
> ? ?if value is not SkipStatement:
> ? ? ? ?exc = True
> ? ? ? ?try:
> ? ? ? ? ? ?try:
> ? ? ? ? ? ? ? ?VAR = value ?# Only if "as VAR" is present
> ? ? ? ? ? ? ? ?BLOCK
> ? ? ? ? ? ?except:
> ? ? ? ? ? ? ? ?# The exceptional case is handled here
> ? ? ? ? ? ? ? ?exc = False
> ? ? ? ? ? ? ? ?if not exit(*sys.exc_info()):
> ? ? ? ? ? ? ? ? ? ?raise
> ? ? ? ? ? ? ? ?# The exception is swallowed if exit() returns true
> ? ? ? ?finally:
> ? ? ? ? ? ?# The normal and non-local-goto cases are handled here
> ? ? ? ? ? ?if exc:
> ? ? ? ? ? ? ? ?exit(None, None, None)
>
> (keeping in mind that I already plan to change PEP 377 to drop the idea
> of assigning anything to VAR when the statement body is skipped)
>
> The major drawback of that approach is that it becomes a little trickier
> to write a context manager like nested() correctly - it would need to
> check all of the __enter__() return values and start unwinding the
> context manager stack if it encountered SkipStatement. The fix isn't
> particularly complicated*, but it does contrast with the fact that
> having SkipStatement as an exception means that the current
> implementation of nested() will "just work" with the new semantics.
>
> Cheers,
> Nick.
>
> * For reference, to support a "SkipStatement as return value" approach
> the main loop in nested() would have to change from this:
>
> ? ?for mgr in managers:
> ? ? ? ?exit = mgr.__exit__
> ? ? ? ?enter = mgr.__enter__
> ? ? ? ?vars.append(enter())
> ? ? ? ?exits.append(exit)
> ? ?yield vars
>
> To this:
>
> ? ?for mgr in managers:
> ? ? ? ?exit = mgr.__exit__
> ? ? ? ?enter = mgr.__enter__
> ? ? ? ?var = enter()
> ? ? ? ?if var is SkipStatement:
> ? ? ? ? ? ?break
> ? ? ? ?vars.append(var)
> ? ? ? ?exits.append(exit)
> ? ?else:
> ? ? ? ?yield vars
>
> As mentioned above, if SkipStatement is an exception then nested() works
> correctly without any changes.
>
> --
> Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia
> ---------------------------------------------------------------
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Mar 16 19:06:01 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Mar 2009 11:06:01 -0700
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
In-Reply-To: <ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>
References: <49BCFAEE.2070603@gmail.com>
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk> <20090315223016.GA27106@panix.com>
	<49BD8317.3090704@voidspace.org.uk> <49BE3B4F.3060300@gmail.com>
	<ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>
Message-ID: <ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>

Moreover, since the main use case seems to be fixing a corner case of
the nested() context manager, perhaps the effort towards changing the
language would be better directed towards supporting "with a, b:" as a
shorthand for "with a: with b:" .

On Mon, Mar 16, 2009 at 10:01 AM, Guido van Rossum <guido at python.org> wrote:
> I have no right to speak because I haven't read through all the
> details of the proposal, but reading this I am very sad that we have
> to introduce a whole new exception (and one with special status as
> well) in order to fix such a niggly corner case of the context manager
> protocol.
>
> Since IIUC the original context manager design was intended to have
> exactly one yield in the body of the context manager -- can't we just
> declare fewer (or more) yields an error and rase an appropriate
> TypeError or something?
>
> --Guido
>
> On Mon, Mar 16, 2009 at 4:43 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Michael Foord wrote:
>>> Well, StopIteration is still an implementation detail that only
>>> occasionally bleeds through to actual programming. It says nothing about
>>> whether using exceptions for non-exceptional circumstances (control
>>> flow) is good practise. Personally I think it makes the intent of code
>>> less easy to understand - in effect the exceptions *are* being used as a
>>> goto.
>>
>> Note that raising SkipStatement manually is likely to be even rarer than
>> raising StopIteration. Catching it should almost never happen other than
>> implicitly inside a with statement (that's the reason I made it a peer
>> of SystemExit and GeneratorExit rather than a peer of StopIteration).
>>
>> It is primarily proposed as a way for contextlib.contextmanager to tell
>> the interpreter that the underlying generator didn't yield, so the body
>> of the with statement should be skipped completely. It just so happens
>> that manually implemented context managers will also be free to use it
>> if they need to for some reason.
>>
>> An alternative approach worth considering may be to use NotImplemented
>> as a model instead of StopIteration. With that approach, instead of
>> having SkipStatement be an exception, have it be a singleton that can be
>> returned from __enter__ to indicate that the with statement body would
>> be skipped.
>>
>> That has a big advantage over using an exception when it comes to
>> execution speed. The statement semantics in that case would become:
>>
>> ? ?mgr = (EXPR)
>> ? ?exit = mgr.__exit__ ?# Not calling it yet
>> ? ?value = mgr.__enter__()
>> ? ?if value is not SkipStatement:
>> ? ? ? ?exc = True
>> ? ? ? ?try:
>> ? ? ? ? ? ?try:
>> ? ? ? ? ? ? ? ?VAR = value ?# Only if "as VAR" is present
>> ? ? ? ? ? ? ? ?BLOCK
>> ? ? ? ? ? ?except:
>> ? ? ? ? ? ? ? ?# The exceptional case is handled here
>> ? ? ? ? ? ? ? ?exc = False
>> ? ? ? ? ? ? ? ?if not exit(*sys.exc_info()):
>> ? ? ? ? ? ? ? ? ? ?raise
>> ? ? ? ? ? ? ? ?# The exception is swallowed if exit() returns true
>> ? ? ? ?finally:
>> ? ? ? ? ? ?# The normal and non-local-goto cases are handled here
>> ? ? ? ? ? ?if exc:
>> ? ? ? ? ? ? ? ?exit(None, None, None)
>>
>> (keeping in mind that I already plan to change PEP 377 to drop the idea
>> of assigning anything to VAR when the statement body is skipped)
>>
>> The major drawback of that approach is that it becomes a little trickier
>> to write a context manager like nested() correctly - it would need to
>> check all of the __enter__() return values and start unwinding the
>> context manager stack if it encountered SkipStatement. The fix isn't
>> particularly complicated*, but it does contrast with the fact that
>> having SkipStatement as an exception means that the current
>> implementation of nested() will "just work" with the new semantics.
>>
>> Cheers,
>> Nick.
>>
>> * For reference, to support a "SkipStatement as return value" approach
>> the main loop in nested() would have to change from this:
>>
>> ? ?for mgr in managers:
>> ? ? ? ?exit = mgr.__exit__
>> ? ? ? ?enter = mgr.__enter__
>> ? ? ? ?vars.append(enter())
>> ? ? ? ?exits.append(exit)
>> ? ?yield vars
>>
>> To this:
>>
>> ? ?for mgr in managers:
>> ? ? ? ?exit = mgr.__exit__
>> ? ? ? ?enter = mgr.__enter__
>> ? ? ? ?var = enter()
>> ? ? ? ?if var is SkipStatement:
>> ? ? ? ? ? ?break
>> ? ? ? ?vars.append(var)
>> ? ? ? ?exits.append(exit)
>> ? ?else:
>> ? ? ? ?yield vars
>>
>> As mentioned above, if SkipStatement is an exception then nested() works
>> correctly without any changes.
>>
>> --
>> Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia
>> ---------------------------------------------------------------
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steven.bethard at gmail.com  Mon Mar 16 19:26:22 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 16 Mar 2009 11:26:22 -0700
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
In-Reply-To: <ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>
References: <49BCFAEE.2070603@gmail.com>
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk> <20090315223016.GA27106@panix.com>
	<49BD8317.3090704@voidspace.org.uk> <49BE3B4F.3060300@gmail.com>
	<ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>
	<ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>
Message-ID: <d11dcfba0903161126y6fc234f9pb387f65cdcd7b605@mail.gmail.com>

On Mon, Mar 16, 2009 at 11:06 AM, Guido van Rossum <guido at python.org> wrote:
> Moreover, since the main use case seems to be fixing a corner case of
> the nested() context manager, perhaps the effort towards changing the
> language would be better directed towards supporting "with a, b:" as a
> shorthand for "with a: with b:" .
>
> On Mon, Mar 16, 2009 at 10:01 AM, Guido van Rossum <guido at python.org> wrote:
>> I have no right to speak because I haven't read through all the
>> details of the proposal, but reading this I am very sad that we have
>> to introduce a whole new exception (and one with special status as
>> well) in order to fix such a niggly corner case of the context manager
>> protocol.
>>
>> Since IIUC the original context manager design was intended to have
>> exactly one yield in the body of the context manager -- can't we just
>> declare fewer (or more) yields an error and rase an appropriate
>> TypeError or something?

It's not really a generator specific thing. You can generate similar
problems by just defining a class with an __enter__() method that
raises an exception.

But I agree that it seems like a big change for a small corner case.
Is there anything other than contextlib.nested() which needs this? If
there is no other use case, then I'm a strong +1 for Guido's
suggestion of providing syntactic support for ``with a, b:`` instead.

BTW, I think the explanation of the problem isn't as clear as it could
be. The core problem, if I understand it right, is that
contextlib.nested() is not equivalent to a real nested with statement
because it calls the nested __enter__() methods too early. A real
nested with statement translates into something like::

    mgr1.__enter__()
    try:
        mgr2.__enter__()
        try:
            BLOCK
        except:
            ...
    except:
        if not mgr1.__exit__(*sys.exc_info()):
            raise

But contextlib.nested() calls all the __enter__() methods in its own
__enter__() so it translates into something like::

    mgr1.__enter__()
    mgr2.__enter__()
    try:
        BLOCK
    except:
        ...

The key problem here is that ``mgr2.__enter__()`` is outside of the
try block, and the context manager has no way to put it inside. So the
thing that contextlib.nested() really needs is a way to be able to
insert statements into the BLOCK part of the code. (I'm not actually
suggesting we go this route, but that seems to be what
contextlib.nested() is really after.)

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From guido at python.org  Mon Mar 16 19:33:33 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Mar 2009 11:33:33 -0700
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
In-Reply-To: <d11dcfba0903161126y6fc234f9pb387f65cdcd7b605@mail.gmail.com>
References: <49BCFAEE.2070603@gmail.com>
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk> <20090315223016.GA27106@panix.com>
	<49BD8317.3090704@voidspace.org.uk> <49BE3B4F.3060300@gmail.com>
	<ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>
	<ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>
	<d11dcfba0903161126y6fc234f9pb387f65cdcd7b605@mail.gmail.com>
Message-ID: <ca471dc20903161133g3afcae32q5500c94f6c924def@mail.gmail.com>

On Mon, Mar 16, 2009 at 11:26 AM, Steven Bethard
<steven.bethard at gmail.com> wrote:
> On Mon, Mar 16, 2009 at 11:06 AM, Guido van Rossum <guido at python.org> wrote:
>> Moreover, since the main use case seems to be fixing a corner case of
>> the nested() context manager, perhaps the effort towards changing the
>> language would be better directed towards supporting "with a, b:" as a
>> shorthand for "with a: with b:" .
>>
>> On Mon, Mar 16, 2009 at 10:01 AM, Guido van Rossum <guido at python.org> wrote:
>>> I have no right to speak because I haven't read through all the
>>> details of the proposal, but reading this I am very sad that we have
>>> to introduce a whole new exception (and one with special status as
>>> well) in order to fix such a niggly corner case of the context manager
>>> protocol.
>>>
>>> Since IIUC the original context manager design was intended to have
>>> exactly one yield in the body of the context manager -- can't we just
>>> declare fewer (or more) yields an error and rase an appropriate
>>> TypeError or something?
>
> It's not really a generator specific thing. You can generate similar
> problems by just defining a class with an __enter__() method that
> raises an exception.

Huh? According to PEP 343, if __enter__ raises an exception, that's
the end of the story. __exit__ shouldn't be called, the exception is
not modified, the flow is interrupted right there.

> But I agree that it seems like a big change for a small corner case.
> Is there anything other than contextlib.nested() which needs this? If
> there is no other use case, then I'm a strong +1 for Guido's
> suggestion of providing syntactic support for ``with a, b:`` instead.
>
> BTW, I think the explanation of the problem isn't as clear as it could
> be. The core problem, if I understand it right, is that
> contextlib.nested() is not equivalent to a real nested with statement
> because it calls the nested __enter__() methods too early. A real
> nested with statement translates into something like::
>
> ? ?mgr1.__enter__()
> ? ?try:
> ? ? ? ?mgr2.__enter__()
> ? ? ? ?try:
> ? ? ? ? ? ?BLOCK
> ? ? ? ?except:
> ? ? ? ? ? ?...
> ? ?except:
> ? ? ? ?if not mgr1.__exit__(*sys.exc_info()):
> ? ? ? ? ? ?raise
>
> But contextlib.nested() calls all the __enter__() methods in its own
> __enter__() so it translates into something like::
>
> ? ?mgr1.__enter__()
> ? ?mgr2.__enter__()
> ? ?try:
> ? ? ? ?BLOCK
> ? ?except:
> ? ? ? ?...
>
> The key problem here is that ``mgr2.__enter__()`` is outside of the
> try block, and the context manager has no way to put it inside. So the
> thing that contextlib.nested() really needs is a way to be able to
> insert statements into the BLOCK part of the code. (I'm not actually
> suggesting we go this route, but that seems to be what
> contextlib.nested() is really after.)

Yeah, it really seems pretty much limited to contextlib.nested(). I'd
be happy to sacrifice the possibility to *exactly* emulate two nested
with-statements. The cost of a new exception is huge -- everyone will
have to explain its existence, and historically "you don't need to
know about this little detail" isn't acceptable for Python docs.
Little details you don't need to know about add up.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From dialtone at gmail.com  Mon Mar 16 21:33:06 2009
From: dialtone at gmail.com (Valentino Volonghi)
Date: Mon, 16 Mar 2009 13:33:06 -0700
Subject: [Python-Dev] Ext4 data loss
In-Reply-To: <49BD8040.7010903@canterbury.ac.nz>
References: <gp6hhq$hss$1@ger.gmane.org> <49B85D42.9080406@gmail.com>
	<loom.20090312T010957-202@post.gmane.org>
	<200903122254.21316.steve@pearwood.info>
	<loom.20090312T122030-705@post.gmane.org>
	<49B91FAB.2010101@gmail.com> <49B95F43.3000708@v.loewis.de>
	<873EFBC3-E729-4C02-B941-38D0C5314035@zope.com>
	<49BB1FFF.4080105@gmail.com> <49BD8040.7010903@canterbury.ac.nz>
Message-ID: <E5030510-A082-476D-BE87-1DBF621E5027@gmail.com>


On Mar 15, 2009, at 3:25 PM, Greg Ewing wrote:

>  with renaming_file("blarg.txt", "w") as f:
>    ...


By chance during the weekend I actually wrote something like that:

from __future__ import with_statement

import os
import codecs
import shutil
import tempfile

from contextlib import contextmanager

TDIR = tempfile.mktemp(dir='/tmp/')

@contextmanager
def topen(filepath, mode='wb', bufsize=-1, encoding=None,
           inplace=False, tmpd=TDIR, sync=False):
     """
     C{topen} is a transactional version of the Python builtin C{open}
     function for file IO. It manages transactionality by using a
     temporary file and then moving it to the final position once its
     content has been written to disk.
     If the mode used to open the file doesn't modify the file, this
     function is equivalent to the built-in C{open} with automatic
     file closing behavior.

     @param filepath: The path of the file that you want to open
     @type filepath: C{str}

     @param mode: POSIX mode in which you want to open the file.
     @type mode: C{str} see documentation for the format.

     @param bufsize: Buffer size for file IO
     @type bufsize: C{int} see documentation for the meaning

     @param encoding: Encoding that should be used to read the file
     @type encoding: C{str}

     @param inplace: Indicates if the temporary file should reside
                     in the same directory of the final file.
     @type inplace: C{bool}

     @param tmpd: The temporary directory in which file IO is
                  performed. Then files are moved from here to
                  their original destination.
     @type tmpd: C{str}

     @param sync: Force topen to fsync the file before closing it
     @type sync: C{bool}
     """
     if 'r' in mode or 'a' in mode:
         fp = filepath
     else:
         if inplace:
             source_dir, _ = os.path.split(filepath)
             tmpd = source_dir

         if not os.path.exists(tmpd):
             os.makedirs(tmpd)
         _f, fp = tempfile.mkstemp(dir=tmpd)

     if encoding is not None:
         f = codecs.open(fp, mode, encoding=encoding)
     else:
         f = open(fp, mode, bufsize)

     try:
         yield f
     finally:
         if 'r' in mode:
             if "+" in mode:
                 f.flush()
                 if sync:
                     os.fsync(f.fileno())
             f.close()
             return

         f.flush()
         if sync:
             os.fsync(f.fileno())
         f.close()
         if 'w' in mode:
             shutil.move(fp, filepath)

if __name__ == "__main__":
     with topen("a_test") as f:
         f.write("hello")
     assert file("a_test", "rb").read() == 'hello'
     assert os.path.exists(TDIR)
     os.rmdir(TDIR)

     with topen("a_test", mode="rb") as f:
         assert f.read() == "hello"
     assert not os.path.exists(TDIR)
     os.remove("a_test")


--
Valentino Volonghi aka Dialtone
Now running MacOS X 10.5
Home Page: http://www.twisted.it
http://www.adroll.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090316/774d297d/attachment.pgp>

From baptiste13z at free.fr  Mon Mar 16 09:15:50 2009
From: baptiste13z at free.fr (Baptiste Carvello)
Date: Mon, 16 Mar 2009 09:15:50 +0100
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
In-Reply-To: <49BD64F5.104@gmail.com>
References: <49BCFAEE.2070603@gmail.com>	<20090315183752.12555.868958475.divmod.xquotient.6154@weber.divmod.com>
	<49BD64F5.104@gmail.com>
Message-ID: <gpl1s2$sie$1@ger.gmane.org>

Nick Coghlan a ?crit :
> 
> Implementing __with__ instead would give the CM complete control over
> whether or not to execute the block.
> 
please note, however, that this is an important change in the semantics of the 
with statement. As things are today, barring exceptional circunstances, the body 
of the with statement *will* be executed immediately. This allows to forget 
about the with statement when first reading code, as it doesn't change the 
intent of the programmer. What Glyph is proposing is more akin to Ruby code blocks.

cheers,
Baptiste


From ncoghlan at gmail.com  Mon Mar 16 22:37:37 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 17 Mar 2009 07:37:37 +1000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <ca471dc20903161133g3afcae32q5500c94f6c924def@mail.gmail.com>
References: <49BCFAEE.2070603@gmail.com>	
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>	
	<49BD3FD7.3080309@voidspace.org.uk>
	<20090315223016.GA27106@panix.com>	
	<49BD8317.3090704@voidspace.org.uk> <49BE3B4F.3060300@gmail.com>	
	<ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>	
	<ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>	
	<d11dcfba0903161126y6fc234f9pb387f65cdcd7b605@mail.gmail.com>
	<ca471dc20903161133g3afcae32q5500c94f6c924def@mail.gmail.com>
Message-ID: <49BEC6A1.3050504@gmail.com>

Guido van Rossum wrote:
> Yeah, it really seems pretty much limited to contextlib.nested(). I'd
> be happy to sacrifice the possibility to *exactly* emulate two nested
> with-statements.

Then I really haven't explained the problem well at all. One of the
premises of PEP 343 was "Got a frequently recurring block of code that
only has one variant sequence of statements somewhere in the middle?
Well, now you can factor that out by putting it in a generator,
replacing the part that varies with a yield statement and decorating the
generator with contextlib.contextmanager."

It turns out that there's a caveat that needs to go on the end of that
though: "Be very, very sure that the yield statement can *never* be
skipped or your context manager based version will raise a RuntimeError
in cases where the original code would have just skipped over the
variant section of code and resumed execution afterwards."

Nested context managers (whether through contextlib.nested or through
syntactic support) just turns out to be a common case where you *don't
necessarily know* just by looking at the code whether it can skip over
the body of the code or not.

Suppose you have 3 context managers that are regularly used together
(call them cmA(), cmB(), cmC() for now).

Writing that as:

  with cmA():
    with cmB():
      with cmC():
        do_something()

Or the tentatively proposed:

  with cmA(), cmB(), cmC():
        do_something()

is definitely OK, regardless of the details of the context managers.

However, whether or not you can bundle that up into a *new* context
manager (regardless of nesting syntax) depends on whether or not an
outer context manager can suppress an exception raised by an inner one.

  @contextmanager
  def cmABC():
    with cmA():
      with cmB():
        with cmC():
          yield

  with cmABC():
    do_something()

The above is broken if cmB().__enter__() or cmC.__enter__() can raise an
exception that cmA().__exit__() suppresses, or cmB.__enter__() raises an
exception that cmB().__exit__() suppresses. So whereas the inline
versions were clearly correct, the correctness of the second version
currently depends on details of the context managers themselves.
Changing the syntax to allow the three context managers to be written on
one line does nothing to fix that semantic discrepancy between the
original inline code and the factored out version.

PEP 377 is about changing the with statement semantics and the
@contextmanager implementation so that the semantics of the factored out
version actually matches that of the original inline code.

You can get yourself into similar trouble without nesting context
managers - all it takes is some way of skipping the variant code in a
context manager that wouldn't have raised an exception if the code was
written out inline instead of being factored out into the context manager.

Suppose for instance you wanted to use a context manager as a different
way of running tests:

  @contextmanager
  def inline_test(self, *setup_args):
    try:
      self.setup(*setup_args)
    except:
      # Setup exception occurred, trap it and log it
      return
    try:
      yield
    except:
      # Test exception occurred, trap it and log it
    finally:
      try:
        self.teardown()
      except:
        # Teardown exception occurred, trap it and log it

  with inline_test(setup1):
    test_one()
  with inline_test(setup2):
    test_two()
  with inline_test(setup3):
    test_three()

That approach isn't actually valid - a context manager is not permitted
to decide in it's __enter__() method that executing the body of the with
statement would be a bad idea.

The early return in the above makes it obvious that that CM is broken
under the current semantics, but what about the following one:

  @contextmanager
  def broken_cm(self):
    try:
      call_user_setup()
      try:
        yield
      finally:
        call_user_teardown()
    except UserCancel:
      show_message("Operation aborted by user")

That CM will raise RuntimeError if the user attempts to cancel an
operation during the execution of the "call_user_setup()" method.
Without SkipStatement or something like it, that can't be fixed.

Hell, I largely wrote PEP 377 to try to get out of having to document
these semantic problems with the with statement - if I'm having trouble
getting *python-dev* to grasp the problem, what hope do other users of
Python have?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From guido at python.org  Mon Mar 16 22:57:40 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Mar 2009 14:57:40 -0700
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
In-Reply-To: <49BEC6A1.3050504@gmail.com>
References: <49BCFAEE.2070603@gmail.com> <49BD3FD7.3080309@voidspace.org.uk>
	<20090315223016.GA27106@panix.com> <49BD8317.3090704@voidspace.org.uk>
	<49BE3B4F.3060300@gmail.com>
	<ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>
	<ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>
	<d11dcfba0903161126y6fc234f9pb387f65cdcd7b605@mail.gmail.com>
	<ca471dc20903161133g3afcae32q5500c94f6c924def@mail.gmail.com>
	<49BEC6A1.3050504@gmail.com>
Message-ID: <ca471dc20903161457w2be669deud7b41abf57ecb56e@mail.gmail.com>

On Mon, Mar 16, 2009 at 2:37 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Guido van Rossum wrote:
>> Yeah, it really seems pretty much limited to contextlib.nested(). I'd
>> be happy to sacrifice the possibility to *exactly* emulate two nested
>> with-statements.
>
> Then I really haven't explained the problem well at all. One of the
> premises of PEP 343 was "Got a frequently recurring block of code that
> only has one variant sequence of statements somewhere in the middle?
> Well, now you can factor that out by putting it in a generator,
> replacing the part that varies with a yield statement and decorating the
> generator with contextlib.contextmanager."
>
> It turns out that there's a caveat that needs to go on the end of that
> though: "Be very, very sure that the yield statement can *never* be
> skipped or your context manager based version will raise a RuntimeError
> in cases where the original code would have just skipped over the
> variant section of code and resumed execution afterwards."

Well, I don't think you can take that premise (promise? :-) literally
anyways, since you cannot turn a loop into a with-statement either. So
I would be fine with just adding the condition that the variant
sequence should be executed exactly once.

> Nested context managers (whether through contextlib.nested or through
> syntactic support) just turns out to be a common case where you *don't
> necessarily know* just by looking at the code whether it can skip over
> the body of the code or not.
>
> Suppose you have 3 context managers that are regularly used together
> (call them cmA(), cmB(), cmC() for now).
>
> Writing that as:
>
> ?with cmA():
> ? ?with cmB():
> ? ? ?with cmC():
> ? ? ? ?do_something()
>
> Or the tentatively proposed:
>
> ?with cmA(), cmB(), cmC():
> ? ? ? ?do_something()
>
> is definitely OK, regardless of the details of the context managers.
>
> However, whether or not you can bundle that up into a *new* context
> manager (regardless of nesting syntax) depends on whether or not an
> outer context manager can suppress an exception raised by an inner one.
>
> ?@contextmanager
> ?def cmABC():
> ? ?with cmA():
> ? ? ?with cmB():
> ? ? ? ?with cmC():
> ? ? ? ? ?yield
>
> ?with cmABC():
> ? ?do_something()

While all this may make sense to the original inventor of context
managers (== you ;-), I personally find this example quite perverse.
Do you have an example taken from real life?

> The above is broken if cmB().__enter__() or cmC.__enter__() can raise an
> exception that cmA().__exit__() suppresses, or cmB.__enter__() raises an
> exception that cmB().__exit__() suppresses. So whereas the inline
> versions were clearly correct, the correctness of the second version
> currently depends on details of the context managers themselves.
> Changing the syntax to allow the three context managers to be written on
> one line does nothing to fix that semantic discrepancy between the
> original inline code and the factored out version.

I think I understand that, I just don't see a use case so important as
to warrant introducing a brand new exception deriving from
BaseException.

> PEP 377 is about changing the with statement semantics and the
> @contextmanager implementation so that the semantics of the factored out
> version actually matches that of the original inline code.
>
> You can get yourself into similar trouble without nesting context
> managers - all it takes is some way of skipping the variant code in a
> context manager that wouldn't have raised an exception if the code was
> written out inline instead of being factored out into the context manager.

Yeah, see above -- you can't write a context manager that implements a
loop either.

> Suppose for instance you wanted to use a context manager as a different
> way of running tests:
>
> ?@contextmanager
> ?def inline_test(self, *setup_args):
> ? ?try:
> ? ? ?self.setup(*setup_args)
> ? ?except:
> ? ? ?# Setup exception occurred, trap it and log it
> ? ? ?return
> ? ?try:
> ? ? ?yield
> ? ?except:
> ? ? ?# Test exception occurred, trap it and log it
> ? ?finally:
> ? ? ?try:
> ? ? ? ?self.teardown()
> ? ? ?except:
> ? ? ? ?# Teardown exception occurred, trap it and log it
>
> ?with inline_test(setup1):
> ? ?test_one()
> ?with inline_test(setup2):
> ? ?test_two()
> ?with inline_test(setup3):
> ? ?test_three()
>
> That approach isn't actually valid

(but your proposal would make it valid right?)

> - a context manager is not permitted
> to decide in it's __enter__() method that executing the body of the with
> statement would be a bad idea.

A setup failure sounds like a catastrophic error to me.

> The early return in the above makes it obvious that that CM is broken
> under the current semantics, but what about the following one:
>
> ?@contextmanager
> ?def broken_cm(self):
> ? ?try:
> ? ? ?call_user_setup()
> ? ? ?try:
> ? ? ? ?yield
> ? ? ?finally:
> ? ? ? ?call_user_teardown()
> ? ?except UserCancel:
> ? ? ?show_message("Operation aborted by user")
>
> That CM will raise RuntimeError if the user attempts to cancel an
> operation during the execution of the "call_user_setup()" method.
> Without SkipStatement or something like it, that can't be fixed.

Well pretty much anything that tries to catch asynchronous exceptions
is doomed to be 100% correct. Again the example is too abstract to be
convincing.

> Hell, I largely wrote PEP 377 to try to get out of having to document
> these semantic problems with the with statement - if I'm having trouble
> getting *python-dev* to grasp the problem, what hope do other users of
> Python have?

Hell, if you can't come up with a real use case, why bother? :-)

Perhaps you could address my worry about introducing an obscure
BaseException subclass that will forever add to the weight of the list
of built-in exceptions in all documentation?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Mon Mar 16 22:58:11 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 17 Mar 2009 07:58:11 +1000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <gpl1s2$sie$1@ger.gmane.org>
References: <49BCFAEE.2070603@gmail.com>	<20090315183752.12555.868958475.divmod.xquotient.6154@weber.divmod.com>	<49BD64F5.104@gmail.com>
	<gpl1s2$sie$1@ger.gmane.org>
Message-ID: <49BECB73.3050800@gmail.com>

Baptiste Carvello wrote:
> Nick Coghlan a ?crit :
>>
>> Implementing __with__ instead would give the CM complete control over
>> whether or not to execute the block.
>>
> please note, however, that this is an important change in the semantics
> of the with statement. As things are today, barring exceptional
> circunstances, the body of the with statement *will* be executed
> immediately. This allows to forget about the with statement when first
> reading code, as it doesn't change the intent of the programmer. What
> Glyph is proposing is more akin to Ruby code blocks.

Yep - the idea is intriguing, powerful, and far beyond the scope of what
I'm trying to achieve with PEP 377. I suspect such blocks would have to
be more along the lines of what PJE suggested in order to be practical
anyway (i.e. nested functions where all "local" variables were
implicitly declared as nonlocal), which then leads to the question of
how much of a gain they would actually provide now that you *can*
explicitly declare nonlocal variables.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Mar 16 23:19:07 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 17 Mar 2009 08:19:07 +1000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <ca471dc20903161457w2be669deud7b41abf57ecb56e@mail.gmail.com>
References: <49BCFAEE.2070603@gmail.com> <49BD3FD7.3080309@voidspace.org.uk>	
	<20090315223016.GA27106@panix.com>
	<49BD8317.3090704@voidspace.org.uk>	 <49BE3B4F.3060300@gmail.com>	
	<ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>	
	<ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>	
	<d11dcfba0903161126y6fc234f9pb387f65cdcd7b605@mail.gmail.com>	
	<ca471dc20903161133g3afcae32q5500c94f6c924def@mail.gmail.com>	
	<49BEC6A1.3050504@gmail.com>
	<ca471dc20903161457w2be669deud7b41abf57ecb56e@mail.gmail.com>
Message-ID: <49BED05B.1020504@gmail.com>

Guido van Rossum wrote:
>> Hell, I largely wrote PEP 377 to try to get out of having to document
>> these semantic problems with the with statement - if I'm having trouble
>> getting *python-dev* to grasp the problem, what hope do other users of
>> Python have?
> 
> Hell, if you can't come up with a real use case, why bother? :-)

I figured I'd try for a solution that didn't offend my sense of
aesthetics before caving in and working out how to better document the
limitations of the status quo :)

> Perhaps you could address my worry about introducing an obscure
> BaseException subclass that will forever add to the weight of the list
> of built-in exceptions in all documentation?

Since this is really just a matter of the aesthetics of the underlying
design from my point of view rather than solving a real world problem, I
don't have a good answer for you.

In the absence of reports of actual problems caused by this limitation,
should I consider the PEP rejected?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From guido at python.org  Mon Mar 16 23:27:52 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Mar 2009 15:27:52 -0700
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
In-Reply-To: <49BED05B.1020504@gmail.com>
References: <49BCFAEE.2070603@gmail.com> <49BD8317.3090704@voidspace.org.uk>
	<49BE3B4F.3060300@gmail.com>
	<ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>
	<ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>
	<d11dcfba0903161126y6fc234f9pb387f65cdcd7b605@mail.gmail.com>
	<ca471dc20903161133g3afcae32q5500c94f6c924def@mail.gmail.com>
	<49BEC6A1.3050504@gmail.com>
	<ca471dc20903161457w2be669deud7b41abf57ecb56e@mail.gmail.com>
	<49BED05B.1020504@gmail.com>
Message-ID: <ca471dc20903161527m3252cf09u64d44ac20f73f4e9@mail.gmail.com>

On Mon, Mar 16, 2009 at 3:19 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Guido van Rossum wrote:
>>> Hell, I largely wrote PEP 377 to try to get out of having to document
>>> these semantic problems with the with statement - if I'm having trouble
>>> getting *python-dev* to grasp the problem, what hope do other users of
>>> Python have?
>>
>> Hell, if you can't come up with a real use case, why bother? :-)
>
> I figured I'd try for a solution that didn't offend my sense of
> aesthetics before caving in and working out how to better document the
> limitations of the status quo :)
>
>> Perhaps you could address my worry about introducing an obscure
>> BaseException subclass that will forever add to the weight of the list
>> of built-in exceptions in all documentation?
>
> Since this is really just a matter of the aesthetics of the underlying
> design from my point of view rather than solving a real world problem, I
> don't have a good answer for you.
>
> In the absence of reports of actual problems caused by this limitation,
> should I consider the PEP rejected?

Yes -- sorry for your wasted efforts.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Mon Mar 16 23:39:13 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 16 Mar 2009 15:39:13 -0700
Subject: [Python-Dev] What level of detail wanted for import and the
	language reference?
Message-ID: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>

At this point importlib is done for its public API for Python 3.1. That
means it's time to turn my attention to making sure the semantics of import
are well documented. But where to put all of the details? The language
reference for import (
http://docs.python.org/dev/py3k/reference/simple_stmts.html#the-import-statement)
explains the basics, but is lacking all of the details of PEP 302 and other
stuff like __path__ that have existed for ages.

My question is if I should flesh out the details in the language reference
or do it in importlib's intro docs. The main reason I could see not doing it
in the langauge reference (or at least duplicating it) is it would be
somewhat easier to reference specific objects in importlib but I am not sure
if the language reference should try to stay away from stdlib references.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090316/01132043/attachment-0001.htm>

From ncoghlan at gmail.com  Mon Mar 16 23:40:49 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 17 Mar 2009 08:40:49 +1000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <ca471dc20903161527m3252cf09u64d44ac20f73f4e9@mail.gmail.com>
References: <49BCFAEE.2070603@gmail.com> <49BD8317.3090704@voidspace.org.uk>	
	<49BE3B4F.3060300@gmail.com>	
	<ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>	
	<ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>	
	<d11dcfba0903161126y6fc234f9pb387f65cdcd7b605@mail.gmail.com>	
	<ca471dc20903161133g3afcae32q5500c94f6c924def@mail.gmail.com>	
	<49BEC6A1.3050504@gmail.com>	
	<ca471dc20903161457w2be669deud7b41abf57ecb56e@mail.gmail.com>	
	<49BED05B.1020504@gmail.com>
	<ca471dc20903161527m3252cf09u64d44ac20f73f4e9@mail.gmail.com>
Message-ID: <49BED571.6070002@gmail.com>

Guido van Rossum wrote:
> On Mon, Mar 16, 2009 at 3:19 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Guido van Rossum wrote:
>>>> Hell, I largely wrote PEP 377 to try to get out of having to document
>>>> these semantic problems with the with statement - if I'm having trouble
>>>> getting *python-dev* to grasp the problem, what hope do other users of
>>>> Python have?
>>> Hell, if you can't come up with a real use case, why bother? :-)
>> I figured I'd try for a solution that didn't offend my sense of
>> aesthetics before caving in and working out how to better document the
>> limitations of the status quo :)
>>
>>> Perhaps you could address my worry about introducing an obscure
>>> BaseException subclass that will forever add to the weight of the list
>>> of built-in exceptions in all documentation?
>> Since this is really just a matter of the aesthetics of the underlying
>> design from my point of view rather than solving a real world problem, I
>> don't have a good answer for you.
>>
>> In the absence of reports of actual problems caused by this limitation,
>> should I consider the PEP rejected?
> 
> Yes -- sorry for your wasted efforts.

Not wasted - I prefer having this as a recognised limitation of the
semantics rather than as an accident of the implementation.

Who knows, maybe somebody will come up with a real world use case some
day and we can drag the PEP out and dust it off a bit :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From fuzzyman at voidspace.org.uk  Mon Mar 16 23:43:31 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 16 Mar 2009 22:43:31 +0000
Subject: [Python-Dev] What level of detail wanted for import and
 the	language reference?
In-Reply-To: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
References: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
Message-ID: <49BED612.7010706@voidspace.org.uk>

Brett Cannon wrote:
> At this point importlib is done for its public API for Python 3.1. 
> That means it's time to turn my attention to making sure the semantics 
> of import are well documented. But where to put all of the details? 
> The language reference for import 
> (http://docs.python.org/dev/py3k/reference/simple_stmts.html#the-import-statement) 
> explains the basics, but is lacking all of the details of PEP 302 and 
> other stuff like __path__ that have existed for ages.
>
> My question is if I should flesh out the details in the language 
> reference or do it in importlib's intro docs. The main reason I could 
> see not doing it in the langauge reference (or at least duplicating 
> it) is it would be somewhat easier to reference specific objects in 
> importlib but I am not sure if the language reference should try to 
> stay away from stdlib references.

Having the Python import semantics well documented will be an 
*excellent* side effect of importlib. Thank you for your astonishing 
efforts on this project.

Personally I would rather see the import semantics themselves in the 
language reference.

Michael
>
> -Brett
> ------------------------------------------------------------------------
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From benjamin at python.org  Mon Mar 16 23:59:52 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 16 Mar 2009 17:59:52 -0500
Subject: [Python-Dev] What level of detail wanted for import and the
	language reference?
In-Reply-To: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
References: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
Message-ID: <1afaf6160903161559w4fe82af2ta4e1f4ad1762d603@mail.gmail.com>

2009/3/16 Brett Cannon <brett at python.org>:
> At this point importlib is done for its public API for Python 3.1. That
> means it's time to turn my attention to making sure the semantics of import
> are well documented. But where to put all of the details? The language
> reference for import
> (http://docs.python.org/dev/py3k/reference/simple_stmts.html#the-import-statement)
> explains the basics, but is lacking all of the details of PEP 302 and other
> stuff like __path__ that have existed for ages.
>
> My question is if I should flesh out the details in the language reference
> or do it in importlib's intro docs. The main reason I could see not doing it
> in the langauge reference (or at least duplicating it) is it would be
> somewhat easier to reference specific objects in importlib but I am not sure
> if the language reference should try to stay away from stdlib references.

Thanks so much for doing this! Personally, I think you should put it
in the language reference. (I think it deserves it's own file if it's
as big as I suspect it will be.) If I wanted to use importlib, I
wouldn't really want to slog through a in-depth description of how
exactly import works.



-- 
Regards,
Benjamin

From aahz at pythoncraft.com  Tue Mar 17 00:51:32 2009
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 16 Mar 2009 16:51:32 -0700
Subject: [Python-Dev] What level of detail wanted for import and
	the	language reference?
In-Reply-To: <49BED612.7010706@voidspace.org.uk>
References: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
	<49BED612.7010706@voidspace.org.uk>
Message-ID: <20090316235132.GA26292@panix.com>

On Mon, Mar 16, 2009, Michael Foord wrote:
>
> Personally I would rather see the import semantics themselves in the  
> language reference.

Either way is fine with me, but it needs to be cross-referenced.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Adopt A Process -- stop killing all your children!

From greg.ewing at canterbury.ac.nz  Tue Mar 17 01:11:57 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 17 Mar 2009 12:11:57 +1200
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <49BEC6A1.3050504@gmail.com>
References: <49BCFAEE.2070603@gmail.com>
	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>
	<49BD3FD7.3080309@voidspace.org.uk> <20090315223016.GA27106@panix.com>
	<49BD8317.3090704@voidspace.org.uk> <49BE3B4F.3060300@gmail.com>
	<ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>
	<ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>
	<d11dcfba0903161126y6fc234f9pb387f65cdcd7b605@mail.gmail.com>
	<ca471dc20903161133g3afcae32q5500c94f6c924def@mail.gmail.com>
	<49BEC6A1.3050504@gmail.com>
Message-ID: <49BEEACD.1050709@canterbury.ac.nz>

Nick Coghlan wrote:
> One of the
> premises of PEP 343 was "Got a frequently recurring block of code that
> only has one variant sequence of statements somewhere in the middle?
> Well, now you can factor that out

Um, no -- it says explicitly right at the very top of
PEP 343 that it's only about factoring out try/finally
statements.

There's no way that

   try:
     code_block
   finally:
     ...

can fail to enter the code block if you get as far as
the "try". So it's not reasonable to expect the with
statement to provide this ability.

-- 
Greg

From steve at holdenweb.com  Tue Mar 17 04:30:08 2009
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 16 Mar 2009 23:30:08 -0400
Subject: [Python-Dev] What level of detail wanted for import and the
	language reference?
In-Reply-To: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
References: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
Message-ID: <49BF1940.6050101@holdenweb.com>

Brett Cannon wrote:
> At this point importlib is done for its public API for Python 3.1. That
> means it's time to turn my attention to making sure the semantics of
> import are well documented. But where to put all of the details? The
> language reference for import
> (http://docs.python.org/dev/py3k/reference/simple_stmts.html#the-import-statement)
> explains the basics, but is lacking all of the details of PEP 302 and
> other stuff like __path__ that have existed for ages.
> 
> My question is if I should flesh out the details in the language
> reference or do it in importlib's intro docs. The main reason I could
> see not doing it in the langauge reference (or at least duplicating it)
> is it would be somewhat easier to reference specific objects in
> importlib but I am not sure if the language reference should try to stay
> away from stdlib references.

Why not just put a section in both places that says "can't be bothered
to spell this out right now" and put a URL in referring to this thread
on Google ... that appears to have been the traditional approach to
import semantics :)

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Tue Mar 17 04:30:08 2009
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 16 Mar 2009 23:30:08 -0400
Subject: [Python-Dev] What level of detail wanted for import and the
	language reference?
In-Reply-To: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
References: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
Message-ID: <49BF1940.6050101@holdenweb.com>

Brett Cannon wrote:
> At this point importlib is done for its public API for Python 3.1. That
> means it's time to turn my attention to making sure the semantics of
> import are well documented. But where to put all of the details? The
> language reference for import
> (http://docs.python.org/dev/py3k/reference/simple_stmts.html#the-import-statement)
> explains the basics, but is lacking all of the details of PEP 302 and
> other stuff like __path__ that have existed for ages.
> 
> My question is if I should flesh out the details in the language
> reference or do it in importlib's intro docs. The main reason I could
> see not doing it in the langauge reference (or at least duplicating it)
> is it would be somewhat easier to reference specific objects in
> importlib but I am not sure if the language reference should try to stay
> away from stdlib references.

Why not just put a section in both places that says "can't be bothered
to spell this out right now" and put a URL in referring to this thread
on Google ... that appears to have been the traditional approach to
import semantics :)

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From lists at jwp.name  Tue Mar 17 04:36:26 2009
From: lists at jwp.name (James Pye)
Date: Mon, 16 Mar 2009 20:36:26 -0700
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
	statement body
In-Reply-To: <49BED571.6070002@gmail.com>
References: <49BCFAEE.2070603@gmail.com> <49BD8317.3090704@voidspace.org.uk>	
	<49BE3B4F.3060300@gmail.com>	
	<ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>	
	<ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>	
	<d11dcfba0903161126y6fc234f9pb387f65cdcd7b605@mail.gmail.com>	
	<ca471dc20903161133g3afcae32q5500c94f6c924def@mail.gmail.com>	
	<49BEC6A1.3050504@gmail.com>	
	<ca471dc20903161457w2be669deud7b41abf57ecb56e@mail.gmail.com>	
	<49BED05B.1020504@gmail.com>
	<ca471dc20903161527m3252cf09u64d44ac20f73f4e9@mail.gmail.com>
	<49BED571.6070002@gmail.com>
Message-ID: <A58DCFB4-C0EC-418C-A8A1-2BEB5AE06EC1@jwp.name>

On Mar 16, 2009, at 3:40 PM, Nick Coghlan wrote:
> Not wasted - I prefer having this as a recognised limitation of the
> semantics rather than as an accident of the implementation.

Well, I'm glad some good came from the issue report. =)

> Who knows, maybe somebody will come up with a real world use case some
> day and we can drag the PEP out and dust it off a bit :)


I don't expect this to be at all compelling, but FWIW:

The identification of this issue came from an *experiment* attempting  
to create a *single* "daemonized()" CM that would execute the with- 
statement's block in a new child process and, of course, not execute  
it in the parent. At first, I ran into the RuntimeError in the parent  
process, and then after rewriting the CMs as classes, I realized the  
futility.

with daemonized():
  run_some_subprocess()

Of course it was all possible if I used the component CMs directly:

with parent_trap():
  with fork_but_raise_in_parent():
   run_some_subprocess()

And thus:

def daemonized():
  return contextlib.nested(parent_trap(), fork_but_raise_in_parent())

From ben+python at benfinney.id.au  Tue Mar 17 05:16:19 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 17 Mar 2009 15:16:19 +1100
Subject: [Python-Dev] =?utf-8?q?Daemon_process_context_as_a_=E2=80=98with?=
 =?utf-8?q?=E2=80=99_context_manager_=28was=3A_PEP_377_-_allow_=5F=5Fenter?=
 =?utf-8?q?=5F=5F=28=29_methods_to_skip_the_statement_body=29?=
References: <49BCFAEE.2070603@gmail.com> <49BD8317.3090704@voidspace.org.uk>
	<49BE3B4F.3060300@gmail.com>
	<ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>
	<ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>
	<d11dcfba0903161126y6fc234f9pb387f65cdcd7b605@mail.gmail.com>
	<ca471dc20903161133g3afcae32q5500c94f6c924def@mail.gmail.com>
	<49BEC6A1.3050504@gmail.com>
	<ca471dc20903161457w2be669deud7b41abf57ecb56e@mail.gmail.com>
	<49BED05B.1020504@gmail.com>
	<ca471dc20903161527m3252cf09u64d44ac20f73f4e9@mail.gmail.com>
	<49BED571.6070002@gmail.com>
	<A58DCFB4-C0EC-418C-A8A1-2BEB5AE06EC1@jwp.name>
Message-ID: <87k56oss18.fsf_-_@benfinney.id.au>

James Pye <lists at jwp.name> writes:

> The identification of this issue came from an *experiment* attempting
> to create a *single* "daemonized()" CM that would execute the with- 
> statement's block in a new child process and, of course, not execute
> it in the parent. At first, I ran into the RuntimeError in the parent
> process, and then after rewriting the CMs as classes, I realized the
> futility.
> 
> with daemonized():
>  run_some_subprocess()

This use case is addressed by my up-coming PEP, currently in draft
form but submitted to the PEP editor. A reference implementation is at
<URL:http://pypi.python.org/pypi/python-daemon/>.

I'd like to know whether this meets your needs; follow-ups privately
to me or in ?python-ideas? as you think appropriate.

-- 
 \                             ?Holy tintinnabulation, Batman!? ?Robin |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From ben+python at benfinney.id.au  Tue Mar 17 07:35:23 2009
From: ben+python at benfinney.id.au (Ben Finney)
Date: Tue, 17 Mar 2009 17:35:23 +1100
Subject: [Python-Dev]
 =?utf-8?q?Daemon_process_context_as_a_=E2=80=98with?=
 =?utf-8?q?=E2=80=99__context_manager?=
References: <49BCFAEE.2070603@gmail.com> <49BD8317.3090704@voidspace.org.uk>
	<49BE3B4F.3060300@gmail.com>
	<ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>
	<ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>
	<d11dcfba0903161126y6fc234f9pb387f65cdcd7b605@mail.gmail.com>
	<ca471dc20903161133g3afcae32q5500c94f6c924def@mail.gmail.com>
	<49BEC6A1.3050504@gmail.com>
	<ca471dc20903161457w2be669deud7b41abf57ecb56e@mail.gmail.com>
	<49BED05B.1020504@gmail.com>
	<ca471dc20903161527m3252cf09u64d44ac20f73f4e9@mail.gmail.com>
	<49BED571.6070002@gmail.com>
	<A58DCFB4-C0EC-418C-A8A1-2BEB5AE06EC1@jwp.name>
	<87k56oss18.fsf_-_@benfinney.id.au>
Message-ID: <87bps0sllg.fsf@benfinney.id.au>

Ben Finney <ben+python at benfinney.id.au> writes:

> James Pye <lists at jwp.name> writes:
> 
> > with daemonized():
> >  run_some_subprocess()
> 
> This use case is addressed by my up-coming PEP, currently in draft
> form but submitted to the PEP editor. A reference implementation is
> at <URL:http://pypi.python.org/pypi/python-daemon/>.

On second blush, it addresses a smaller use case, with the same syntax
as your example. It should still be applicable at some level.

> I'd like to know whether this meets your needs; follow-ups privately
> to me or in ?python-ideas? as you think appropriate.

Ditto.

-- 
 \      ?Life does not cease to be funny when people die any more than |
  `\  it ceases to be serious when people laugh.? ?George Bernard Shaw |
_o__)                                                                  |
Ben Finney


From victor.stinner at haypocalc.com  Tue Mar 17 13:52:16 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 17 Mar 2009 13:52:16 +0100
Subject: [Python-Dev] py3k: accept unicode for 'c' and byte for 'C' in
	getarg?
Message-ID: <200903171352.16487.victor.stinner@haypocalc.com>

Hi,

I realised with the issue #3446 that getarg('c') (get a byte) accepts not only 
a byte string of 1 byte, but also an unicode string of 1 character (if the 
character code is in [0; 255]). I don't think that it's a good idea to accept 
unicode here. Example: b"x".center(5, "\xe9") should be a TypeError.

The "C" format (get a character) has the opposite problem: it accepts both 
byte and unicode, whereas byte should be rejected. Example: 
mmap.write_byte('?') should be a TypeError.

The problem was already discuss in the email thread "What type of object 
mmap.read_byte should return on py3k?" started by Hirokazu Yamamoto related 
to issue #5391.

Short history:
 - r55109: Guido changes 'c' format to accept unicode (struni branch).
   getarg('c') => char accepts byte and character
 - r56044: walter.doerwald changes the 'c' format to return an int (an 
   unicode character) for datetime.datetime.isoformat().
   getarg('c') => int accepts byte and character
 - r56140: Revert r56044 and creates 'C' format
   getarg('c') => char accepts byte and character
   getarg('C') => int accepts byte and character

So we have:
 - getarg('c') -> one byte (integer in [0; 255])
 - getarg('C') -> one character (code in [0; INTMAX])
   Note: Why not using Py_UNICODE instead of int?

Usage of "C" format:
  datetime.datetime.isoformat(sep)
  array.array(type, data): type

Usage of "c" format:
  msvcrt.putch(char)
  msvcrt.ungetch(char)
  <mmap object>.write_byte(char)

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From victor.stinner at haypocalc.com  Tue Mar 17 14:00:14 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 17 Mar 2009 14:00:14 +0100
Subject: [Python-Dev] py3k: accept unicode for 'c' and byte for 'C' in
	getarg?
In-Reply-To: <200903171352.16487.victor.stinner@haypocalc.com>
References: <200903171352.16487.victor.stinner@haypocalc.com>
Message-ID: <200903171400.14653.victor.stinner@haypocalc.com>

Le Tuesday 17 March 2009 13:52:16 Victor Stinner, vous avez ?crit?:
> I realised with the issue #3446 that getarg('c') (get a byte) accepts not
> only a byte string of 1 byte, but also an unicode string of 1 character (if
> the character code is in [0; 255]). I don't think that it's a good idea to
> accept unicode here. Example: b"x".center(5, "\xe9") should be a TypeError.
>
> The "C" format (get a character) has the opposite problem: it accepts both
> byte and unicode, whereas byte should be rejected. Example:
> mmap.write_byte('?') should be a TypeError.

I opened an issue (with a patch) for my request:
   http://bugs.python.org/issue5499

Note: except of the bytes.{center,ljust,rjust} tests, all other tests are ok. 
It should not break the standard library, but maybe 3rd party library.

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From ncoghlan at gmail.com  Tue Mar 17 14:55:53 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 17 Mar 2009 23:55:53 +1000
Subject: [Python-Dev] What level of detail wanted for import and
 the	language reference?
In-Reply-To: <49BF1940.6050101@holdenweb.com>
References: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
	<49BF1940.6050101@holdenweb.com>
Message-ID: <49BFABE9.1070806@gmail.com>

Steve Holden wrote:
> Why not just put a section in both places that says "can't be bothered
> to spell this out right now" and put a URL in referring to this thread
> on Google ... that appears to have been the traditional approach to
> import semantics :)

Well, first we point to Guido's original package essay, then to PEP 302,
then to PEP 338, then to PEP 366... and then we count the number of
people that slogged through all that without needing to take our shoes
off :)

On a more helpful note, Brett, you may find the write-up I did of the
import system a year or two ago helpful:
http://svn.python.org/view/sandbox/trunk/userref/ODF/Chapter07_ModulesAndApplications.odt?view=log

(some parts are a little dated now obviously, but you may still find it
better than starting with a blank page)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Tue Mar 17 15:11:44 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 18 Mar 2009 00:11:44 +1000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <49BEEACD.1050709@canterbury.ac.nz>
References: <49BCFAEE.2070603@gmail.com>	<bbaeab100903151040w2c1ff20av97e5fd8c00da1800@mail.gmail.com>	<49BD3FD7.3080309@voidspace.org.uk>
	<20090315223016.GA27106@panix.com>	<49BD8317.3090704@voidspace.org.uk>
	<49BE3B4F.3060300@gmail.com>	<ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>	<ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>	<d11dcfba0903161126y6fc234f9pb387f65cdcd7b605@mail.gmail.com>	<ca471dc20903161133g3afcae32q5500c94f6c924def@mail.gmail.com>	<49BEC6A1.3050504@gmail.com>
	<49BEEACD.1050709@canterbury.ac.nz>
Message-ID: <49BFAFA0.5040003@gmail.com>

Greg Ewing wrote:
> Um, no -- it says explicitly right at the very top of
> PEP 343 that it's only about factoring out try/finally
> statements.
> 
> There's no way that
> 
>   try:
>     code_block
>   finally:
>     ...
> 
> can fail to enter the code block if you get as far as
> the "try". So it's not reasonable to expect the with
> statement to provide this ability.

We were working on the PEP for over a year though - expectations changed
a bit when the ability to suppress exceptions was added (that wasn't in
the original version of the PEP which was when most of the first few
sections was written).

I agree that try/finally was by far the main use case - it just isn't
the only use case or we would never have included the option to suppress
exceptions at all.

Still, I'm happy to let this rest indefinitely - I doubt the
daemonisation use case is going to be enough to change Guido's mind, and
it really is rather difficult to come up with practical examples where
the current behaviour is a genuine problem.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From rdmurray at bitdance.com  Tue Mar 17 15:18:52 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 17 Mar 2009 10:18:52 -0400 (EDT)
Subject: [Python-Dev] python-3000 is still mentioned on the mailing lists
	page
Message-ID: <Pine.LNX.4.64.0903171010460.26433@kimball.webabinitio.net>

I just noticed that the python-3000 list is still mentioned on
http://www.python.org/community/lists/.  I searched around a bit
but it isn't obvious to me where other than here I should report this,
so I'm reporting it here :).

--
R. David Murray           http://www.bitdance.com

From skip at pobox.com  Tue Mar 17 16:20:01 2009
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 17 Mar 2009 10:20:01 -0500
Subject: [Python-Dev] python-3000 is still mentioned on the mailing
 lists page
In-Reply-To: <Pine.LNX.4.64.0903171010460.26433@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0903171010460.26433@kimball.webabinitio.net>
Message-ID: <18879.49057.770936.289161@montanaro.dyndns.org>

    David> I just noticed that the python-3000 list is still mentioned on
    David> http://www.python.org/community/lists/.

Thanks.  I passed your note along to the postmaster(s).

Skip

From guido at python.org  Tue Mar 17 17:27:39 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Mar 2009 09:27:39 -0700
Subject: [Python-Dev] py3k: accept unicode for 'c' and byte for 'C' in
	getarg?
In-Reply-To: <200903171352.16487.victor.stinner@haypocalc.com>
References: <200903171352.16487.victor.stinner@haypocalc.com>
Message-ID: <ca471dc20903170927y711667a3r5543b7cc1cfaaac8@mail.gmail.com>

On Tue, Mar 17, 2009 at 5:52 AM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> I realised with the issue #3446 that getarg('c') (get a byte) accepts not only
> a byte string of 1 byte, but also an unicode string of 1 character (if the
> character code is in [0; 255]). I don't think that it's a good idea to accept
> unicode here. Example: b"x".center(5, "\xe9") should be a TypeError.

Agreed.

> The "C" format (get a character) has the opposite problem: it accepts both
> byte and unicode, whereas byte should be rejected. Example:
> mmap.write_byte('?') should be a TypeError.

YEah, mmap should be defined exclusively in terms of bytes.

> The problem was already discuss in the email thread "What type of object
> mmap.read_byte should return on py3k?" started by Hirokazu Yamamoto related
> to issue #5391.
>
> Short history:
> ?- r55109: Guido changes 'c' format to accept unicode (struni branch).
> ? getarg('c') => char accepts byte and character
> ?- r56044: walter.doerwald changes the 'c' format to return an int (an
> ? unicode character) for datetime.datetime.isoformat().
> ? getarg('c') => int accepts byte and character
> ?- r56140: Revert r56044 and creates 'C' format
> ? getarg('c') => char accepts byte and character
> ? getarg('C') => int accepts byte and character
>
> So we have:
> ?- getarg('c') -> one byte (integer in [0; 255])
> ?- getarg('C') -> one character (code in [0; INTMAX])
> ? Note: Why not using Py_UNICODE instead of int?
>
> Usage of "C" format:
> ?datetime.datetime.isoformat(sep)
> ?array.array(type, data): type
>
> Usage of "c" format:
> ?msvcrt.putch(char)
> ?msvcrt.ungetch(char)

ISTM that putch() and ungetch() are text operations so should use 'C'.

> ?<mmap object>.write_byte(char)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Tue Mar 17 17:46:43 2009
From: brett at python.org (Brett Cannon)
Date: Tue, 17 Mar 2009 09:46:43 -0700
Subject: [Python-Dev] What level of detail wanted for import and the
	language reference?
In-Reply-To: <49BFABE9.1070806@gmail.com>
References: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
	<49BF1940.6050101@holdenweb.com> <49BFABE9.1070806@gmail.com>
Message-ID: <bbaeab100903170946s1b1ea57dgebe2a246aabb236a@mail.gmail.com>

On Tue, Mar 17, 2009 at 06:55, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Steve Holden wrote:
> > Why not just put a section in both places that says "can't be bothered
> > to spell this out right now" and put a URL in referring to this thread
> > on Google ... that appears to have been the traditional approach to
> > import semantics :)
>
> Well, first we point to Guido's original package essay, then to PEP 302,
> then to PEP 338, then to PEP 366... and then we count the number of
> people that slogged through all that without needing to take our shoes
> off :)
>

The See Also section in importlib probably has the most links of any other
module in the stdlib.


>
> On a more helpful note, Brett, you may find the write-up I did of the
> import system a year or two ago helpful:
>
> http://svn.python.org/view/sandbox/trunk/userref/ODF/Chapter07_ModulesAndApplications.odt?view=log
>
> (some parts are a little dated now obviously, but you may still find it
> better than starting with a blank page)
>

I'll have a look.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090317/1f4cf883/attachment.htm>

From victor.stinner at haypocalc.com  Tue Mar 17 18:03:32 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 17 Mar 2009 18:03:32 +0100
Subject: [Python-Dev] py3k: accept unicode for 'c' and byte for 'C' in
	getarg?
In-Reply-To: <ca471dc20903170927y711667a3r5543b7cc1cfaaac8@mail.gmail.com>
References: <200903171352.16487.victor.stinner@haypocalc.com>
	<ca471dc20903170927y711667a3r5543b7cc1cfaaac8@mail.gmail.com>
Message-ID: <200903171803.32730.victor.stinner@haypocalc.com>

Le Tuesday 17 March 2009 17:27:39, vous avez ?crit?:
> > The "C" format (get a character) has the opposite problem: it accepts
> > both byte and unicode, whereas byte should be rejected. Example:
> > mmap.write_byte('?') should be a TypeError.
>
> YEah, mmap should be defined exclusively in terms of bytes.

It's already the fix (only use bytes) choosen for the mmap issue:
   http://bugs.python.org/issue5391
(the problem is bigger than mmap.write_byte, other methods have to be changed)

> > Usage of "c" format:
> > ?msvcrt.putch(char)
> > ?msvcrt.ungetch(char)
>
> ISTM that putch() and ungetch() are text operations so should use 'C'.

The low level functions use the C type "char":
  _putch(char)=>void
  _ungetch(char)=>char

For text, we have unicode versions of these functions:
  msvcrt.ungetwch(unicode string of 1 character)
  msvcrt.putwch(unicode string of 1 character)

So "c" looks to be the right format for putch() and ungetch().

See also http://bugs.python.org/issue5410

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From guido at python.org  Tue Mar 17 18:31:22 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Mar 2009 10:31:22 -0700
Subject: [Python-Dev] py3k: accept unicode for 'c' and byte for 'C' in
	getarg?
In-Reply-To: <200903171803.32730.victor.stinner@haypocalc.com>
References: <200903171352.16487.victor.stinner@haypocalc.com>
	<ca471dc20903170927y711667a3r5543b7cc1cfaaac8@mail.gmail.com>
	<200903171803.32730.victor.stinner@haypocalc.com>
Message-ID: <ca471dc20903171031l5d700fbbub5e1ae57172a941d@mail.gmail.com>

On Tue, Mar 17, 2009 at 10:03 AM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Le Tuesday 17 March 2009 17:27:39, vous avez ?crit?:
>> > The "C" format (get a character) has the opposite problem: it accepts
>> > both byte and unicode, whereas byte should be rejected. Example:
>> > mmap.write_byte('?') should be a TypeError.
>>
>> YEah, mmap should be defined exclusively in terms of bytes.
>
> It's already the fix (only use bytes) choosen for the mmap issue:
> ? http://bugs.python.org/issue5391
> (the problem is bigger than mmap.write_byte, other methods have to be changed)
>
>> > Usage of "c" format:
>> > ?msvcrt.putch(char)
>> > ?msvcrt.ungetch(char)
>>
>> ISTM that putch() and ungetch() are text operations so should use 'C'.
>
> The low level functions use the C type "char":
> ?_putch(char)=>void
> ?_ungetch(char)=>char

Where did you find these signatures? I looked them up on microsoft.com
and found definitions for _putch() taking an int and _getch()
returning an int.

http://msdn.microsoft.com/en-us/library/azb6c04e.aspx
http://msdn.microsoft.com/en-us/library/078sfkak.aspx

AFAIK the versions without leading underscores are the same.

Also, just because it only takes an ASCII character doesn't mean it's
meant for binary I/O. putch() and getch() are clearly meant for text
I/O.

> For text, we have unicode versions of these functions:
> ?msvcrt.ungetwch(unicode string of 1 character)
> ?msvcrt.putwch(unicode string of 1 character)
>
> So "c" looks to be the right format for putch() and ungetch().
>
> See also http://bugs.python.org/issue5410

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Tue Mar 17 22:52:44 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 17 Mar 2009 14:52:44 -0700
Subject: [Python-Dev] In-place operators
Message-ID: <823F3A4D25064381A7EF2ABFBA57AB3D@RaymondLaptop1>

Does anyone think it was not a good idea to put in-place operations in the operator module?  For some objects, they don't map() as 
well as their regular counterparts.  Some in-place operations rely on the interpreter to take care of the actual assignment.   I've 
not yet seen good use cases for operator.isub() for example.


Raymond 


From benjamin at python.org  Tue Mar 17 22:54:51 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 17 Mar 2009 16:54:51 -0500
Subject: [Python-Dev] In-place operators
In-Reply-To: <823F3A4D25064381A7EF2ABFBA57AB3D@RaymondLaptop1>
References: <823F3A4D25064381A7EF2ABFBA57AB3D@RaymondLaptop1>
Message-ID: <1afaf6160903171454x50993dcs8a51ab6bae9e4f18@mail.gmail.com>

2009/3/17 Raymond Hettinger <python at rcn.com>:
> Does anyone think it was not a good idea to put in-place operations in the
> operator module? ?For some objects, they don't map() as well as their
> regular counterparts. ?Some in-place operations rely on the interpreter to
> take care of the actual assignment. ? I've not yet seen good use cases for
> operator.isub() for example.

I thought the point of the operator module (unlike most modules) was
to provide a comprehensive set of Python operators as functions for
consistency even if there usefulness was questionable.



-- 
Regards,
Benjamin

From guido at python.org  Tue Mar 17 22:58:06 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Mar 2009 14:58:06 -0700
Subject: [Python-Dev] In-place operators
In-Reply-To: <1afaf6160903171454x50993dcs8a51ab6bae9e4f18@mail.gmail.com>
References: <823F3A4D25064381A7EF2ABFBA57AB3D@RaymondLaptop1>
	<1afaf6160903171454x50993dcs8a51ab6bae9e4f18@mail.gmail.com>
Message-ID: <ca471dc20903171458m621223di609ca57b1a9b9441@mail.gmail.com>

On Tue, Mar 17, 2009 at 2:54 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2009/3/17 Raymond Hettinger <python at rcn.com>:
>> Does anyone think it was not a good idea to put in-place operations in the
>> operator module? ?For some objects, they don't map() as well as their
>> regular counterparts. ?Some in-place operations rely on the interpreter to
>> take care of the actual assignment. ? I've not yet seen good use cases for
>> operator.isub() for example.
>
> I thought the point of the operator module (unlike most modules) was
> to provide a comprehensive set of Python operators as functions for
> consistency even if there usefulness was questionable.

Right. Since Python doesn't have a notation like "operator +" for
turning operators into functions, the operator module provides this
functionality. Better safe than sorry.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Tue Mar 17 23:10:53 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 17 Mar 2009 15:10:53 -0700
Subject: [Python-Dev] In-place operators
References: <823F3A4D25064381A7EF2ABFBA57AB3D@RaymondLaptop1>
	<1afaf6160903171454x50993dcs8a51ab6bae9e4f18@mail.gmail.com>
	<ca471dc20903171458m621223di609ca57b1a9b9441@mail.gmail.com>
Message-ID: <C6A5CCDC39D548D482883BE3A832D9DD@RaymondLaptop1>

> Right. Since Python doesn't have a notation like "operator +" for
> turning operators into functions, the operator module provides this
> functionality. Better safe than sorry.

It doesn't really expose this functionality because some of the functionality
is buried in ceval.c and is not exposed by operator.isub() for example.
I don't see that it offers anything useful that can't be accomplished 
by calling a magic method directly.

I'm sure that consistency/completeness/safe_vs_sorry 
was the reason they were added.  But, if they aren't useful, 
they never should have been (IMO).  It wastes the time of 
people who try to use them and then find-out that they don't 
act as expected (the assignment doesn't take place) or that 
you can't use them with containers s[k] += x etc.)  

Maybe someone somewhere has some interesting use for
these in-place operator function.  I hope so.


Raymond

From solipsis at pitrou.net  Tue Mar 17 23:22:00 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 17 Mar 2009 22:22:00 +0000 (UTC)
Subject: [Python-Dev] In-place operators
References: <823F3A4D25064381A7EF2ABFBA57AB3D@RaymondLaptop1>
	<1afaf6160903171454x50993dcs8a51ab6bae9e4f18@mail.gmail.com>
	<ca471dc20903171458m621223di609ca57b1a9b9441@mail.gmail.com>
	<C6A5CCDC39D548D482883BE3A832D9DD@RaymondLaptop1>
Message-ID: <loom.20090317T221909-15@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:
> 
> I'm sure that consistency/completeness/safe_vs_sorry 
> was the reason they were added.  But, if they aren't useful, 
> they never should have been (IMO).  It wastes the time of 
> people who try to use them and then find-out that they don't 
> act as expected (the assignment doesn't take place)

>>> l = []
>>> operator.iadd(l, [5])
[5]
>>> l
[5]

The operation /does/ occur in-place for mutable types. Yes, the semantics are
non-obvious for non-mutable types, but that doesn't make it completely useless.



From martin at v.loewis.de  Tue Mar 17 23:31:02 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 17 Mar 2009 23:31:02 +0100
Subject: [Python-Dev] In-place operators
In-Reply-To: <C6A5CCDC39D548D482883BE3A832D9DD@RaymondLaptop1>
References: <823F3A4D25064381A7EF2ABFBA57AB3D@RaymondLaptop1>	<1afaf6160903171454x50993dcs8a51ab6bae9e4f18@mail.gmail.com>	<ca471dc20903171458m621223di609ca57b1a9b9441@mail.gmail.com>
	<C6A5CCDC39D548D482883BE3A832D9DD@RaymondLaptop1>
Message-ID: <49C024A6.7090609@v.loewis.de>

> I'm sure that consistency/completeness/safe_vs_sorry was the reason they
> were added.  But, if they aren't useful, they never should have been
> (IMO).

Why is that? [you are then giving a reason:]

> It wastes the time of people who try to use them and then
> find-out that they don't act as expected

What people in particular? Certainly, the doc string is wrong:

  isub(a, b) -- Same as a -= b.

That's not quite the same - you would have to write

  a = isub(a, b) -- Same as a -= b.

(right?) However, anybody who understands what isub does already
knows that property. I can't imagine users browsing through the
operator module and thinking "hmm, what might that isub function
do?".

> or that you can't use them with containers s[k] += x etc.) 

Why not?

  s[k] = iadd(s[k], x)

works fine, no?

> Maybe someone somewhere has some interesting use for
> these in-place operator function.  I hope so.

It could be important if you want apply it to mutable objects, i.e.
where the assignment doesn't do anything.

Regards,
Martin

From python at rcn.com  Tue Mar 17 23:35:58 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 17 Mar 2009 15:35:58 -0700
Subject: [Python-Dev] In-place operators
References: <823F3A4D25064381A7EF2ABFBA57AB3D@RaymondLaptop1>	<1afaf6160903171454x50993dcs8a51ab6bae9e4f18@mail.gmail.com>	<ca471dc20903171458m621223di609ca57b1a9b9441@mail.gmail.com>
	<C6A5CCDC39D548D482883BE3A832D9DD@RaymondLaptop1>
	<49C024A6.7090609@v.loewis.de>
Message-ID: <73BD3A6D2A9D480B9B6C0BC6A5815751@RaymondLaptop1>


>> Maybe someone somewhere has some interesting use for
>> these in-place operator function.  I hope so.
> 
> It could be important if you want apply it to mutable objects, i.e.
> where the assignment doesn't do anything.

I give up.  My original question was whether anyone thought these
were a good idea.  Look's like the answer is yes, everyone is 
happy with the functions and are glad they were added in 2.5.


Raymond

From martin at v.loewis.de  Tue Mar 17 23:43:37 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 17 Mar 2009 23:43:37 +0100
Subject: [Python-Dev] In-place operators
In-Reply-To: <73BD3A6D2A9D480B9B6C0BC6A5815751@RaymondLaptop1>
References: <823F3A4D25064381A7EF2ABFBA57AB3D@RaymondLaptop1>	<1afaf6160903171454x50993dcs8a51ab6bae9e4f18@mail.gmail.com>	<ca471dc20903171458m621223di609ca57b1a9b9441@mail.gmail.com>
	<C6A5CCDC39D548D482883BE3A832D9DD@RaymondLaptop1>
	<49C024A6.7090609@v.loewis.de>
	<73BD3A6D2A9D480B9B6C0BC6A5815751@RaymondLaptop1>
Message-ID: <49C02799.5010302@v.loewis.de>

> I give up.  My original question was whether anyone thought these
> were a good idea. Look's like the answer is yes,

Correct.

> everyone is happy with
> the functions and are glad they were added in 2.5.

No. I don't really care - I don't use them, nor do I use anything
else of the operator module (so I wouldn't miss it if it was
removed entirely). I would object to their removal, though,
because it would hurt my sense of symmetry.

Regards,
Martin

P.S. If anything was to be removed, I would remove the in-place
operations altogether; i.e. a+=b should always be a=a+b. Allowing
them to be in-place complicates matters for little added value.

From python at rcn.com  Tue Mar 17 23:55:19 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 17 Mar 2009 15:55:19 -0700
Subject: [Python-Dev] In-place operators
References: <823F3A4D25064381A7EF2ABFBA57AB3D@RaymondLaptop1>	<1afaf6160903171454x50993dcs8a51ab6bae9e4f18@mail.gmail.com>	<ca471dc20903171458m621223di609ca57b1a9b9441@mail.gmail.com>
	<C6A5CCDC39D548D482883BE3A832D9DD@RaymondLaptop1>
	<49C024A6.7090609@v.loewis.de>
	<73BD3A6D2A9D480B9B6C0BC6A5815751@RaymondLaptop1>
	<49C02799.5010302@v.loewis.de>
Message-ID: <3EDA588B50A747B4AA2081DDE3C9ADDB@RaymondLaptop1>


[Martin v. L?wis]
>  I would object to their removal, though,
> because it would hurt my sense of symmetry.

I wasn't going to propose removal.  If everyone had
agreed that the operator in-place functions were
problematic, I was going to suggest moving their
docs to a second page and documenting their limatations
(like we had done long ago with some of the builtins
that were no longer essential and had become obsolete).
That would leave the main page full of the operator
functions that have real utility.

But, if we want them to remain first-class citizens,
they are probably fine where they are.


Raymond


From tjreedy at udel.edu  Wed Mar 18 02:17:03 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 17 Mar 2009 21:17:03 -0400
Subject: [Python-Dev] In-place operators
In-Reply-To: <823F3A4D25064381A7EF2ABFBA57AB3D@RaymondLaptop1>
References: <823F3A4D25064381A7EF2ABFBA57AB3D@RaymondLaptop1>
Message-ID: <gppi2f$nps$1@ger.gmane.org>

Raymond Hettinger wrote:
> Does anyone think it was not a good idea to put in-place operations in 
> the operator module?  For some objects, they don't map() as well as 
> their regular counterparts.  Some in-place operations rely on the 
> interpreter to take care of the actual assignment.   I've not yet seen 
> good use cases for operator.isub() for example.

Given that Python has augmented assignment delimiters, but no 'in-place 
operators', and that the 'in-place operations' used to partially 
implemented them cannot be 'in-place' for immutables (and hence are 
actually aliases for the corresponding 'regular' operations, I agree 
that they are a bit odd and mostly useless.  About the only use case I 
can think of is something like map(operator.iadd, mutable_seqs, items), 
where mutable_seqs includes instances of user classes than defind 
.__iadd__ but not .append ;-)

On the other hand, the kitchen-sink policy saves debate.

tjr


From ncoghlan at gmail.com  Wed Mar 18 12:38:49 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 18 Mar 2009 21:38:49 +1000
Subject: [Python-Dev] In-place operators
In-Reply-To: <3EDA588B50A747B4AA2081DDE3C9ADDB@RaymondLaptop1>
References: <823F3A4D25064381A7EF2ABFBA57AB3D@RaymondLaptop1>	<1afaf6160903171454x50993dcs8a51ab6bae9e4f18@mail.gmail.com>	<ca471dc20903171458m621223di609ca57b1a9b9441@mail.gmail.com>	<C6A5CCDC39D548D482883BE3A832D9DD@RaymondLaptop1>	<49C024A6.7090609@v.loewis.de>	<73BD3A6D2A9D480B9B6C0BC6A5815751@RaymondLaptop1>	<49C02799.5010302@v.loewis.de>
	<3EDA588B50A747B4AA2081DDE3C9ADDB@RaymondLaptop1>
Message-ID: <49C0DD49.8070709@gmail.com>

Raymond Hettinger wrote:
> 
> [Martin v. L?wis]
>>  I would object to their removal, though,
>> because it would hurt my sense of symmetry.
> 
> I wasn't going to propose removal.  If everyone had
> agreed that the operator in-place functions were
> problematic, I was going to suggest moving their
> docs to a second page and documenting their limatations
> (like we had done long ago with some of the builtins
> that were no longer essential and had become obsolete).
> That would leave the main page full of the operator
> functions that have real utility.

Splitting their documentation out to a separate page that explains their
lack of usefulness when dealing with containers or immutable objects
sounds like a great idea. As you say, due to their reliance on a
separate assignment step they really are more limited than the other
functions in the operator module.

You didn't actually make that proposal in your original message though -
you just asked if people thought it was a mistake to have added them to
the operator module (which implied, at least to me, that you were going
to suggest deprecating them).

So +1 from me for changing the operator module docs as you suggest.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From hrvoje.niksic at avl.com  Wed Mar 18 12:47:18 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Wed, 18 Mar 2009 12:47:18 +0100
Subject: [Python-Dev] In-place operators
In-Reply-To: <16028440.3363847.1237329099031.JavaMail.xicrypt@atgrzls001>
References: <823F3A4D25064381A7EF2ABFBA57AB3D@RaymondLaptop1>	<1afaf6160903171454x50993dcs8a51ab6bae9e4f18@mail.gmail.com>	<ca471dc20903171458m621223di609ca57b1a9b9441@mail.gmail.com>	<C6A5CCDC39D548D482883BE3A832D9DD@RaymondLaptop1>
	<16028440.3363847.1237329099031.JavaMail.xicrypt@atgrzls001>
Message-ID: <49C0DF46.4090705@avl.com>

Martin v. L?wis wrote:
> Certainly, the doc string is wrong:
> 
>   isub(a, b) -- Same as a -= b.
> 
> That's not quite the same - you would have to write
> 
>   a = isub(a, b) -- Same as a -= b.

It seems a perfectly fine solution is simply to fix the docstring, 
exactly like that:

a = isub(a, b) -- Same as a -= b.

This both corrects the error when using isub with immutable types and 
educates user as to how a -= b actually works.

From arcriley at gmail.com  Wed Mar 18 21:56:40 2009
From: arcriley at gmail.com (Arc Riley)
Date: Wed, 18 Mar 2009 16:56:40 -0400
Subject: [Python-Dev] Core projects for Summer of Code
Message-ID: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>

Hey guys/gals

Summer of Code is ramping up.  Every year the common complaint is that not
enough Python core projects get proposed by students, and of course a big
reason for that is often the only encouragement we offer prospective
students is a link to the PEP index.

So let's make this year different.

Accepted students are paid a total of $4500 to work for roughly 30 hours a
week, 12 weeks, on their proposed project.

The challenge is finding project ideas for them that could reasonably occupy
them for the entire Summer and which the results of their work can be
demonstrated.  They're being paid for specific projects so "Spend the Summer
fixing bugs on the tracker" is a no-go, and Google has outlined that Summer
of Code is about code, not documentation.

I've seen and heard that a lot of work is still needed on
http://svn.python.org/view/python/trunk both during the 3.1 release cycle,
optimization possible all over the place.  It'd be great if those of you
working closely with this can shout out some ideas, brainstorm a bit.

PSF was announced as one of the mentoring orgs today, this week before
student applications are open is for students to talk to their prospective
mentors and iron out the wrinkles in their plans, so there's not much time
to get core project ideas together.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090318/25d663be/attachment.htm>

From rdmurray at bitdance.com  Wed Mar 18 22:13:27 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 18 Mar 2009 17:13:27 -0400 (EDT)
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>

On Wed, 18 Mar 2009 at 16:56, Arc Riley wrote:
> Summer of Code is ramping up.  Every year the common complaint is that not
> enough Python core projects get proposed by students, and of course a big
> reason for that is often the only encouragement we offer prospective
> students is a link to the PEP index.
>
> So let's make this year different.
>
> Accepted students are paid a total of $4500 to work for roughly 30 hours a
> week, 12 weeks, on their proposed project.
>
> The challenge is finding project ideas for them that could reasonably occupy
> them for the entire Summer and which the results of their work can be
> demonstrated.  They're being paid for specific projects so "Spend the Summer

How about improving 2to3?  Seems like that could be an interesting,
challenging, useful, and rewarding project :).

--
R. David Murray           http://www.bitdance.com

From benjamin at python.org  Wed Mar 18 22:21:46 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 18 Mar 2009 16:21:46 -0500
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
Message-ID: <1afaf6160903181421l5b1fcdd9ka36c30a68e9eabbb@mail.gmail.com>

2009/3/18 Arc Riley <arcriley at gmail.com>:
> Hey guys/gals

Thanks for organizing this!

>
> Summer of Code is ramping up.? Every year the common complaint is that not
> enough Python core projects get proposed by students, and of course a big
> reason for that is often the only encouragement we offer prospective
> students is a link to the PEP index.

Well, there's a list of core projects on the wiki if remember correctly.

>
> I've seen and heard that a lot of work is still needed on
> http://svn.python.org/view/python/trunk both during the 3.1 release cycle,
> optimization possible all over the place.? It'd be great if those of you
> working closely with this can shout out some ideas, brainstorm a bit.

(Just so you know, 3.1 is in http://svn.python.org/view/python/branches/py3k)

3.1 is scheduled to be released in June, so that's probably too early
for projects now. Of course, that doesn't take them out of the running
for 3.2.

One project I can think of off the top of my head is integrating ABCs
further into the interpreter (optimizing and probably rewriting the
abc module in C), so that C code can make use of them.

While working on the core is admirable, I think gsoc would provide an
opportunity to port important Python libraries to 3.x. It's important
to remember that doing ports helps the core immensely by uncovering
2to3 and py3k bugs.




-- 
Regards,
Benjamin

From solipsis at pitrou.net  Wed Mar 18 22:24:25 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 18 Mar 2009 21:24:25 +0000 (UTC)
Subject: [Python-Dev] Core projects for Summer of Code
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
Message-ID: <loom.20090318T212139-468@post.gmane.org>

Hello Arc,

Arc Riley <arcriley <at> gmail.com> writes:
> 
> I've seen and heard that a lot of work is still needed on http://svn.python.org
/view/python/trunk both during the 3.1 release cycle, optimization possible all
over the place.

Well, first, it's too late for 3.1, which will (should) be out before July.
Second, all the eagerly needed optimization work on py3k has now been done.

A quick skim through the bug tracker's "performance" and "feature request"
tickets did not seem to yield anything interesting and big enough for a GSOC
project (although I could have overlooked something).

Rather than performance, I think some more interesting areas would be related to
some of the standard library modules. For instance, the unittest module could
welcome some new features (test discovery, support for skipped tests, probably
others that I'm forgetting about). Since this is pure Python stuff not needing
any deep experience with the interpreter's internals, it would be appropriate
for an outsider.

Regards

Antoine.



From tjreedy at udel.edu  Wed Mar 18 22:29:34 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 18 Mar 2009 17:29:34 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
Message-ID: <gprp3t$icr$2@ger.gmane.org>

R. David Murray wrote:

> How about improving 2to3?  Seems like that could be an interesting,
> challenging, useful, and rewarding project :).

Or the much requested 3to2 using the same tools.


From solipsis at pitrou.net  Wed Mar 18 22:36:27 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 18 Mar 2009 21:36:27 +0000 (UTC)
Subject: [Python-Dev] Core projects for Summer of Code
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org>
Message-ID: <loom.20090318T213554-597@post.gmane.org>

Terry Reedy <tjreedy <at> udel.edu> writes:
> 
> Or the much requested 3to2 using the same tools.

I didn't know there was such a request. I thought it was only a PyPy April fool.



From arcriley at gmail.com  Wed Mar 18 22:40:37 2009
From: arcriley at gmail.com (Arc Riley)
Date: Wed, 18 Mar 2009 17:40:37 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <gprp3t$icr$2@ger.gmane.org>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org>
Message-ID: <bad82a80903181440l79107048jc6ebf34c9c107da0@mail.gmail.com>

I've heard from four people that improving 2to3 would be a great project
(plus many more suggesting "port X to Python 3" as project ideas).

Note the SoC timeline;
http://socghop.appspot.com/document/show/program/google/gsoc2009/timeline

So maybe it won't work for 3.1, but perhaps 3.1.1? 3.2?

We should have many ideas up for students to consider.  The more student who
apply and the more mentors we have ready determines how many students we get
total.

As part of this I should add, we need at least one mentor per student,
preferably two.  These should be people familiar with and actively working
in the area the student would be.  We're putting mentor contact info on the
wiki so potential students can hash out the details with them before
applying.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090318/83e20bef/attachment.htm>

From benjamin at python.org  Wed Mar 18 22:43:44 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 18 Mar 2009 16:43:44 -0500
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bad82a80903181440l79107048jc6ebf34c9c107da0@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org>
	<bad82a80903181440l79107048jc6ebf34c9c107da0@mail.gmail.com>
Message-ID: <1afaf6160903181443k5bc56c1q162ca8b38957395c@mail.gmail.com>

2009/3/18 Arc Riley <arcriley at gmail.com>:
> I've heard from four people that improving 2to3 would be a great project
> (plus many more suggesting "port X to Python 3" as project ideas).

Ok. That's excellent.

>
> Note the SoC timeline;
> http://socghop.appspot.com/document/show/program/google/gsoc2009/timeline
>
> So maybe it won't work for 3.1, but perhaps 3.1.1? 3.2?

Well, there won't be any major changes in 3.1.1, but 3.2 is definitely open.

>
> We should have many ideas up for students to consider.? The more student who
> apply and the more mentors we have ready determines how many students we get
> total.
>
> As part of this I should add, we need at least one mentor per student,
> preferably two.? These should be people familiar with and actively working
> in the area the student would be.? We're putting mentor contact info on the
> wiki so potential students can hash out the details with them before
> applying.




-- 
Regards,
Benjamin

From victor.stinner at haypocalc.com  Wed Mar 18 22:44:47 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 18 Mar 2009 22:44:47 +0100
Subject: [Python-Dev] SoC: Optimize Python3
In-Reply-To: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
Message-ID: <200903182244.47505.victor.stinner@haypocalc.com>

Hi,

There are some very interresting issues about long type optimization:

faster long multiplication
http://bugs.python.org/issue3944

Asymptotically faster divmod and str(long)
http://bugs.python.org/issue3451

Use 30-bit digits instead of 15-bit digits for Python integers.
http://bugs.python.org/issue4258
(mostly done)

But the issues are still open because the code is not easy to review and it's 
already difficult to benchmark (make sure that it's always faster!).

The project can be different than the 3 issues, it should be possible to find 
new ways to optimize operations on integers ;-)

There are other importants features to optimize like:
 - unicode string ("str" in python3)
 - I/O: io-c in py3k branch is already much better, but I'm sure that 
   we can do better ;-)
 - etc.

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From victor.stinner at haypocalc.com  Wed Mar 18 22:47:08 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 18 Mar 2009 22:47:08 +0100
Subject: [Python-Dev] SoC: regex 2.7
In-Reply-To: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
Message-ID: <200903182247.08924.victor.stinner@haypocalc.com>

There are different patches to improve regex library. You can start with:
   http://bugs.python.org/issue2636

I don't know that status of the issues, but it can be an interresting project.

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From victor.stinner at haypocalc.com  Wed Mar 18 22:49:18 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 18 Mar 2009 22:49:18 +0100
Subject: [Python-Dev] SoC: security
In-Reply-To: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
Message-ID: <200903182249.18168.victor.stinner@haypocalc.com>

Tav organized a challenge to prove that its simple idea could be used to write 
a new "restricted python" module. Restart Python security is an huge project, 
but also an interesting project.

http://tav.espians.com/a-challenge-to-break-python-security.html
http://wiki.python.org/moin/Security

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From arcriley at gmail.com  Wed Mar 18 22:50:54 2009
From: arcriley at gmail.com (Arc Riley)
Date: Wed, 18 Mar 2009 17:50:54 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <1afaf6160903181443k5bc56c1q162ca8b38957395c@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org>
	<bad82a80903181440l79107048jc6ebf34c9c107da0@mail.gmail.com>
	<1afaf6160903181443k5bc56c1q162ca8b38957395c@mail.gmail.com>
Message-ID: <bad82a80903181450k5cc5be19p203a71189615d3a4@mail.gmail.com>

>
> > Note the SoC timeline;
> >
> http://socghop.appspot.com/document/show/program/google/gsoc2009/timeline
> >
> > So maybe it won't work for 3.1, but perhaps 3.1.1? 3.2?
>
> Well, there won't be any major changes in 3.1.1, but 3.2 is definitely
> open.


Cool, these are of course details you can work out with interested students.

Would you be willing to field questions from prospective students and
possibly mentor one?

The process is as follows; we're compiling ideas for
http://wiki.python.org/moin/SummerOfCode/2009 and getting mentors signed up
at http://socghop.appspot.com/

Students are already starting to look over the different organizations ideas
pages and connect with mentors.  Student application period opens next
week.  All the mentors for PSF read and review them and we assign mentors to
them (often whatever mentor the student worked with to build the proposal).

Do you want prospective students contacting the list or the mentor they're
interested in working with directly?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090318/bacb81a6/attachment-0001.htm>

From benjamin at python.org  Wed Mar 18 22:56:28 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 18 Mar 2009 16:56:28 -0500
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bad82a80903181450k5cc5be19p203a71189615d3a4@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org>
	<bad82a80903181440l79107048jc6ebf34c9c107da0@mail.gmail.com>
	<1afaf6160903181443k5bc56c1q162ca8b38957395c@mail.gmail.com>
	<bad82a80903181450k5cc5be19p203a71189615d3a4@mail.gmail.com>
Message-ID: <1afaf6160903181456y44c77fa7ha46c0e57ed4c134a@mail.gmail.com>

2009/3/18 Arc Riley <arcriley at gmail.com>:
>> > Note the SoC timeline;
>> >
>> > http://socghop.appspot.com/document/show/program/google/gsoc2009/timeline
>> >
>> > So maybe it won't work for 3.1, but perhaps 3.1.1? 3.2?
>>
>> Well, there won't be any major changes in 3.1.1, but 3.2 is definitely
>> open.
>
> Cool, these are of course details you can work out with interested students.
>
> Would you be willing to field questions from prospective students and
> possibly mentor one?

Certainly

>
> The process is as follows; we're compiling ideas for
> http://wiki.python.org/moin/SummerOfCode/2009 and getting mentors signed up
> at http://socghop.appspot.com/
>
> Students are already starting to look over the different organizations ideas
> pages and connect with mentors.? Student application period opens next
> week.? All the mentors for PSF read and review them and we assign mentors to
> them (often whatever mentor the student worked with to build the proposal).
>
> Do you want prospective students contacting the list or the mentor they're
> interested in working with directly?

IMO, mentors should get direct mail.



-- 
Regards,
Benjamin

From ctb at msu.edu  Wed Mar 18 22:56:40 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Wed, 18 Mar 2009 14:56:40 -0700
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <loom.20090318T212139-468@post.gmane.org>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<loom.20090318T212139-468@post.gmane.org>
Message-ID: <20090318215640.GF28622@idyll.org>

On Wed, Mar 18, 2009 at 09:24:25PM +0000, Antoine Pitrou wrote:
-> Rather than performance, I think some more interesting areas would be related to
-> some of the standard library modules. For instance, the unittest module could
-> welcome some new features (test discovery, support for skipped tests, probably
-> others that I'm forgetting about). Since this is pure Python stuff not needing
-> any deep experience with the interpreter's internals, it would be appropriate
-> for an outsider.

Hi Antoine,

interesting idea -- but I've seen too many arguments about what the
*right* functionality to add to unittest would be to want to give it to
a student.  I think a student would probably not be willing or able to
fight the battles necessary to get his/her changes into the core...

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From fijall at gmail.com  Wed Mar 18 22:59:21 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 18 Mar 2009 22:59:21 +0100
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <1afaf6160903181421l5b1fcdd9ka36c30a68e9eabbb@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<1afaf6160903181421l5b1fcdd9ka36c30a68e9eabbb@mail.gmail.com>
Message-ID: <693bc9ab0903181459i707e3412w968bd78c8e617b50@mail.gmail.com>

>
> While working on the core is admirable, I think gsoc would provide an
> opportunity to port important Python libraries to 3.x. It's important
> to remember that doing ports helps the core immensely by uncovering
> 2to3 and py3k bugs.
>

Hello.

It's a very noble task to have important python libraries ported to
python 3.x. I've played almost whole last year with porting important
python libraries to work on different python interpreters (running the
same version of python). They differ in tiny details, only a bit. And
guess what, it was not only hard, but also very tedious. And now
consider student, who looks for joy and is facing python library (say
medium, couple tens of k lines of code). With help of 2to3 is getting
something that almost works on top of python 3. Except for few small
details. This probably means couple weeks spend on debugging obscure
failures that end up depending on different string representation of
exception or something like that. Assuming he knows python well enough
to understand not only major differences (which are handled  by 2to3
anyway), but also all minor ones. And those tiny which makes you
wonder why unicode subclasses and string subclasses are not exactly
behaving how they're defined in a spec.

Suppose student is smart and likes debugging and it's all working. Now
the question is, who will maintain the resulting library? Will
original team of say twisted maintain it or will it be up to student?
Will it just rot in a corner? Who'll maintain buildbots for that?

I think we need to ask first guys who spend their live maintaining
libraries instead of just proposing "let's make some poor student port
it to py3k", but I might be just wrong, I don't know.

Cheers,
fijal

From brett at python.org  Wed Mar 18 23:07:38 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 18 Mar 2009 15:07:38 -0700
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <gprp3t$icr$2@ger.gmane.org>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org>
Message-ID: <bbaeab100903181507w7d5dc80an4d804b6037f76334@mail.gmail.com>

2009/3/18 Terry Reedy <tjreedy at udel.edu>

> R. David Murray wrote:
>
>  How about improving 2to3?  Seems like that could be an interesting,
>> challenging, useful, and rewarding project :).
>>
>
> Or the much requested 3to2 using the same tools.


I'm not in a position to mentor this, but I too think this would be a great
thing to have.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090318/e94efa32/attachment.htm>

From arcriley at gmail.com  Wed Mar 18 23:12:35 2009
From: arcriley at gmail.com (Arc Riley)
Date: Wed, 18 Mar 2009 18:12:35 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <693bc9ab0903181459i707e3412w968bd78c8e617b50@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<1afaf6160903181421l5b1fcdd9ka36c30a68e9eabbb@mail.gmail.com>
	<693bc9ab0903181459i707e3412w968bd78c8e617b50@mail.gmail.com>
Message-ID: <bad82a80903181512id3d9ebel4f6775a5f1e77873@mail.gmail.com>

On Wed, Mar 18, 2009 at 5:59 PM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> I think we need to ask first guys who spend their live maintaining
> libraries instead of just proposing "let's make some poor student port
> it to py3k", but I might be just wrong, I don't know.


I agree.  Part of Summer of Code is about getting students involved so they
stick around, and heck my second SoC student is still with our project as
the #2 committer, but he's an exception.

Also, we need the projects involved to want the tasks done by a student.  As
a project maintainer I wouldn't want an intern being the most familiar
person with our Py3 migration, I'd rather students stick with new features
or optimization and coordinate the migration process as a group-wide effort.

I added the 2to3 improvement idea to the list, a good start :-)  We need a
couple more at least.

If a 3to2 tool (for backporting Py3 code to Py2, so projects can develop
primarily in Py3?) is something that's wanted, who would be a good mentor
for it?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090318/9cbe4335/attachment.htm>

From brett at python.org  Wed Mar 18 23:13:03 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 18 Mar 2009 15:13:03 -0700
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <1afaf6160903181456y44c77fa7ha46c0e57ed4c134a@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org>
	<bad82a80903181440l79107048jc6ebf34c9c107da0@mail.gmail.com>
	<1afaf6160903181443k5bc56c1q162ca8b38957395c@mail.gmail.com>
	<bad82a80903181450k5cc5be19p203a71189615d3a4@mail.gmail.com>
	<1afaf6160903181456y44c77fa7ha46c0e57ed4c134a@mail.gmail.com>
Message-ID: <bbaeab100903181513m691bfd48pb311019fed8bd8f2@mail.gmail.com>

On Wed, Mar 18, 2009 at 14:56, Benjamin Peterson <benjamin at python.org>wrote:

> 2009/3/18 Arc Riley <arcriley at gmail.com>:
> >> > Note the SoC timeline;
> >> >
> >> >
> http://socghop.appspot.com/document/show/program/google/gsoc2009/timeline
> >> >
> >> > So maybe it won't work for 3.1, but perhaps 3.1.1? 3.2?
> >>
> >> Well, there won't be any major changes in 3.1.1, but 3.2 is definitely
> >> open.
> >
> > Cool, these are of course details you can work out with interested
> students.
> >
> > Would you be willing to field questions from prospective students and
> > possibly mentor one?
>
> Certainly



I would double-check Benjamin can do this since I don't think he will be 18
by the time GSoC starts. The FAQ at
http://socghop.appspot.com/document/show/program/google/gsoc2009/faqs#mentor_eligibilityseems
to suggest it won't be an issue, but you never know.

Also be aware that a university student might not like being told what to do
by someone in high school (although if they want $4500 they better).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090318/ae1be6b1/attachment-0001.htm>

From brett at python.org  Wed Mar 18 23:17:48 2009
From: brett at python.org (Brett Cannon)
Date: Wed, 18 Mar 2009 15:17:48 -0700
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bad82a80903181512id3d9ebel4f6775a5f1e77873@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<1afaf6160903181421l5b1fcdd9ka36c30a68e9eabbb@mail.gmail.com>
	<693bc9ab0903181459i707e3412w968bd78c8e617b50@mail.gmail.com>
	<bad82a80903181512id3d9ebel4f6775a5f1e77873@mail.gmail.com>
Message-ID: <bbaeab100903181517n496ecf10x81c0e30f098c94fa@mail.gmail.com>

2009/3/18 Arc Riley <arcriley at gmail.com>

> On Wed, Mar 18, 2009 at 5:59 PM, Maciej Fijalkowski <fijall at gmail.com>wrote:
>
>> I think we need to ask first guys who spend their live maintaining
>> libraries instead of just proposing "let's make some poor student port
>> it to py3k", but I might be just wrong, I don't know.
>
>
> I agree.  Part of Summer of Code is about getting students involved so they
> stick around, and heck my second SoC student is still with our project as
> the #2 committer, but he's an exception.
>
> Also, we need the projects involved to want the tasks done by a student.
> As a project maintainer I wouldn't want an intern being the most familiar
> person with our Py3 migration, I'd rather students stick with new features
> or optimization and coordinate the migration process as a group-wide effort.
>
> I added the 2to3 improvement idea to the list, a good start :-)  We need a
> couple more at least.
>
> If a 3to2 tool (for backporting Py3 code to Py2, so projects can develop
> primarily in Py3?)


Exactly. The semantics are cleaner in 3.x, suggesting it would be easier to
backport it to 2.x.


> is something that's wanted, who would be a good mentor for it?


Benjamin has done the most work on 2to3 recently. Thomas Wouters originally
came up with the idea for 3to2 but I suspect he doesn't want to mentor.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090318/4a4b6e7d/attachment.htm>

From arcriley at gmail.com  Wed Mar 18 23:22:17 2009
From: arcriley at gmail.com (Arc Riley)
Date: Wed, 18 Mar 2009 18:22:17 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bbaeab100903181513m691bfd48pb311019fed8bd8f2@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org>
	<bad82a80903181440l79107048jc6ebf34c9c107da0@mail.gmail.com>
	<1afaf6160903181443k5bc56c1q162ca8b38957395c@mail.gmail.com>
	<bad82a80903181450k5cc5be19p203a71189615d3a4@mail.gmail.com>
	<1afaf6160903181456y44c77fa7ha46c0e57ed4c134a@mail.gmail.com>
	<bbaeab100903181513m691bfd48pb311019fed8bd8f2@mail.gmail.com>
Message-ID: <bad82a80903181522m19d5920la136018049d7adb2@mail.gmail.com>

On Wed, Mar 18, 2009 at 6:13 PM, Brett Cannon <brett at python.org> wrote:

>
> I would double-check Benjamin can do this since I don't think he will be
> 18 by the time GSoC starts. The FAQ at
> http://socghop.appspot.com/document/show/program/google/gsoc2009/faqs#mentor_eligibilityseems to suggest it won't be an issue, but you never know.
>
> Also be aware that a university student might not like being told what to
> do by someone in high school (although if they want $4500 they better).
>

This isn't a problem.  Drupal had a GHOP (SoC for highschool students)
mentor who was too young to be a student (11 or 12 at the time), I believe
Dmitri also served them as a SoC mentor, I know several SoC mentors are
teenagers.

I'd rather we find a different primary mentor for each student, it'd be best
if every student had a backup mentor in-place from the start as well, but
one person can be a contact point for many ideas.

Ben, would you be OK with being a contact point for 3to2 as well?  We'll
figure out who's mentoring who once we see student apps and decide which
ones we want mentored this year.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090318/3f350a61/attachment.htm>

From fetchinson at googlemail.com  Wed Mar 18 23:23:11 2009
From: fetchinson at googlemail.com (Daniel Fetchinson)
Date: Wed, 18 Mar 2009 15:23:11 -0700
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
Message-ID: <fbe2e2100903181523m25adb0a0g2ac79d8e1cd7977c@mail.gmail.com>

> Hey guys/gals
>
> Summer of Code is ramping up.  Every year the common complaint is that not
> enough Python core projects get proposed by students, and of course a big
> reason for that is often the only encouragement we offer prospective
> students is a link to the PEP index.
>
> So let's make this year different.
>
> Accepted students are paid a total of $4500 to work for roughly 30 hours a
> week, 12 weeks, on their proposed project.
>
> The challenge is finding project ideas for them that could reasonably occupy
> them for the entire Summer and which the results of their work can be
> demonstrated.  They're being paid for specific projects so "Spend the Summer
> fixing bugs on the tracker" is a no-go, and Google has outlined that Summer
> of Code is about code, not documentation.
>
> I've seen and heard that a lot of work is still needed on
> http://svn.python.org/view/python/trunk both during the 3.1 release cycle,
> optimization possible all over the place.  It'd be great if those of you
> working closely with this can shout out some ideas, brainstorm a bit.
>
> PSF was announced as one of the mentoring orgs today, this week before
> student applications are open is for students to talk to their prospective
> mentors and iron out the wrinkles in their plans, so there's not much time
> to get core project ideas together.

How about porting PIL to 3.0?
There were many such requests on python-list and image-sig (including mine :))

Cheers,
Daniel


-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown

From python at rcn.com  Wed Mar 18 23:26:56 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 18 Mar 2009 15:26:56 -0700
Subject: [Python-Dev] Core projects for Summer of Code
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com><1afaf6160903181421l5b1fcdd9ka36c30a68e9eabbb@mail.gmail.com><693bc9ab0903181459i707e3412w968bd78c8e617b50@mail.gmail.com>
	<bad82a80903181512id3d9ebel4f6775a5f1e77873@mail.gmail.com>
Message-ID: <BAF8EF1295F44064855B6EC7A83F8B15@RaymondLaptop1>


>   As a project maintainer I wouldn't want an intern being the most familiar 
>  person with our Py3 migration, I'd rather students stick with new features 
> or optimization and coordinate the migration process as a group-wide effort.

Without help, it is going to take a long time to get many packages converted to 3.x.
I think the students can be invaluable in this process.  Ideally, they will tweak
the 2.6 code until it converts cleanly using 2-to-3.  That sort of work will be
easy to maintain.

IMO, this is the most important thing that can be done for Python at the moment.
I would much rather this sort of work than having a student build a new library
module and then not be around to maintain it.


Raymond

From arcriley at gmail.com  Wed Mar 18 23:37:10 2009
From: arcriley at gmail.com (Arc Riley)
Date: Wed, 18 Mar 2009 18:37:10 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <BAF8EF1295F44064855B6EC7A83F8B15@RaymondLaptop1>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<1afaf6160903181421l5b1fcdd9ka36c30a68e9eabbb@mail.gmail.com>
	<693bc9ab0903181459i707e3412w968bd78c8e617b50@mail.gmail.com>
	<bad82a80903181512id3d9ebel4f6775a5f1e77873@mail.gmail.com>
	<BAF8EF1295F44064855B6EC7A83F8B15@RaymondLaptop1>
Message-ID: <bad82a80903181537y3804623ax415265ab4ffa5664@mail.gmail.com>

On Wed, Mar 18, 2009 at 6:26 PM, Raymond Hettinger <python at rcn.com> wrote:

> Without help, it is going to take a long time to get many packages
> converted to 3.x.


I don't disagree, I just don't want to volunteer projects for something they
don't want.

Unless I misunderstand the situation, PIL doesn't seem applicable for SoC
given that their development tree is closed/proprietary (only free releases
are available under a free license).
Does anyone here work with PIL or can provide further insight into their Py3
plans?

Note also that some of the largest Python-based projects, Django, Mercurial,
Plone/Zope, Scons, etc, are setup as their own SoC mentoring orgs.  Only
Mercurial has Py3 migration on their ideas list.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090318/e38ec7fe/attachment-0001.htm>

From ggpolo at gmail.com  Wed Mar 18 23:43:07 2009
From: ggpolo at gmail.com (Guilherme Polo)
Date: Wed, 18 Mar 2009 19:43:07 -0300
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <fbe2e2100903181523m25adb0a0g2ac79d8e1cd7977c@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<fbe2e2100903181523m25adb0a0g2ac79d8e1cd7977c@mail.gmail.com>
Message-ID: <ac2200130903181543r6a00af10s2c558b5d46611c9c@mail.gmail.com>

On Wed, Mar 18, 2009 at 7:23 PM, Daniel Fetchinson
<fetchinson at googlemail.com> wrote:
>> Hey guys/gals
>>
>> Summer of Code is ramping up. ?Every year the common complaint is that not
>> enough Python core projects get proposed by students, and of course a big
>> reason for that is often the only encouragement we offer prospective
>> students is a link to the PEP index.
>>
>> So let's make this year different.
>>
>> Accepted students are paid a total of $4500 to work for roughly 30 hours a
>> week, 12 weeks, on their proposed project.
>>
>> The challenge is finding project ideas for them that could reasonably occupy
>> them for the entire Summer and which the results of their work can be
>> demonstrated. ?They're being paid for specific projects so "Spend the Summer
>> fixing bugs on the tracker" is a no-go, and Google has outlined that Summer
>> of Code is about code, not documentation.
>>
>> I've seen and heard that a lot of work is still needed on
>> http://svn.python.org/view/python/trunk both during the 3.1 release cycle,
>> optimization possible all over the place. ?It'd be great if those of you
>> working closely with this can shout out some ideas, brainstorm a bit.
>>
>> PSF was announced as one of the mentoring orgs today, this week before
>> student applications are open is for students to talk to their prospective
>> mentors and iron out the wrinkles in their plans, so there's not much time
>> to get core project ideas together.
>
> How about porting PIL to 3.0?
> There were many such requests on python-list and image-sig (including mine :))
>

I have ported it to the stage where its tests passes (which are far
from covering all the code) and some of my own tests, there is a git
repo on the image-sig that points to it. I wasn't really careful with
some of the things (and I would even consider redoing some of them),
but only one or two people got a copy of it so apparently people don't
want/need it on python 3.0 just yet (not it alone at least).

> Cheers,
> Daniel
>
>
> --
> Psss, psss, put it down! - http://www.cafepress.com/putitdown
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com
>



-- 
-- Guilherme H. Polo Goncalves

From ncoghlan at gmail.com  Wed Mar 18 23:43:50 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 19 Mar 2009 08:43:50 +1000
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <20090318215640.GF28622@idyll.org>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<loom.20090318T212139-468@post.gmane.org>
	<20090318215640.GF28622@idyll.org>
Message-ID: <49C17926.8090501@gmail.com>

C. Titus Brown wrote:
> On Wed, Mar 18, 2009 at 09:24:25PM +0000, Antoine Pitrou wrote:
> -> Rather than performance, I think some more interesting areas would be related to
> -> some of the standard library modules. For instance, the unittest module could
> -> welcome some new features (test discovery, support for skipped tests, probably
> -> others that I'm forgetting about). Since this is pure Python stuff not needing
> -> any deep experience with the interpreter's internals, it would be appropriate
> -> for an outsider.
> 
> Hi Antoine,
> 
> interesting idea -- but I've seen too many arguments about what the
> *right* functionality to add to unittest would be to want to give it to
> a student.  I think a student would probably not be willing or able to
> fight the battles necessary to get his/her changes into the core...

Yeah, that was my thought on that specific example as well - since there
isn't anyone to say "we're doing it this way" for the unittest module
the way we have with the language itself (i.e. Guido) or with other
modules (e.g. Raymond for itertools, Vinay for logging), unittest
improvement discussions tend to get stuck on the details :P

If an existing module has a strong owner that can say yes/no to large
changes then work in that area can be a good student project (especially
if the owner can serve as the mentor). But modules with only collective
python-dev responsibility can be tricky to work on more from the social
angle rather than the technical one.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From benjamin at python.org  Thu Mar 19 00:13:43 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 18 Mar 2009 18:13:43 -0500
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bad82a80903181522m19d5920la136018049d7adb2@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org>
	<bad82a80903181440l79107048jc6ebf34c9c107da0@mail.gmail.com>
	<1afaf6160903181443k5bc56c1q162ca8b38957395c@mail.gmail.com>
	<bad82a80903181450k5cc5be19p203a71189615d3a4@mail.gmail.com>
	<1afaf6160903181456y44c77fa7ha46c0e57ed4c134a@mail.gmail.com>
	<bbaeab100903181513m691bfd48pb311019fed8bd8f2@mail.gmail.com>
	<bad82a80903181522m19d5920la136018049d7adb2@mail.gmail.com>
Message-ID: <1afaf6160903181613m1ff13cbh69a19ca071a94e6f@mail.gmail.com>

2009/3/18 Arc Riley <arcriley at gmail.com>:
> On Wed, Mar 18, 2009 at 6:13 PM, Brett Cannon <brett at python.org> wrote:
>>
>> I would double-check Benjamin can do this since I don't think he will be
>> 18 by the time GSoC starts. The FAQ at
>> http://socghop.appspot.com/document/show/program/google/gsoc2009/faqs#mentor_eligibility
>> seems to suggest it won't be an issue, but you never know.
>>
>> Also be aware that a university student might not like being told what to
>> do by someone in high school (although if they want $4500 they better).
>
> This isn't a problem.? Drupal had a GHOP (SoC for highschool students)
> mentor who was too young to be a student (11 or 12 at the time), I believe
> Dmitri also served them as a SoC mentor, I know several SoC mentors are
> teenagers.

Excellent!

>
> I'd rather we find a different primary mentor for each student, it'd be best
> if every student had a backup mentor in-place from the start as well, but
> one person can be a contact point for many ideas.
>
> Ben, would you be OK with being a contact point for 3to2 as well?? We'll
> figure out who's mentoring who once we see student apps and decide which
> ones we want mentored this year.

Sure.



-- 
Regards,
Benjamin

From g.brandl at gmx.net  Thu Mar 19 00:18:51 2009
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 19 Mar 2009 00:18:51 +0100
Subject: [Python-Dev] What level of detail wanted for import and the
	language reference?
In-Reply-To: <1afaf6160903161559w4fe82af2ta4e1f4ad1762d603@mail.gmail.com>
References: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
	<1afaf6160903161559w4fe82af2ta4e1f4ad1762d603@mail.gmail.com>
Message-ID: <gprvic$6vh$1@ger.gmane.org>

Benjamin Peterson schrieb:
> 2009/3/16 Brett Cannon <brett at python.org>:
>> At this point importlib is done for its public API for Python 3.1. That
>> means it's time to turn my attention to making sure the semantics of import
>> are well documented. But where to put all of the details? The language
>> reference for import
>> (http://docs.python.org/dev/py3k/reference/simple_stmts.html#the-import-statement)
>> explains the basics, but is lacking all of the details of PEP 302 and other
>> stuff like __path__ that have existed for ages.
>>
>> My question is if I should flesh out the details in the language reference
>> or do it in importlib's intro docs. The main reason I could see not doing it
>> in the langauge reference (or at least duplicating it) is it would be
>> somewhat easier to reference specific objects in importlib but I am not sure
>> if the language reference should try to stay away from stdlib references.
> 
> Thanks so much for doing this! Personally, I think you should put it
> in the language reference. (I think it deserves it's own file if it's
> as big as I suspect it will be.)

If you asked me, I'd concur with Benjamin.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From tjreedy at udel.edu  Thu Mar 19 00:53:59 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 18 Mar 2009 19:53:59 -0400
Subject: [Python-Dev] Non-Core project: IDLE
In-Reply-To: <gprp3t$icr$2@ger.gmane.org>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org>
Message-ID: <gps1in$e1i$1@ger.gmane.org>

IDLE needs lots of attention -- more than any one experienced person is 
likely to have


From arcriley at gmail.com  Thu Mar 19 00:56:12 2009
From: arcriley at gmail.com (Arc Riley)
Date: Wed, 18 Mar 2009 19:56:12 -0400
Subject: [Python-Dev] Non-Core project: IDLE
In-Reply-To: <gps1in$e1i$1@ger.gmane.org>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org> <gps1in$e1i$1@ger.gmane.org>
Message-ID: <bad82a80903181656p422c7e91j9779976ce945bc46@mail.gmail.com>

On Wed, Mar 18, 2009 at 7:53 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> IDLE needs lots of attention -- more than any one experienced person is
> likely to have
>
>
I've actually heard this from several people, IDLE on Py3 etc

Who would be a good person to mentor such a project?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090318/bb80f9ba/attachment-0001.htm>

From steve at holdenweb.com  Thu Mar 19 01:04:08 2009
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 18 Mar 2009 20:04:08 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bad82a80903181512id3d9ebel4f6775a5f1e77873@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<1afaf6160903181421l5b1fcdd9ka36c30a68e9eabbb@mail.gmail.com>	<693bc9ab0903181459i707e3412w968bd78c8e617b50@mail.gmail.com>
	<bad82a80903181512id3d9ebel4f6775a5f1e77873@mail.gmail.com>
Message-ID: <49C18BF8.3010503@holdenweb.com>

Arc Riley wrote:
> On Wed, Mar 18, 2009 at 5:59 PM, Maciej Fijalkowski <fijall at gmail.com
> <mailto:fijall at gmail.com>> wrote:
> 
>     I think we need to ask first guys who spend their live maintaining
>     libraries instead of just proposing "let's make some poor student port
>     it to py3k", but I might be just wrong, I don't know.
> 
> 
> I agree.  Part of Summer of Code is about getting students involved so
> they stick around, and heck my second SoC student is still with our
> project as the #2 committer, but he's an exception.
> 
> Also, we need the projects involved to want the tasks done by a
> student.  As a project maintainer I wouldn't want an intern being the
> most familiar person with our Py3 migration, I'd rather students stick
> with new features or optimization and coordinate the migration process
> as a group-wide effort.
> 
> I added the 2to3 improvement idea to the list, a good start :-)  We need
> a couple more at least.
> 
> If a 3to2 tool (for backporting Py3 code to Py2, so projects can develop
> primarily in Py3?) is something that's wanted, who would be a good
> mentor for it?
> 
We also need projects for people who may want to do some coding and then
just walk away - the SoC experience might teach them that programming
isn't for them ;-)

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From tjreedy at udel.edu  Thu Mar 19 01:04:02 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 18 Mar 2009 20:04:02 -0400
Subject: [Python-Dev] Core projects: 3to2
In-Reply-To: <loom.20090318T213554-597@post.gmane.org>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>	<gprp3t$icr$2@ger.gmane.org>
	<loom.20090318T213554-597@post.gmane.org>
Message-ID: <gps25i$fjs$1@ger.gmane.org>

Antoine Pitrou wrote:
> Terry Reedy <tjreedy <at> udel.edu> writes:
>> Or the much requested 3to2 using the same tools.
> 
> I didn't know there was such a request. I thought it was only a PyPy April fool.

Some of the people who need to support both late 2.x and 3.x would 
prefer to write 3.x code and backport.  The OP of a current python-list 
thread asked whether there was any way to write something like

	@alias('__nonzero__')
	def __bool__(self): return True

(in preference to explicit 'if version....') and have .__bool__ be 
either replaced or aliased as .__nonzero__ for a 2.x version.  Answer: 
No.  Use 3to2 if/when available.

This has been the answer in other threads as well.

I believe my own 3.0 code will mainly also need
print() to print statement
except e as a to 2.x version
class C() to class C(object)

An easily doable project would be both used and appreciated.

tjr


From steve at holdenweb.com  Thu Mar 19 01:04:08 2009
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 18 Mar 2009 20:04:08 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bad82a80903181512id3d9ebel4f6775a5f1e77873@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<1afaf6160903181421l5b1fcdd9ka36c30a68e9eabbb@mail.gmail.com>	<693bc9ab0903181459i707e3412w968bd78c8e617b50@mail.gmail.com>
	<bad82a80903181512id3d9ebel4f6775a5f1e77873@mail.gmail.com>
Message-ID: <49C18BF8.3010503@holdenweb.com>

Arc Riley wrote:
> On Wed, Mar 18, 2009 at 5:59 PM, Maciej Fijalkowski <fijall at gmail.com
> <mailto:fijall at gmail.com>> wrote:
> 
>     I think we need to ask first guys who spend their live maintaining
>     libraries instead of just proposing "let's make some poor student port
>     it to py3k", but I might be just wrong, I don't know.
> 
> 
> I agree.  Part of Summer of Code is about getting students involved so
> they stick around, and heck my second SoC student is still with our
> project as the #2 committer, but he's an exception.
> 
> Also, we need the projects involved to want the tasks done by a
> student.  As a project maintainer I wouldn't want an intern being the
> most familiar person with our Py3 migration, I'd rather students stick
> with new features or optimization and coordinate the migration process
> as a group-wide effort.
> 
> I added the 2to3 improvement idea to the list, a good start :-)  We need
> a couple more at least.
> 
> If a 3to2 tool (for backporting Py3 code to Py2, so projects can develop
> primarily in Py3?) is something that's wanted, who would be a good
> mentor for it?
> 
We also need projects for people who may want to do some coding and then
just walk away - the SoC experience might teach them that programming
isn't for them ;-)

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Thu Mar 19 01:11:46 2009
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 18 Mar 2009 20:11:46 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bad82a80903181440l79107048jc6ebf34c9c107da0@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>	<gprp3t$icr$2@ger.gmane.org>
	<bad82a80903181440l79107048jc6ebf34c9c107da0@mail.gmail.com>
Message-ID: <gps2k2$e6i$2@ger.gmane.org>

Arc Riley wrote:
> I've heard from four people that improving 2to3 would be a great project
> (plus many more suggesting "port X to Python 3" as project ideas).
> 
> Note the SoC timeline;
> http://socghop.appspot.com/document/show/program/google/gsoc2009/timeline
> 
> So maybe it won't work for 3.1, but perhaps 3.1.1? 3.2?
> 
Let's not forget that we could (fairly easily?) make an interim release
of a 3.1-based 2to3 before adding in anything 3.2-specific, so there's
no reason why a SoC project shouldn't work, with a release after the
project (perhaps as a patch to 3.1 installations) and hence over a year
before 3.2. Or perhaps a 3.1.1 release could incorporate the more
advanced 2to3 features with the same 3.1 language ...

> We should have many ideas up for students to consider.  The more student
> who apply and the more mentors we have ready determines how many
> students we get total.
> 
> As part of this I should add, we need at least one mentor per student,
> preferably two.  These should be people familiar with and actively
> working in the area the student would be.  We're putting mentor contact
> info on the wiki so potential students can hash out the details with
> them before applying.
> 
Realistically I am not sure I will have time to mentor, but if you need
any help from the PSF please feel free to get in touch. Thanks for
taking this challenging role up on behalf of the Python community.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From solipsis at pitrou.net  Thu Mar 19 01:15:02 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 19 Mar 2009 00:15:02 +0000 (UTC)
Subject: [Python-Dev] Core projects: 3to2
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>	<gprp3t$icr$2@ger.gmane.org>
	<loom.20090318T213554-597@post.gmane.org>
	<gps25i$fjs$1@ger.gmane.org>
Message-ID: <loom.20090319T001035-310@post.gmane.org>

Terry Reedy <tjreedy <at> udel.edu> writes:
> 
> Some of the people who need to support both late 2.x and 3.x would 
> prefer to write 3.x code and backport.  The OP of a current python-list 
> thread asked whether there was any way to write something like
> 
> 	@alias('__nonzero__')
> 	def __bool__(self): return True

How about simply:
    __nonzero__ = __bool__

> I believe my own 3.0 code will mainly also need
> print() to print statement

If this is only about supporting "late 2.x" (i.e., 2.6 and upwards), you can
already write:

from __future__ import print_function

> except e as a to 2.x version

Works in 2.6.

> class C() to class C(object)

__metaclass__ = type

Now I'm not saying that all 3.0 code will work in 2.6 with such simple
precautions, far from it!

Regards

Antoine.



From ajaksu at gmail.com  Thu Mar 19 01:16:13 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Wed, 18 Mar 2009 21:16:13 -0300
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bad82a80903181450k5cc5be19p203a71189615d3a4@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org>
	<bad82a80903181440l79107048jc6ebf34c9c107da0@mail.gmail.com>
	<1afaf6160903181443k5bc56c1q162ca8b38957395c@mail.gmail.com>
	<bad82a80903181450k5cc5be19p203a71189615d3a4@mail.gmail.com>
Message-ID: <2d75d7660903181716l282e9da0y53a7fa098a6e0131@mail.gmail.com>

Arc Riley wrote:
> The process is as follows; we're compiling ideas for
> http://wiki.python.org/moin/SummerOfCode/2009 and getting mentors signed up
> at http://socghop.appspot.com/

Any chance that we can keep
http://wiki.python.org/moin/SummerOfCode/2009 light on markup? I
simply can't add a 'tidy struct and finish buffer interface/bytearray
details' proposal as it is :/

Daniel

From arcriley at gmail.com  Thu Mar 19 01:23:02 2009
From: arcriley at gmail.com (Arc Riley)
Date: Wed, 18 Mar 2009 20:23:02 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <2d75d7660903181716l282e9da0y53a7fa098a6e0131@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org>
	<bad82a80903181440l79107048jc6ebf34c9c107da0@mail.gmail.com>
	<1afaf6160903181443k5bc56c1q162ca8b38957395c@mail.gmail.com>
	<bad82a80903181450k5cc5be19p203a71189615d3a4@mail.gmail.com>
	<2d75d7660903181716l282e9da0y53a7fa098a6e0131@mail.gmail.com>
Message-ID: <bad82a80903181723g7bda7846vb0b8fa85c472be5b@mail.gmail.com>

Feel free to email willg at bluesock.org or me with your ideas if the markup is
difficult to work with.  We've been on wiki duty all afternoon.

description, any specific skills they'll need (special library, compiler
theory, etc), what mentor should they talk to if they're interested.

The markup complexity makes it much easier for students to navigate while
including the info Google suggested.  I'm hoping someone from the python web
team will add the CSS class we need to greatly simplify it.

On Wed, Mar 18, 2009 at 8:16 PM, Daniel (ajax) Diniz <ajaksu at gmail.com>wrote:

> Arc Riley wrote:
> > The process is as follows; we're compiling ideas for
> > http://wiki.python.org/moin/SummerOfCode/2009 and getting mentors signed
> up
> > at http://socghop.appspot.com/
>
> Any chance that we can keep
> http://wiki.python.org/moin/SummerOfCode/2009 light on markup? I
> simply can't add a 'tidy struct and finish buffer interface/bytearray
> details' proposal as it is :/
>
> Daniel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090318/46d1916a/attachment-0001.htm>

From benjamin at python.org  Thu Mar 19 01:35:16 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 18 Mar 2009 19:35:16 -0500
Subject: [Python-Dev] Core projects: 3to2
In-Reply-To: <loom.20090319T001035-310@post.gmane.org>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org> <loom.20090318T213554-597@post.gmane.org>
	<gps25i$fjs$1@ger.gmane.org> <loom.20090319T001035-310@post.gmane.org>
Message-ID: <1afaf6160903181735u2b63dc3tb3c2a9d2e2195eba@mail.gmail.com>

2009/3/18 Antoine Pitrou <solipsis at pitrou.net>:
>
>> class C() to class C(object)
>
> __metaclass__ = type

Or even better: just inherit from object in 3.0 and 2.x. :)



-- 
Regards,
Benjamin

From fetchinson at googlemail.com  Thu Mar 19 01:44:28 2009
From: fetchinson at googlemail.com (Daniel Fetchinson)
Date: Wed, 18 Mar 2009 17:44:28 -0700
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <ac2200130903181543r6a00af10s2c558b5d46611c9c@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<fbe2e2100903181523m25adb0a0g2ac79d8e1cd7977c@mail.gmail.com>
	<ac2200130903181543r6a00af10s2c558b5d46611c9c@mail.gmail.com>
Message-ID: <fbe2e2100903181744x55d1b8e5s39a419c4b74854fc@mail.gmail.com>

>>> Summer of Code is ramping up.  Every year the common complaint is that
>>> not
>>> enough Python core projects get proposed by students, and of course a big
>>> reason for that is often the only encouragement we offer prospective
>>> students is a link to the PEP index.
>>>
>>> So let's make this year different.
>>>
>>> Accepted students are paid a total of $4500 to work for roughly 30 hours
>>> a
>>> week, 12 weeks, on their proposed project.
>>>
>>> The challenge is finding project ideas for them that could reasonably
>>> occupy
>>> them for the entire Summer and which the results of their work can be
>>> demonstrated.  They're being paid for specific projects so "Spend the
>>> Summer
>>> fixing bugs on the tracker" is a no-go, and Google has outlined that
>>> Summer
>>> of Code is about code, not documentation.
>>>
>>> I've seen and heard that a lot of work is still needed on
>>> http://svn.python.org/view/python/trunk both during the 3.1 release
>>> cycle,
>>> optimization possible all over the place.  It'd be great if those of you
>>> working closely with this can shout out some ideas, brainstorm a bit.
>>>
>>> PSF was announced as one of the mentoring orgs today, this week before
>>> student applications are open is for students to talk to their
>>> prospective
>>> mentors and iron out the wrinkles in their plans, so there's not much
>>> time
>>> to get core project ideas together.
>>
>> How about porting PIL to 3.0?
>> There were many such requests on python-list and image-sig (including mine
>> :))
>>
>
> I have ported it to the stage where its tests passes (which are far
> from covering all the code) and some of my own tests, there is a git
> repo on the image-sig that points to it. I wasn't really careful with
> some of the things (and I would even consider redoing some of them),
> but only one or two people got a copy of it so apparently people don't
> want/need it on python 3.0 just yet (not it alone at least).

I did a "git clone git://gpolo.ath.cx/pil-py3k.git" but it failed:

gpolo.ath.cx[0: 189.7.18.241]: errno=Connection timed out
fatal: unable to connect a socket (Connection timed out)
fetch-pack from 'git://gpolo.ath.cx/pil-py3k.git' failed.

By the way the reason I think few people checked it out is that people
mostly are waiting for an "official" PIL release that is known to be
stable. Did you try making your port part of the "official" PIL
distribution?

Cheers,
Daniel


-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown

From ggpolo at gmail.com  Thu Mar 19 01:55:06 2009
From: ggpolo at gmail.com (Guilherme Polo)
Date: Wed, 18 Mar 2009 21:55:06 -0300
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <fbe2e2100903181744x55d1b8e5s39a419c4b74854fc@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<fbe2e2100903181523m25adb0a0g2ac79d8e1cd7977c@mail.gmail.com>
	<ac2200130903181543r6a00af10s2c558b5d46611c9c@mail.gmail.com>
	<fbe2e2100903181744x55d1b8e5s39a419c4b74854fc@mail.gmail.com>
Message-ID: <ac2200130903181755v2b720698ufec105892ab81dfa@mail.gmail.com>

On Wed, Mar 18, 2009 at 9:44 PM, Daniel Fetchinson
<fetchinson at googlemail.com> wrote:
>>>> Summer of Code is ramping up. ?Every year the common complaint is that
>>>> not
>>>> enough Python core projects get proposed by students, and of course a big
>>>> reason for that is often the only encouragement we offer prospective
>>>> students is a link to the PEP index.
>>>>
>>>> So let's make this year different.
>>>>
>>>> Accepted students are paid a total of $4500 to work for roughly 30 hours
>>>> a
>>>> week, 12 weeks, on their proposed project.
>>>>
>>>> The challenge is finding project ideas for them that could reasonably
>>>> occupy
>>>> them for the entire Summer and which the results of their work can be
>>>> demonstrated. ?They're being paid for specific projects so "Spend the
>>>> Summer
>>>> fixing bugs on the tracker" is a no-go, and Google has outlined that
>>>> Summer
>>>> of Code is about code, not documentation.
>>>>
>>>> I've seen and heard that a lot of work is still needed on
>>>> http://svn.python.org/view/python/trunk both during the 3.1 release
>>>> cycle,
>>>> optimization possible all over the place. ?It'd be great if those of you
>>>> working closely with this can shout out some ideas, brainstorm a bit.
>>>>
>>>> PSF was announced as one of the mentoring orgs today, this week before
>>>> student applications are open is for students to talk to their
>>>> prospective
>>>> mentors and iron out the wrinkles in their plans, so there's not much
>>>> time
>>>> to get core project ideas together.
>>>
>>> How about porting PIL to 3.0?
>>> There were many such requests on python-list and image-sig (including mine
>>> :))
>>>
>>
>> I have ported it to the stage where its tests passes (which are far
>> from covering all the code) and some of my own tests, there is a git
>> repo on the image-sig that points to it. I wasn't really careful with
>> some of the things (and I would even consider redoing some of them),
>> but only one or two people got a copy of it so apparently people don't
>> want/need it on python 3.0 just yet (not it alone at least).
>
> I did a "git clone git://gpolo.ath.cx/pil-py3k.git" but it failed:
>
> gpolo.ath.cx[0: 189.7.18.241]: errno=Connection timed out
> fatal: unable to connect a socket (Connection timed out)
> fetch-pack from 'git://gpolo.ath.cx/pil-py3k.git' failed.
>

Thanks for noticing that, maybe more people had this same problem
then, I will consider using github or some similar service (or maybe
take the chance to bazaar, or mercurial, or svn, or..).

> By the way the reason I think few people checked it out is that people
> mostly are waiting for an "official" PIL release that is known to be
> stable. Did you try making your port part of the "official" PIL
> distribution?
>

I have talked with Fredrik, he said he would be running it on another
test suite to check how much of it really works. But, no, I didn't
really try pushing it to be integrated into the next PIL release and
it also wouldn't be possible without distributing a py3k version only
-- I didn't do the port with the ability to work in python 3.x and
python 2.x but this can be arranged.

> Cheers,
> Daniel
>
>
> --
> Psss, psss, put it down! - http://www.cafepress.com/putitdown

Regards,

-- 
-- Guilherme H. Polo Goncalves

From fetchinson at googlemail.com  Thu Mar 19 02:11:51 2009
From: fetchinson at googlemail.com (Daniel Fetchinson)
Date: Wed, 18 Mar 2009 18:11:51 -0700
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <ac2200130903181755v2b720698ufec105892ab81dfa@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<fbe2e2100903181523m25adb0a0g2ac79d8e1cd7977c@mail.gmail.com>
	<ac2200130903181543r6a00af10s2c558b5d46611c9c@mail.gmail.com>
	<fbe2e2100903181744x55d1b8e5s39a419c4b74854fc@mail.gmail.com>
	<ac2200130903181755v2b720698ufec105892ab81dfa@mail.gmail.com>
Message-ID: <fbe2e2100903181811v737a0cf6k9f8617ef9fbd1634@mail.gmail.com>

>>>>> Summer of Code is ramping up.  Every year the common complaint is that
>>>>> not
>>>>> enough Python core projects get proposed by students, and of course a
>>>>> big
>>>>> reason for that is often the only encouragement we offer prospective
>>>>> students is a link to the PEP index.
>>>>>
>>>>> So let's make this year different.
>>>>>
>>>>> Accepted students are paid a total of $4500 to work for roughly 30
>>>>> hours
>>>>> a
>>>>> week, 12 weeks, on their proposed project.
>>>>>
>>>>> The challenge is finding project ideas for them that could reasonably
>>>>> occupy
>>>>> them for the entire Summer and which the results of their work can be
>>>>> demonstrated.  They're being paid for specific projects so "Spend the
>>>>> Summer
>>>>> fixing bugs on the tracker" is a no-go, and Google has outlined that
>>>>> Summer
>>>>> of Code is about code, not documentation.
>>>>>
>>>>> I've seen and heard that a lot of work is still needed on
>>>>> http://svn.python.org/view/python/trunk both during the 3.1 release
>>>>> cycle,
>>>>> optimization possible all over the place.  It'd be great if those of
>>>>> you
>>>>> working closely with this can shout out some ideas, brainstorm a bit.
>>>>>
>>>>> PSF was announced as one of the mentoring orgs today, this week before
>>>>> student applications are open is for students to talk to their
>>>>> prospective
>>>>> mentors and iron out the wrinkles in their plans, so there's not much
>>>>> time
>>>>> to get core project ideas together.
>>>>
>>>> How about porting PIL to 3.0?
>>>> There were many such requests on python-list and image-sig (including
>>>> mine
>>>> :))
>>>>
>>>
>>> I have ported it to the stage where its tests passes (which are far
>>> from covering all the code) and some of my own tests, there is a git
>>> repo on the image-sig that points to it. I wasn't really careful with
>>> some of the things (and I would even consider redoing some of them),
>>> but only one or two people got a copy of it so apparently people don't
>>> want/need it on python 3.0 just yet (not it alone at least).
>>
>> I did a "git clone git://gpolo.ath.cx/pil-py3k.git" but it failed:
>>
>> gpolo.ath.cx[0: 189.7.18.241]: errno=Connection timed out
>> fatal: unable to connect a socket (Connection timed out)
>> fetch-pack from 'git://gpolo.ath.cx/pil-py3k.git' failed.
>>
>
> Thanks for noticing that, maybe more people had this same problem
> then, I will consider using github or some similar service (or maybe
> take the chance to bazaar, or mercurial, or svn, or..).
>
>> By the way the reason I think few people checked it out is that people
>> mostly are waiting for an "official" PIL release that is known to be
>> stable. Did you try making your port part of the "official" PIL
>> distribution?
>>
>
> I have talked with Fredrik, he said he would be running it on another
> test suite to check how much of it really works. But, no, I didn't
> really try pushing it to be integrated into the next PIL release and
> it also wouldn't be possible without distributing a py3k version only
> -- I didn't do the port with the ability to work in python 3.x and
> python 2.x but this can be arranged.

Maybe I'm misunderstanding you but I didn't mean to say that this
version should work on both python 2.x and python 3.x. Ideally, there
would be a PIL distribution for 2.x only and another one for 3.x only.
The only thing is that people (myself included) will only be
comfortable with the PIL for 3.x version if it comes with the
blessings of Fredrik, i.e. if I were you I'd try pushing this with
Fredrik. After all if all tests pass including the ones Fredrik has
for himself, there should be no problem and I suppose he would be
happy to have a PIL for python 3.x.

Until then I'd be happy to check out your own port, whenever you have
a working repository copy please let us know.

Cheers,
Daniel

-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown

From benjamin at python.org  Thu Mar 19 02:48:07 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 18 Mar 2009 20:48:07 -0500
Subject: [Python-Dev] what's new in 3.1 anyone?
Message-ID: <1afaf6160903181848w3115a943u32755d5b0453747b@mail.gmail.com>

It seems Andrew will be doing "What's new in Python 2.7?" again, but
we don't seem to have a volunteer to do the 3.1 version? Would anyone
like to volunteer?

-- 
Regards,
Benjamin

From python at rcn.com  Thu Mar 19 03:02:20 2009
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 18 Mar 2009 19:02:20 -0700
Subject: [Python-Dev] what's new in 3.1 anyone?
References: <1afaf6160903181848w3115a943u32755d5b0453747b@mail.gmail.com>
Message-ID: <400047BDC33848EDB80FE1A1C46D39D0@RaymondLaptop1>


[Benjamin Peterson]
> It seems Andrew will be doing "What's new in Python 2.7?" again, but
> we don't seem to have a volunteer to do the 3.1 version? Would anyone
> like to volunteer?

I'll pick-up this responsibility.


Raymond

From tleeuwenburg at gmail.com  Thu Mar 19 04:33:51 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Thu, 19 Mar 2009 14:33:51 +1100
Subject: [Python-Dev] tracker status options
Message-ID: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>

Hi all,

I'm continuing to (slowly) work through issues. I have been looking
particularly at a lot of the open issues regarding strftime.

It would be great to put in some of those extra status options that were
discussed recently...

   "Open/New"
   "Needs help / Chatting"
   "Under development"
   "Pending feedback"
   "Closed"

For everyone's reference, after some debate, the above list of status
options was where the conversation finished. So the two options "Needs help
/ chatting" and "Under development" would need to be added. Issues that are
"Under development" should revert to "Needs help / chatting" after a month
of inactivity.

Would it be possible to get these put in? Maybe I'm one of a small number of
people who are nibbling at the bottom end of the bugs, but I find it
somewhat frustrating not to be able to classify the issues that I find into
"needs help / chatting" vs "under development" to help make more sense of
the search results.

Cheers,
-T
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090319/4d765c26/attachment.htm>

From martin at v.loewis.de  Thu Mar 19 07:47:58 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 19 Mar 2009 07:47:58 +0100
Subject: [Python-Dev] tracker status options
In-Reply-To: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>
References: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>
Message-ID: <49C1EA9E.3040603@v.loewis.de>

> It would be great to put in some of those extra status options that were
> discussed recently...
> 
>    "Open/New"
>    "Needs help / Chatting"
>    "Under development"
>    "Pending feedback"
>    "Closed"

Are you sure that you want them to be status options? Why not stages?
ISTM that an issue that "Needs help" is still "Open".

Martin


From tjreedy at udel.edu  Thu Mar 19 09:58:27 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 19 Mar 2009 04:58:27 -0400
Subject: [Python-Dev] Core projects: 3to2
In-Reply-To: <loom.20090319T001035-310@post.gmane.org>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>	<gprp3t$icr$2@ger.gmane.org>	<loom.20090318T213554-597@post.gmane.org>	<gps25i$fjs$1@ger.gmane.org>
	<loom.20090319T001035-310@post.gmane.org>
Message-ID: <gpt1fi$i3u$1@ger.gmane.org>

Antoine Pitrou wrote:
> Terry Reedy <tjreedy <at> udel.edu> writes:
>> Some of the people who need to support both late 2.x and 3.x would 
>> prefer to write 3.x code and backport.  The OP of a current python-list 
>> thread asked whether there was any way to write something like
>>
>> 	@alias('__nonzero__')
>> 	def __bool__(self): return True
> 
> How about simply:
>     __nonzero__ = __bool__
> 
>> I believe my own 3.0 code will mainly also need
>> print() to print statement
> 
> If this is only about supporting "late 2.x" (i.e., 2.6 and upwards), you can
> already write:

People often do not specify.  I suspect some are thinking back to 2.5, 
but that will change in the future.

> from __future__ import print_function

I was not aware of that.  Would be ok for my current project which has 
print isolated in a few modules, at least so far.

>> except e as a to 2.x version
> 
> Works in 2.6.

Did not know that.  Perhaps a 3 to 2.6+ guide would help.
> 
>> class C() to class C(object)

Part of the reason to move to 3.0 is to not have to do that.
> 
> __metaclass__ = type
> 
> Now I'm not saying that all 3.0 code will work in 2.6 with such simple
> precautions, far from it!

tjr


From martin at v.loewis.de  Thu Mar 19 10:04:11 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 19 Mar 2009 10:04:11 +0100
Subject: [Python-Dev] Security fixes in 2.5 and 2.4
Message-ID: <49C20A8B.8040007@v.loewis.de>

I just got a few questions on how to apply security fixes.
To clarify, I recommend the following guidelines:

- whether something constitutes a security bug is sometimes
  debatable - in case of doubt, discussion is needed. I would
  be in favor of fixing it if the patch is small and obviously
  correct, and opposed if the patch looks tricky. Double check
  that the routine behavior (the "good" cases) stay completely
  unchanged (in particular, be aware of not allowing new
  exceptions to occur).
- if you want to backport a security bug fix to 2.5, ALWAYS
  consider 2.4 as well. They are in the same state, and should
  get the same care (2.3 is closed for good). Of course, it
  might be that the bug doesn't exist in 2.4.
- ALWAYS notify security at python.org. For one thing, they might
  offer advise on how to proceed, but also, they might consider
  publishing an advisory, and/or notifying some CERT. Notification
  is in particular necessary if you are unfamiliar with security
  issues, how they get classified, and so on - so do ask the
  experts. (and no, I'm not one of them :-)

Regards,
Martin

From andrew at acooke.org  Thu Mar 19 11:28:46 2009
From: andrew at acooke.org (andrew cooke)
Date: Thu, 19 Mar 2009 06:28:46 -0400 (CLT)
Subject: [Python-Dev] Core projects: 3to2
In-Reply-To: <gpt1fi$i3u$1@ger.gmane.org>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org> <loom.20090318T213554-597@post.gmane.org>
	<gps25i$fjs$1@ger.gmane.org> <loom.20090319T001035-310@post.gmane.org>
	<gpt1fi$i3u$1@ger.gmane.org>
Message-ID: <7bdaab0f1b26321c53386f123e70c461.squirrel@localhost>

Terry Reedy wrote:
> Antoine Pitrou wrote:
>> Terry Reedy <tjreedy <at> udel.edu> writes:
>>> Some of the people who need to support both late 2.x and 3.x would
>>> prefer to write 3.x code and backport.  The OP of a current python-list
>>> thread asked whether there was any way to write something like
>>>
>>> 	@alias('__nonzero__')
>>> 	def __bool__(self): return True
>>
>> How about simply:
>>     __nonzero__ = __bool__
>>
>>> I believe my own 3.0 code will mainly also need
>>> print() to print statement
>>
>> If this is only about supporting "late 2.x" (i.e., 2.6 and upwards), you
>> can
>> already write:
>
> People often do not specify.  I suspect some are thinking back to 2.5,
> but that will change in the future.


i am the author of the original post quoted above.  i wrote a parser
library (lepl) using 3.0.  backporting to 2.6 was fairly easy, but it
still does not run with 2.5.

if i remember correctly it appeared that i was going to need separate
source files because of significant differences in syntax (print,
exceptions) as well as missing functionality (metaclasses, string
formatting).

andrew



From ajaksu at gmail.com  Thu Mar 19 18:32:03 2009
From: ajaksu at gmail.com (ajaksu)
Date: Thu, 19 Mar 2009 10:32:03 -0700 (PDT)
Subject: [Python-Dev] snakebite for GSoC?
Message-ID: <77f58fdd-9d84-4f1c-9238-009e46973cbc@e18g2000yqo.googlegroups.com>

Hi,

Does anyone have good ideas for assigning students to snakebite? Is it
too early?

I think the  client-side 'Snakebite daemon' and server-side stuff
described at http://tinyurl.com/beyond-buildbot would be great
projects.

Daniel

From ajaksu at gmail.com  Thu Mar 19 19:28:52 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Thu, 19 Mar 2009 15:28:52 -0300
Subject: [Python-Dev] GSoC ideas in need of mentors :)
Message-ID: <2d75d7660903191128g56381136i431d200a3ed0a8db@mail.gmail.com>

Hi,

I'd like to propose a two housecleaning GSoC ideas for discussion,
they both need mentors. First, Mark's suggested overhaul of the struct
module, along with finishing PEP 3118 if possible. Second, a clean up
of the socket module, along with checking its usage in the stdlib if
possible. Details below.

IIUC, any potential mentors for ideas from
http://wiki.python.org/moin/CodingProjectIdeas would help the PSF by
declaring availability at
http://wiki.python.org/moin/SummerOfCode/2009.

And, mentioned in another thread, snakebite ideas:
http://tinyurl.com/beyond-buildbot :)

Regards,
Daniel

---
Overhaul struct + PEP 3118
There are many open issues for the struct module, and a suggestion
that many others fixes should happen. The problems include
inconsistencies in input handling and backwards-compatibility code
that should be gone from 3.x.

As PEP 3118 also requires a few changes in struct, finishing the
memoryview object implementation and backporting it to 2.7 could be
part of this project.

To tackle this, a student should know C and have some familiarity with
the CPython codebase. The proposal should include an overview of open
issues and, if possible, of problems in current code.

Mentors for this task should have a good understanding of the struct
module, the buffer interface and PEP 3118.

---
Clean up and improve the socket module

The socket module is the foundation of many important parts of the
stdlib. It has many open issues and RFEs, with many more being
indirectly dependent on it. Major pain areas are cross-platform
problems, shortcomings for a few use cases and some missing bits from
the API (e.g. recvall(), sendmsg() and recvmsg()). If these can be
solved quickly, going through the uses of the socket module in the
stdlib to would make this project even more useful.

This project has some ugly cross-platform needs, but these can be
handled by the student and the mentor together (maybe snakebite could
help?). For students wanting to work on this, knowing C is a must,
being familiar with CPython and the stdlib would be desirable. Having
some prior knowledge of POSIX and Windows interfaces the socket module
uses would be a nice perk.

Mentors should be able to help the student to test their code on many
different platforms. Knowing the socket API in different platforms
would be desirable.

From python at rcn.com  Thu Mar 19 21:06:04 2009
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 19 Mar 2009 13:06:04 -0700
Subject: [Python-Dev] Core projects for Summer of Code
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<loom.20090318T212139-468@post.gmane.org><20090318215640.GF28622@idyll.org>
	<49C17926.8090501@gmail.com>
Message-ID: <8E310C94793C46A3A5F72B57D68FA081@RaymondLaptop1>

Another thought:  it would be nice is pydoc were built-out with 
an alternate html generator that emitted clean, simple html
with the appropriate div/span tags so that CSS can be used
to control formatting.  Right now, all of the formatting and
color coding is in-line.  If you don't like the appearance of
the output, the module is unusable.  This is likely a two to
three day project, easy and fun.


Raymond

From brett at python.org  Thu Mar 19 22:14:04 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 19 Mar 2009 14:14:04 -0700
Subject: [Python-Dev] tracker status options
In-Reply-To: <49C1EA9E.3040603@v.loewis.de>
References: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>
	<49C1EA9E.3040603@v.loewis.de>
Message-ID: <bbaeab100903191414o1710bd87x3d4c9a713c63c832@mail.gmail.com>

On Wed, Mar 18, 2009 at 23:47, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> > It would be great to put in some of those extra status options that were
> > discussed recently...
> >
> >    "Open/New"
> >    "Needs help / Chatting"
> >    "Under development"
> >    "Pending feedback"
> >    "Closed"
>
> Are you sure that you want them to be status options? Why not stages?


Because Tennessee is after a way to sift through open issues looking for
ones that are not being actively worked on, thus the "Under Dev" status.
Just because an issue "needs [a] patch" doesn't means it is being worked on
currently.


>
> ISTM that an issue that "Needs help" is still "Open".


Once the backlog of bugs has been cleaned up and we get into a habit of
triaging new bugs as soon as they come in this won't be quite as necessary.
I believe Tennessee wanted this status to signify that more input is needed
from others originally and to perform triage. Out of the two this one is
needed the least in my opinion.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090319/f757a496/attachment.htm>

From brett at python.org  Thu Mar 19 22:15:22 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 19 Mar 2009 14:15:22 -0700
Subject: [Python-Dev] Security fixes in 2.5 and 2.4
In-Reply-To: <49C20A8B.8040007@v.loewis.de>
References: <49C20A8B.8040007@v.loewis.de>
Message-ID: <bbaeab100903191415w33305a2fqd8501c83790705fb@mail.gmail.com>

On Thu, Mar 19, 2009 at 02:04, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> I just got a few questions on how to apply security fixes.
> To clarify, I recommend the following guidelines:
>
> - whether something constitutes a security bug is sometimes
>  debatable - in case of doubt, discussion is needed. I would
>  be in favor of fixing it if the patch is small and obviously
>  correct, and opposed if the patch looks tricky. Double check
>  that the routine behavior (the "good" cases) stay completely
>  unchanged (in particular, be aware of not allowing new
>  exceptions to occur).
> - if you want to backport a security bug fix to 2.5, ALWAYS
>  consider 2.4 as well. They are in the same state, and should
>  get the same care (2.3 is closed for good). Of course, it
>  might be that the bug doesn't exist in 2.4.
> - ALWAYS notify security at python.org. For one thing, they might
>  offer advise on how to proceed, but also, they might consider
>  publishing an advisory, and/or notifying some CERT. Notification
>  is in particular necessary if you are unfamiliar with security
>  issues, how they get classified, and so on - so do ask the
>  experts. (and no, I'm not one of them :-)


All sounds reasonable, although getting those of us on security@ to get an
announcement out has not gone so well as of late. =)

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090319/a22ec219/attachment.htm>

From martin at v.loewis.de  Fri Mar 20 00:07:44 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Fri, 20 Mar 2009 00:07:44 +0100
Subject: [Python-Dev] tracker status options
In-Reply-To: <bbaeab100903191414o1710bd87x3d4c9a713c63c832@mail.gmail.com>
References: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>	
	<49C1EA9E.3040603@v.loewis.de>
	<bbaeab100903191414o1710bd87x3d4c9a713c63c832@mail.gmail.com>
Message-ID: <49C2D040.4080609@v.loewis.de>

> On Wed, Mar 18, 2009 at 23:47, "Martin v. L?wis" <martin at v.loewis.de
> <mailto:martin at v.loewis.de>> wrote:
> 
>     > It would be great to put in some of those extra status options
>     that were
>     > discussed recently...
>     >
>     >    "Open/New"
>     >    "Needs help / Chatting"
>     >    "Under development"
>     >    "Pending feedback"
>     >    "Closed"
> 
>     Are you sure that you want them to be status options? Why not stages?
> 
> 
> Because Tennessee is after a way to sift through open issues looking for
> ones that are not being actively worked on, thus the "Under Dev" status.
> Just because an issue "needs [a] patch" doesn't means it is being worked
> on currently.

How is "being actively worked on" defined? How do these additional
status values help in answering that question?

>     ISTM that an issue that "Needs help" is still "Open".
> 
> 
> Once the backlog of bugs has been cleaned up and we get into a habit of
> triaging new bugs as soon as they come in this won't be quite as
> necessary. I believe Tennessee wanted this status to signify that more
> input is needed from others originally and to perform triage. Out of the
> two this one is needed the least in my opinion.

If you want them, we can add them. However, I would then need a complete
specification of how various pieces that currently refer to Open/Closed
should behave with respect to the new status, such as
- list of issues displayed
- weekly email notifications
- initial value that status should get
- relationship to existing status values

In addition, I would like to see a specification of the exact labels to
be used, plus a one-line description that should be attached to the
labels.

Regards,
Martin

From ajaksu at gmail.com  Fri Mar 20 00:53:45 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Thu, 19 Mar 2009 20:53:45 -0300
Subject: [Python-Dev] tracker status options
In-Reply-To: <49C2D040.4080609@v.loewis.de>
References: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>
	<49C1EA9E.3040603@v.loewis.de>
	<bbaeab100903191414o1710bd87x3d4c9a713c63c832@mail.gmail.com>
	<49C2D040.4080609@v.loewis.de>
Message-ID: <2d75d7660903191653v65640c90pc4df8ddcf60343f1@mail.gmail.com>

Martin v. L?wis wrote:
> In addition, I would like to see a specification of the exact labels to
> be used, plus a one-line description that should be attached to the
> labels.

Tennessee,
If you'd like to test those additional status options, I'm setting a
test instance of the Python tracker up at
http://bot.bio.br/python-dev/ . It might be frequently offline for a
while, but once things are stable it'll be reliable enough to perform
such experiments.

If it's easy on resource usage, I might have one instance following
the Python tracker code closely and a more bleeding-edge one :)

Regards,
Daniel

From steve at holdenweb.com  Fri Mar 20 01:54:38 2009
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 19 Mar 2009 20:54:38 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <8E310C94793C46A3A5F72B57D68FA081@RaymondLaptop1>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<loom.20090318T212139-468@post.gmane.org><20090318215640.GF28622@idyll.org>	<49C17926.8090501@gmail.com>
	<8E310C94793C46A3A5F72B57D68FA081@RaymondLaptop1>
Message-ID: <49C2E94E.4010703@holdenweb.com>

Raymond Hettinger wrote:
> Another thought:  it would be nice is pydoc were built-out with an
> alternate html generator that emitted clean, simple html
> with the appropriate div/span tags so that CSS can be used
> to control formatting.  Right now, all of the formatting and
> color coding is in-line.  If you don't like the appearance of
> the output, the module is unusable.  This is likely a two to
> three day project, easy and fun.
> 
That makes it a much better candidate for GHOP that SoC, which requires
projects with a little more meat on them.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Fri Mar 20 01:54:38 2009
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 19 Mar 2009 20:54:38 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <8E310C94793C46A3A5F72B57D68FA081@RaymondLaptop1>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<loom.20090318T212139-468@post.gmane.org><20090318215640.GF28622@idyll.org>	<49C17926.8090501@gmail.com>
	<8E310C94793C46A3A5F72B57D68FA081@RaymondLaptop1>
Message-ID: <49C2E94E.4010703@holdenweb.com>

Raymond Hettinger wrote:
> Another thought:  it would be nice is pydoc were built-out with an
> alternate html generator that emitted clean, simple html
> with the appropriate div/span tags so that CSS can be used
> to control formatting.  Right now, all of the formatting and
> color coding is in-line.  If you don't like the appearance of
> the output, the module is unusable.  This is likely a two to
> three day project, easy and fun.
> 
That makes it a much better candidate for GHOP that SoC, which requires
projects with a little more meat on them.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From arcriley at gmail.com  Fri Mar 20 02:19:42 2009
From: arcriley at gmail.com (Arc Riley)
Date: Thu, 19 Mar 2009 21:19:42 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <49C2E94E.4010703@holdenweb.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<loom.20090318T212139-468@post.gmane.org>
	<20090318215640.GF28622@idyll.org> <49C17926.8090501@gmail.com>
	<8E310C94793C46A3A5F72B57D68FA081@RaymondLaptop1>
	<49C2E94E.4010703@holdenweb.com>
Message-ID: <bad82a80903191819n32c4059dn24d5616b8090565b@mail.gmail.com>

> That makes it a much better candidate for GHOP that SoC, which requires
> projects with a little more meat on them.


Yes it does.

Though many organizations have taken to funding their own GHOPs.  Perhaps
this year PSF can use the SoC funds ($500/student) to host a bounty-sprint
program much like GHOP?

IIRC, GHOP paid $100 per 3 tasks.  With the economy in the tank this could
attract a lot more than highschool students.  Honestly I wish SoC was
structured more like GHOP, it seemed much more effective and for the same
funding could run year-round.

Or the same funds could be used to host various optimization competitions,
replacement Py3 extension/type/function must match API and pass unit
testing.  Fastest solution wins (ie) $1000 and fame for the person or team
that submitted it.

Something to keep in mind and for the PSF board to ponder for this Fall.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090319/0e514ac7/attachment.htm>

From steve at holdenweb.com  Fri Mar 20 03:00:54 2009
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 19 Mar 2009 22:00:54 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bad82a80903191819n32c4059dn24d5616b8090565b@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<loom.20090318T212139-468@post.gmane.org>	<20090318215640.GF28622@idyll.org>
	<49C17926.8090501@gmail.com>	<8E310C94793C46A3A5F72B57D68FA081@RaymondLaptop1>	<49C2E94E.4010703@holdenweb.com>
	<bad82a80903191819n32c4059dn24d5616b8090565b@mail.gmail.com>
Message-ID: <49C2F8D6.3010600@holdenweb.com>

Arc Riley wrote:
> 
>     That makes it a much better candidate for GHOP that SoC, which requires
>     projects with a little more meat on them.
> 
> 
> Yes it does.
> 
> Though many organizations have taken to funding their own GHOPs. 
> Perhaps this year PSF can use the SoC funds ($500/student) to host a
> bounty-sprint program much like GHOP?
> 
> IIRC, GHOP paid $100 per 3 tasks.  With the economy in the tank this
> could attract a lot more than highschool students.  Honestly I wish SoC
> was structured more like GHOP, it seemed much more effective and for the
> same funding could run year-round.
> 
> Or the same funds could be used to host various optimization
> competitions, replacement Py3 extension/type/function must match API and
> pass unit testing.  Fastest solution wins (ie) $1000 and fame for the
> person or team that submitted it.
> 
> Something to keep in mind and for the PSF board to ponder for this Fall.
> 
Why wait until Fall if it's a good idea? The summer vacation would
surely be the ideal time for this, and that would mean we should start
planning soon.

Anyway, the first requirement would be some enthusiasm from the
developer team for mobilizing such a potential source of assistance.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Fri Mar 20 03:00:54 2009
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 19 Mar 2009 22:00:54 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bad82a80903191819n32c4059dn24d5616b8090565b@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<loom.20090318T212139-468@post.gmane.org>	<20090318215640.GF28622@idyll.org>
	<49C17926.8090501@gmail.com>	<8E310C94793C46A3A5F72B57D68FA081@RaymondLaptop1>	<49C2E94E.4010703@holdenweb.com>
	<bad82a80903191819n32c4059dn24d5616b8090565b@mail.gmail.com>
Message-ID: <49C2F8D6.3010600@holdenweb.com>

Arc Riley wrote:
> 
>     That makes it a much better candidate for GHOP that SoC, which requires
>     projects with a little more meat on them.
> 
> 
> Yes it does.
> 
> Though many organizations have taken to funding their own GHOPs. 
> Perhaps this year PSF can use the SoC funds ($500/student) to host a
> bounty-sprint program much like GHOP?
> 
> IIRC, GHOP paid $100 per 3 tasks.  With the economy in the tank this
> could attract a lot more than highschool students.  Honestly I wish SoC
> was structured more like GHOP, it seemed much more effective and for the
> same funding could run year-round.
> 
> Or the same funds could be used to host various optimization
> competitions, replacement Py3 extension/type/function must match API and
> pass unit testing.  Fastest solution wins (ie) $1000 and fame for the
> person or team that submitted it.
> 
> Something to keep in mind and for the PSF board to ponder for this Fall.
> 
Why wait until Fall if it's a good idea? The summer vacation would
surely be the ideal time for this, and that would mean we should start
planning soon.

Anyway, the first requirement would be some enthusiasm from the
developer team for mobilizing such a potential source of assistance.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From arcriley at gmail.com  Fri Mar 20 03:17:37 2009
From: arcriley at gmail.com (Arc Riley)
Date: Thu, 19 Mar 2009 22:17:37 -0400
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <49C2F8D6.3010600@holdenweb.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<loom.20090318T212139-468@post.gmane.org>
	<20090318215640.GF28622@idyll.org> <49C17926.8090501@gmail.com>
	<8E310C94793C46A3A5F72B57D68FA081@RaymondLaptop1>
	<49C2E94E.4010703@holdenweb.com>
	<bad82a80903191819n32c4059dn24d5616b8090565b@mail.gmail.com>
	<49C2F8D6.3010600@holdenweb.com>
Message-ID: <bad82a80903191917m38ba69a5i1af1792921afab79@mail.gmail.com>

If it's organized in time we could scoop up some of the SoC applicants who
we'll like to have but we won't have slots for

Honestly I like the idea of competitions.  Better publicity, the greater
prizes will draw out some better minds from the community, and competitions
based on quality will help ensure usable code.  They could vary in reward
based on how difficult the problem is and perhaps have some Python swag as
runner-up prizes.

It'd also be a great way to promote Python 3.


On Thu, Mar 19, 2009 at 10:00 PM, Steve Holden <steve at holdenweb.com> wrote:

> Arc Riley wrote:
> >
> >     That makes it a much better candidate for GHOP that SoC, which
> requires
> >     projects with a little more meat on them.
> >
> >
> > Yes it does.
> >
> > Though many organizations have taken to funding their own GHOPs.
> > Perhaps this year PSF can use the SoC funds ($500/student) to host a
> > bounty-sprint program much like GHOP?
> >
> > IIRC, GHOP paid $100 per 3 tasks.  With the economy in the tank this
> > could attract a lot more than highschool students.  Honestly I wish SoC
> > was structured more like GHOP, it seemed much more effective and for the
> > same funding could run year-round.
> >
> > Or the same funds could be used to host various optimization
> > competitions, replacement Py3 extension/type/function must match API and
> > pass unit testing.  Fastest solution wins (ie) $1000 and fame for the
> > person or team that submitted it.
> >
> > Something to keep in mind and for the PSF board to ponder for this Fall.
> >
> Why wait until Fall if it's a good idea? The summer vacation would
> surely be the ideal time for this, and that would mean we should start
> planning soon.
>
> Anyway, the first requirement would be some enthusiasm from the
> developer team for mobilizing such a potential source of assistance.
>
> regards
>  Steve
> --
> Steve Holden           +1 571 484 6266   +1 800 494 3119
> Holden Web LLC                 http://www.holdenweb.com/
> Want to know? Come to PyCon - soon! http://us.pycon.org/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090319/bb0f7dab/attachment.htm>

From trent.nelson at snakebite.org  Fri Mar 20 08:37:40 2009
From: trent.nelson at snakebite.org (Trent Nelson)
Date: Fri, 20 Mar 2009 07:37:40 +0000
Subject: [Python-Dev] [snakebite] snakebite for GSoC?
In-Reply-To: <77f58fdd-9d84-4f1c-9238-009e46973cbc@e18g2000yqo.googlegroups.com>
References: <77f58fdd-9d84-4f1c-9238-009e46973cbc@e18g2000yqo.googlegroups.com>
Message-ID: <20090320073740.GA54156@wind.teleri.net>

On Thu, Mar 19, 2009 at 10:32:03AM -0700, ajaksu wrote:
> Does anyone have good ideas for assigning students to snakebite? Is it
> too early?
    
    Perhaps a little too early, python-dev@ won't know anything about
    Snakebite yet as I haven't publicly announced it there ;-)  Watch
    this space closer to PyCon.

    FWIW, though, we're planning for Snakebite to be *very* involved
    with GSoC/GHOP.
 
> I think the  client-side 'Snakebite daemon' and server-side stuff
> described at http://tinyurl.com/beyond-buildbot would be great
> projects.

    Indeed.

        Trent.

From paul.bedaride at gmail.com  Fri Mar 20 14:35:10 2009
From: paul.bedaride at gmail.com (paul bedaride)
Date: Fri, 20 Mar 2009 14:35:10 +0100
Subject: [Python-Dev] Proposal: new list function: pack
Message-ID: <fa7d4c4f0903200635w3bbc3c45m9a4202fd966ea769@mail.gmail.com>

I propose a new function for list for pack values of a list and
sliding over them:

then we can do things like this:
for i, j, k in pack(range(10), 3, partialend=False):
    print i, j, k

I propose this because i need a lot of times pack and slide function
over list and this one
combine the two in a generator way.

def pack(l, size=2, slide=2, partialend=True):
    lenght = len(l)
    for p in range(0,lenght-size,slide):
        def packet():
            for i in range(size):
                yield l[p+i]
        yield packet()
    p = p + slide
    if partialend or lenght-p == size:
        def packet():
            for i in range(lenght-p):
                yield l[p+i]
        yield packet()

a = range(10)
print a
print 'pack(a, 2, 2,  True):',  [list(p) for p in pack(a, 2, 2, True)]
print 'pack(a, 2, 2, False):',  [list(p) for p in pack(a, 2, 2, False)]
print 'pack(a, 2, 3,  True):', [list(p) for p in pack(a, 2, 3, True)]
print 'pack(a, 2, 3, False):', [list(p) for p in pack(a, 2, 3, False)]
print 'pack(a, 3, 2,  True):', [list(p) for p in pack(a, 3, 2, True)]
print 'pack(a, 3, 2, False):', [list(p) for p in pack(a, 3, 2, False)]
print 'pack(a, 3, 3,  True):', [list(p) for p in pack(a, 3, 3, True)]
print 'pack(a, 3, 3, False):', [list(p) for p in pack(a, 3, 3, False)]

paul bedaride

From ctb at msu.edu  Fri Mar 20 15:01:52 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Fri, 20 Mar 2009 07:01:52 -0700
Subject: [Python-Dev] [snakebite] Re: snakebite for GSoC?
In-Reply-To: <20090320073740.GA54156@wind.teleri.net>
References: <77f58fdd-9d84-4f1c-9238-009e46973cbc@e18g2000yqo.googlegroups.com>
	<20090320073740.GA54156@wind.teleri.net>
Message-ID: <20090320140151.GG23043@idyll.org>

On Fri, Mar 20, 2009 at 07:37:40AM +0000, Trent Nelson wrote:
-> 
-> On Thu, Mar 19, 2009 at 10:32:03AM -0700, ajaksu wrote:
-> > Does anyone have good ideas for assigning students to snakebite? Is it
-> > too early?
->     
->     Perhaps a little too early, python-dev@ won't know anything about
->     Snakebite yet as I haven't publicly announced it there ;-)  Watch
->     this space closer to PyCon.

I do have a snakebite-motivated project, listed here:

  http://ivory.idyll.org/blog/mar-09/gsoc-projects.html

(#7)

Right now an independent study student is building something, but he
can't work on it over the summer, so continuing it in various ways could
be a GSoC project.

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From aahz at pythoncraft.com  Fri Mar 20 15:17:02 2009
From: aahz at pythoncraft.com (Aahz)
Date: Fri, 20 Mar 2009 07:17:02 -0700
Subject: [Python-Dev] Proposal: new list function: pack
In-Reply-To: <fa7d4c4f0903200635w3bbc3c45m9a4202fd966ea769@mail.gmail.com>
References: <fa7d4c4f0903200635w3bbc3c45m9a4202fd966ea769@mail.gmail.com>
Message-ID: <20090320141702.GA6002@panix.com>

On Fri, Mar 20, 2009, paul bedaride wrote:
>
> I propose a new function for list for pack values of a list and
> sliding over them:

Please switch this discussion to python-ideas
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Programming language design is not a rational science. Most reasoning
about it is at best rationalization of gut feelings, and at worst plain
wrong."  --GvR, python-ideas, 2009-3-1

From ijmorlan at uwaterloo.ca  Fri Mar 20 15:07:36 2009
From: ijmorlan at uwaterloo.ca (Isaac Morland)
Date: Fri, 20 Mar 2009 10:07:36 -0400 (EDT)
Subject: [Python-Dev] Proposal: new list function: pack
In-Reply-To: <fa7d4c4f0903200635w3bbc3c45m9a4202fd966ea769@mail.gmail.com>
References: <fa7d4c4f0903200635w3bbc3c45m9a4202fd966ea769@mail.gmail.com>
Message-ID: <Pine.GSO.4.64.0903200958310.22372@core.cs.uwaterloo.ca>

On Fri, 20 Mar 2009, paul bedaride wrote:

> I propose a new function for list for pack values of a list and
> sliding over them:
>
> then we can do things like this:
> for i, j, k in pack(range(10), 3, partialend=False):
>    print i, j, k
>
> I propose this because i need a lot of times pack and slide function
> over list and this one
> combine the two in a generator way.

See the Python documentation for zip():

http://docs.python.org/library/functions.html#zip

And this article in which somebody independently rediscovers the idea:

http://drj11.wordpress.com/2009/01/28/my-python-dream-about-groups/

Summary: except for the "partialend" parameter, this can already be done 
in a single line.  It is not for me to say whether this nevertheless would 
be useful as a library routine (if only perhaps to make it easy to specify 
"partialend" explicitly).

It seems to me that sometimes one would want izip instead of zip.  And I 
think you could get the effect of partialend=True in 2.6 by using 
izip_longest (except with an iterator result rather than a list).

> def pack(l, size=2, slide=2, partialend=True):
>    lenght = len(l)
>    for p in range(0,lenght-size,slide):
>        def packet():
>            for i in range(size):
>                yield l[p+i]
>        yield packet()
>    p = p + slide
>    if partialend or lenght-p == size:
>        def packet():
>            for i in range(lenght-p):
>                yield l[p+i]
>        yield packet()

Isaac Morland			CSCF Web Guru
DC 2554C, x36650		WWW Software Specialist

From hrvoje.niksic at avl.com  Fri Mar 20 16:37:13 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Fri, 20 Mar 2009 16:37:13 +0100
Subject: [Python-Dev] Proposal: new list function: pack
In-Reply-To: <1900589.3551164.1237558955033.JavaMail.xicrypt@atgrzls001>
References: <fa7d4c4f0903200635w3bbc3c45m9a4202fd966ea769@mail.gmail.com>
	<1900589.3551164.1237558955033.JavaMail.xicrypt@atgrzls001>
Message-ID: <49C3B829.9060908@avl.com>

Isaac Morland wrote:
>> I propose this because i need a lot of times pack and slide function
>> over list and this one
>> combine the two in a generator way.

I've written functions with a subset of this functionality on more than 
one occasion.  Having it in itertools looks like it would be useful to a 
lot of people.

> See the Python documentation for zip():
> 
> http://docs.python.org/library/functions.html#zip

zip can be used to achieve this purpose, but only with serious 
itertools-fu.  If I want to iterate over a list [1, 2, 3, 4] looking at 
pairs (1, 2) and (3, 4), it would be much nicer to write:

for a, b in itertools.pack(l, 2):
     ...

than

for a, b in itertools.izip(*[iter(l)]*2):
     ...

which is what the zip documentation proposes.  The former would be clear 
to anyone looking at the documentation of "pack" (and maybe even without 
it if we choose a better name), while the latter requires quite some 
deciphering, followed by carefully looking at izip's documentation that 
it's actually legal to rely on argument evaluation order and not peeking 
at iterables, like that code does.

izip is not the only contender for this pattern; something similar is 
possible using groupby, but it's hard to make it fit in an easily 
understable line either.  This is the shortest I came up with:

def pack(iterable, n):
     cycler = (i for i in itertools.count() for j in xrange(n))
     return (g for k, g in
             itertools.groupby(iterable, lambda x: cycler.next()))

This has the nice property that it returns iterables rather than tuples, 
although tuples are probably good enough (they seem to be good enough 
for izip).

The name "pack" is a bit too cryptic, even by itertools standards, so it 
might be better to choose a name that conveys the intention of returning 
"groups of n adjacent elements" (group_adjacent?).  To fit with the rest 
of itertools, and to be really useful, the function shouldn't insist on 
sequences, but should accept any iterable.

> http://drj11.wordpress.com/2009/01/28/my-python-dream-about-groups/

That posting ends with:

"""
It still scares me a bit.
This code is obviously ridiculous. I can?t help feeling I?ve missed a 
more Pythonic way of doing it.
"""

Looking at izip(*[iter(l)]*n), I tend to agree.

From status at bugs.python.org  Fri Mar 20 18:08:09 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 20 Mar 2009 18:08:09 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090320170809.A348978394@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (03/13/09 - 03/20/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2392 open (+27) / 14957 closed (+14) / 17349 total (+41)

Open issues with patches:   836

Average duration of open issues: 661 days.
Median duration of open issues: 402 days.

Open Issues Breakdown
   open  2362 (+25)
pending    30 ( +2)

Issues Created Or Reopened (44)
_______________________________

PATCH: Armin's attribute lookup caching for 3.0                  03/14/09
       http://bugs.python.org/issue1568    reopened pitrou                        
       patch                                                                   

merge json library with latest simplejson 2.0.x                  03/17/09
       http://bugs.python.org/issue4136    reopened pitrou                        
       patch                                                                   

profile doesn't support non-UTF8 source code                     03/20/09
       http://bugs.python.org/issue4282    reopened haypo                         
       patch, needs review                                                     

doc copyedits                                                    03/13/09
CLOSED http://bugs.python.org/issue5486    created  dsm001                        
       patch                                                                   

Parts of Tkinter missing (but not when running from IDLE)        03/14/09
CLOSED http://bugs.python.org/issue5487    created  oc                            
                                                                               

nb_inplace_divide slot is missing in docs                        03/14/09
CLOSED http://bugs.python.org/issue5488    created  donlorenzo                    
       patch                                                                   

Broken DLL                                                       03/14/09
CLOSED http://bugs.python.org/issue5489    created  JCoder                        
                                                                               

Broken DLL                                                       03/14/09
CLOSED http://bugs.python.org/issue5490    created  JCoder                        
                                                                               

Clarify contextlib.nested semantics                              03/15/09
CLOSED http://bugs.python.org/issue5491    created  ncoghlan                      
                                                                               

Error on leaving IDLE with quit() or exit() under Linux          03/15/09
       http://bugs.python.org/issue5492    created  gerluijten                    
                                                                               

Rephrasing the doc of object.__nonzero__                         03/15/09
CLOSED http://bugs.python.org/issue5493    created  ezio.melotti                  
                                                                               

Failure in test_httpservers on Linux                             03/15/09
       http://bugs.python.org/issue5494    created  gerluijten                    
                                                                               

ValueError exception of tuple.index(x) gives imprecise error mes 03/15/09
CLOSED http://bugs.python.org/issue5495    created  Retro                         
       patch                                                                   

codecs.lookup docstring is misleading                            03/15/09
CLOSED http://bugs.python.org/issue5496    created  exarkun                       
                                                                               

openssl compileerror with original source                        03/17/09
       http://bugs.python.org/issue5497    created  ocean-city                    
       patch                                                                   

Can SGMLParser properly handle <empty/> tags?                    03/17/09
       http://bugs.python.org/issue5498    created  once-off                      
                                                                               

only accept byte for getarg('c') and unicode for getarg('C')     03/17/09
       http://bugs.python.org/issue5499    created  haypo                         
       patch                                                                   

tarfile: path problem in arcname under windows                   03/17/09
       http://bugs.python.org/issue5500    created  ellioh                        
                                                                               

Update multiprocessing docs re: freeze_support                   03/17/09
       http://bugs.python.org/issue5501    created  bcorfman                      
                                                                               

io-c: TextIOWrapper is faster than BufferedReader but not protec 03/18/09
       http://bugs.python.org/issue5502    created  haypo                         
                                                                               

multiprocessing/connection.py wrong pipe name under win32        03/18/09
       http://bugs.python.org/issue5503    created  jqcn2003                      
                                                                               

ctypes should work with systems where mmap can't be PROT_WRITE a 03/18/09
       http://bugs.python.org/issue5504    created  agoode                        
       patch                                                                   

sys.stdin.read() doesn't return after first EOF on Windows       03/18/09
       http://bugs.python.org/issue5505    created  r_mosaic                      
                                                                               

io.BytesIO doesn't support the buffer protocol                   03/18/09
       http://bugs.python.org/issue5506    created  pitrou                        
                                                                               

ctypes configuration fails on mips-linux (and probably Irix)     03/18/09
       http://bugs.python.org/issue5507    created  doko                          
                                                                               

maximum recursion depth exceeded in __subclasscheck__            03/18/09
       http://bugs.python.org/issue5508    created  gangesmaster                  
                                                                               

cPickle - module object has no attribute                         03/18/09
CLOSED http://bugs.python.org/issue5509    created  ntroutman                     
                                                                               

patches for Modules/socketmodule.c for NetBSD                    03/18/09
       http://bugs.python.org/issue5510    created  yesmar                        
       patch                                                                   

zipfile - add __exit__ attribute to make ZipFile object compatib 03/18/09
       http://bugs.python.org/issue5511    created  petruchio                     
                                                                               

Streamline integer division                                      03/18/09
       http://bugs.python.org/issue5512    created  marketdickinson               
       patch                                                                   

"What's New" should say VERY CLEARLY that the type file	is gone  03/18/09
       http://bugs.python.org/issue5513    created  MLModel                       
                                                                               

Darwin framework libpython3.0.a is not a normal static library   03/18/09
       http://bugs.python.org/issue5514    created  jhowarth                      
                                                                               

'n' formatting for int and float handles leading zero padding po 03/19/09
       http://bugs.python.org/issue5515    created  eric.smith                    
                                                                               

equality not symmetric for subclasses of datetime.date and datet 03/19/09
       http://bugs.python.org/issue5516    created  jess.austin                   
                                                                               

2to3 haven't convert buffer object to memoryview                 03/19/09
CLOSED http://bugs.python.org/issue5517    created  bhy                           
                                                                               

cPickle produces inconsistent output                             03/19/09
       http://bugs.python.org/issue5518    created  rb                            
                                                                               

Deletion of some statements in re documentation                  03/19/09
       http://bugs.python.org/issue5519    created  MLModel                       
                                                                               

refactor test_datetime.py                                        03/19/09
       http://bugs.python.org/issue5520    created  jess.austin                   
       patch                                                                   

sqlite3.h missing                                                03/20/09
CLOSED http://bugs.python.org/issue5521    created  mendell                       
                                                                               

HTTPRedirectHandler documentation is wrong                       03/20/09
       http://bugs.python.org/issue5522    created  jkugler                       
                                                                               

Python bug tracker won't let you edit your profile               03/20/09
CLOSED http://bugs.python.org/issue5523    created  jkugler                       
                                                                               

execfile() removed from Python3                                  03/20/09
       http://bugs.python.org/issue5524    created  haypo                         
       patch                                                                   

Problem with email.MIME* library, using wrong new line           03/20/09
       http://bugs.python.org/issue5525    created  lclement                      
                                                                               

Local variables unavailable for operation of list comprehension  03/20/09
CLOSED http://bugs.python.org/issue5526    created  evan.greensmith               
                                                                               



Issues Now Closed (43)
______________________

Test issue                                                        563 days
       http://bugs.python.org/issue1064    loewis                        
       patch                                                                   

locale.format bug if thousand separator is space (french	separat  535 days
       http://bugs.python.org/issue1222    eric.smith                    
       patch                                                                   

doctest.testmod gets noisy if called more than once per SystemEx  456 days
       http://bugs.python.org/issue1611    ncoghlan                      
                                                                               

Implement __format__ for Decimal                                  270 days
       http://bugs.python.org/issue2110    marketdickinson               
       patch                                                                   

float compared to decimal is silently incorrect.                  352 days
       http://bugs.python.org/issue2531    jdunck                        
       easy                                                                    

Python 3.0b3 doesn't start on German Win XP SP3/SP2               185 days
       http://bugs.python.org/issue3820    pitrou                        
                                                                               

float(0.0) singleton                                              168 days
       http://bugs.python.org/issue4024    rhettinger                    
       patch                                                                   

traceback attribute error                                         166 days
       http://bugs.python.org/issue4034    haypo                         
       patch, needs review                                                     

open(): use keyword only for arguments other than file and mode   157 days
       http://bugs.python.org/issue4121    haypo                         
       patch, patch, needs review                                              

struct.pack('L', -1)                                              140 days
       http://bugs.python.org/issue4228    marketdickinson               
       patch                                                                   

Use 30-bit digits instead of 15-bit digits for Python integers.   136 days
       http://bugs.python.org/issue4258    marketdickinson               
       patch                                                                   

Idle - incorrectly displaying a character (Latin capital letter   126 days
       http://bugs.python.org/issue4281    vbr                           
                                                                               

PyUnicode_FromWideChar incorrect for characters outside the BMP   108 days
       http://bugs.python.org/issue4474    marketdickinson               
       patch                                                                   

exception traceback sometimes slow                                101 days
       http://bugs.python.org/issue4503    pitrou                        
                                                                               

Issue with RotatingFileHandler logging handler on Windows          81 days
       http://bugs.python.org/issue4749    neyro                         
                                                                               

IDLE startup file .Idle.py not documented                          28 days
       http://bugs.python.org/issue5276    georg.brandl                  
       patch                                                                   

optparse special usage tokens conflict with formatting character   20 days
       http://bugs.python.org/issue5374    andybuckley                   
       patch                                                                   

json needs object_pairs_hook                                       20 days
       http://bugs.python.org/issue5381    rhettinger                    
       patch                                                                   

stack overflow after hitting recursion limit twice                 13 days
       http://bugs.python.org/issue5392    pitrou                        
       patch                                                                   

os.read not handling O_DIRECT flag                                 18 days
       http://bugs.python.org/issue5396    exarkun                       
                                                                               

2to3 wrong for types.StringTypes                                   10 days
       http://bugs.python.org/issue5425    benjamin.peterson             
       patch                                                                   

cmpfunc in Python 3.0.1 windows installer                           3 days
       http://bugs.python.org/issue5431    tjreedy                       
                                                                               

Reference paragraph about the constructs that bind names needs u    5 days
       http://bugs.python.org/issue5469    georg.brandl                  
                                                                               

round(float, ndigits<0) sometimes rounds to odd                     3 days
       http://bugs.python.org/issue5473    marketdickinson               
                                                                               

document mistake xml.dom.minidom.Element                            4 days
       http://bugs.python.org/issue5478    georg.brandl                  
                                                                               

Expand Decimal.__format__() support to include "n"                  5 days
       http://bugs.python.org/issue5481    marketdickinson               
                                                                               

doc copyedits                                                       0 days
       http://bugs.python.org/issue5486    georg.brandl                  
       patch                                                                   

Parts of Tkinter missing (but not when running from IDLE)           0 days
       http://bugs.python.org/issue5487    loewis                        
                                                                               

nb_inplace_divide slot is missing in docs                           1 days
       http://bugs.python.org/issue5488    georg.brandl                  
       patch                                                                   

Broken DLL                                                          0 days
       http://bugs.python.org/issue5489    ocean-city                    
                                                                               

Broken DLL                                                          0 days
       http://bugs.python.org/issue5490    loewis                        
                                                                               

Clarify contextlib.nested semantics                                 1 days
       http://bugs.python.org/issue5491    georg.brandl                  
                                                                               

Rephrasing the doc of object.__nonzero__                            0 days
       http://bugs.python.org/issue5493    georg.brandl                  
                                                                               

ValueError exception of tuple.index(x) gives imprecise error mes    0 days
       http://bugs.python.org/issue5495    benjamin.peterson             
       patch                                                                   

codecs.lookup docstring is misleading                               0 days
       http://bugs.python.org/issue5496    georg.brandl                  
                                                                               

cPickle - module object has no attribute                            0 days
       http://bugs.python.org/issue5509    benjamin.peterson             
                                                                               

2to3 haven't convert buffer object to memoryview                    0 days
       http://bugs.python.org/issue5517    benjamin.peterson             
                                                                               

sqlite3.h missing                                                   0 days
       http://bugs.python.org/issue5521    benjamin.peterson             
                                                                               

Python bug tracker won't let you edit your profile                  0 days
       http://bugs.python.org/issue5523    benjamin.peterson             
                                                                               

Local variables unavailable for operation of list comprehension     0 days
       http://bugs.python.org/issue5526    benjamin.peterson             
                                                                               

dir() should only return strings                                 1603 days
       http://bugs.python.org/issue1056293 haypo                         
       patch                                                                   

Use \r\n, not \n for HTTP headers                                 849 days
       http://bugs.python.org/issue1597000 pitrou                        
       easy                                                                    

if something as x:                                                678 days
       http://bugs.python.org/issue1714448 mrabarnett                    
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 12 Allow auto-numbered replacement fields in str.format() strings    36 days
open    http://bugs.python.org/issue5237   

 11 "What's New" should say VERY CLEARLY that the type file	is gone    2 days
open    http://bugs.python.org/issue5513   

 11 RFC: improve distutils bdist_rpm so it builds pure python modul    7 days
open    http://bugs.python.org/issue5482   

 10 Streamline integer division                                        2 days
open    http://bugs.python.org/issue5512   

 10 contextlib.nested inconsistent with, well, nested with statemen   35 days
open    http://bugs.python.org/issue5251   

 10 if something as x:                                               678 days
closed  http://bugs.python.org/issue1714448

  9 rewrite of minidom.Node.normalize                                391 days
open    http://bugs.python.org/issue2170   

  8 merge json library with latest simplejson 2.0.x                    3 days
open    http://bugs.python.org/issue4136   

  7 mmap: read_byte/write_byte and object type                        20 days
open    http://bugs.python.org/issue5391   

  7 locale.format bug if thousand separator is space (french	separa  535 days
closed  http://bugs.python.org/issue1222   




From theller at ctypes.org  Fri Mar 20 20:00:46 2009
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 20 Mar 2009 20:00:46 +0100
Subject: [Python-Dev] Py_ssize_t support for ctypes arrays and pointers
Message-ID: <gq0p4u$8rf$1@ger.gmane.org>

I received some (so far unfinished) patches for ctypes
that will allow to create arrays with more than 2**31
elements and that will eventually also support pointer
offsets larger than int, on 64-bit platforms.

Since I do not have a machine with so much memory: Does one
of the buildbots allow to run tests for this feature, or
do I have to wait for the snakebite farm?

-- 
Thanks,
Thomas


From kristjan at ccpgames.com  Fri Mar 20 21:36:23 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Fri, 20 Mar 2009 20:36:23 +0000
Subject: [Python-Dev] Py_ssize_t support for ctypes arrays and pointers
In-Reply-To: <gq0p4u$8rf$1@ger.gmane.org>
References: <gq0p4u$8rf$1@ger.gmane.org>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F05068D4F5E@exchis.ccp.ad.local>

Will testing an array of chars do?
You can easily allocate 4-5Gb on a regular 64bit pc, even with only 1G of ram, given that your swap space is sufficient.
If you want to excercise your array, then you might get some paging, but it's not completely impossible.
K

-----Original Message-----
From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Thomas Heller
Sent: 20. mars 2009 19:01
To: python-dev at python.org
Subject: [Python-Dev] Py_ssize_t support for ctypes arrays and pointers

I received some (so far unfinished) patches for ctypes
that will allow to create arrays with more than 2**31
elements and that will eventually also support pointer
offsets larger than int, on 64-bit platforms.

Since I do not have a machine with so much memory: Does one
of the buildbots allow to run tests for this feature, or
do I have to wait for the snakebite farm?

-- 
Thanks,
Thomas

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com


From dreamingforward at gmail.com  Fri Mar 20 23:18:00 2009
From: dreamingforward at gmail.com (average)
Date: Fri, 20 Mar 2009 15:18:00 -0700
Subject: [Python-Dev] Core projects for Summer of Code
Message-ID: <913f9f570903201518l208576e2ndc4ffd75ce2eb16c@mail.gmail.com>

> Summer of Code is ramping up. ?Every year the common complaint is that not
> enough Python core projects get proposed by students, and of course a big
> reason for that is often the only encouragement we offer prospective
> students is a link to the PEP index.
>
> The challenge is finding project ideas for them that could reasonably occupy
> them for the entire Summer and which the results of their work can be
> demonstrated. ?They're being paid for specific projects so "Spend the Summer
> fixing bugs on the tracker" is a no-go, and Google has outlined that Summer
> of Code is about code, not documentation.

Improve doctest by allowing it to be aware of nested test scopes such
that a variable defined at "class-level scope" (i.e. the variable b
defined at the class-level doctest """>>> b=Bag("abacab")""") can be
used in "method-level scopes" without re-defining it every time for
each method's doctest (each method would reset the given variable (if
used) to its original state rather than live mutated between
equal-level scopes).

Would be a great improvement for doctest in my opinion--both in
ease-of-use, and reduction of redundant, error-prone ("did you define
your test variable the same in each method?") code)--as well as other
benefits.

Appreciate any consideration...

marcos

From lists at cheimes.de  Sat Mar 21 01:18:10 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 21 Mar 2009 01:18:10 +0100
Subject: [Python-Dev] Multiprocessing on Solaris
Message-ID: <gq1bo2$uov$1@ger.gmane.org>

Hello fellow co-developers!

Today I was in contact with a Python user who tried to compile
pyprocessing - the ancestor of multiprocessing - on Solaris. It failed
to run because Solaris is missing two features (HAVE_FD_TRANSFER and
HAVE_SEM_TIMEDWAIT). Does anybody have a Solaris box at his disposal to
test the settings? Neither Python 2.6 nor my backup have the correct
settings for Solaris.

Christian


From martin at v.loewis.de  Sat Mar 21 01:44:20 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Mar 2009 01:44:20 +0100
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <gq1bo2$uov$1@ger.gmane.org>
References: <gq1bo2$uov$1@ger.gmane.org>
Message-ID: <49C43864.6050808@v.loewis.de>

> Today I was in contact with a Python user who tried to compile
> pyprocessing - the ancestor of multiprocessing - on Solaris. It failed
> to run because Solaris is missing two features (HAVE_FD_TRANSFER and
> HAVE_SEM_TIMEDWAIT). Does anybody have a Solaris box at his disposal to
> test the settings? Neither Python 2.6 nor my backup have the correct
> settings for Solaris.

I don't quite understand what it is that you want tested - what
"settings"?

Most likely, the answer is yes, I can test stuff on Solaris (both SPARC
and x86/amd64).

Regards,
Martin

From jnoller at gmail.com  Sat Mar 21 01:49:47 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 20 Mar 2009 20:49:47 -0400
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <gq1bo2$uov$1@ger.gmane.org>
References: <gq1bo2$uov$1@ger.gmane.org>
Message-ID: <4222a8490903201749h5565bb96q52c90ed6edb57a64@mail.gmail.com>

Known issue:

http://bugs.python.org/issue3110

I haven't had time to look into it, I was planning on working on many
of the mp bugs during the sprint at pycon.

On Fri, Mar 20, 2009 at 8:18 PM, Christian Heimes <lists at cheimes.de> wrote:
> Hello fellow co-developers!
>
> Today I was in contact with a Python user who tried to compile
> pyprocessing - the ancestor of multiprocessing - on Solaris. It failed
> to run because Solaris is missing two features (HAVE_FD_TRANSFER and
> HAVE_SEM_TIMEDWAIT). Does anybody have a Solaris box at his disposal to
> test the settings? Neither Python 2.6 nor my backup have the correct
> settings for Solaris.
>
> Christian
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com
>

From lists at cheimes.de  Sat Mar 21 01:50:41 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 21 Mar 2009 01:50:41 +0100
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <49C43864.6050808@v.loewis.de>
References: <gq1bo2$uov$1@ger.gmane.org> <49C43864.6050808@v.loewis.de>
Message-ID: <49C439E1.4050403@cheimes.de>

Martin v. L?wis schrieb:
>> Today I was in contact with a Python user who tried to compile
>> pyprocessing - the ancestor of multiprocessing - on Solaris. It failed
>> to run because Solaris is missing two features (HAVE_FD_TRANSFER and
>> HAVE_SEM_TIMEDWAIT). Does anybody have a Solaris box at his disposal to
>> test the settings? Neither Python 2.6 nor my backup have the correct
>> settings for Solaris.
> 
> I don't quite understand what it is that you want tested - what
> "settings"?
> 
> Most likely, the answer is yes, I can test stuff on Solaris (both SPARC
> and x86/amd64).

According to the user's experience multiprocessing should not compile
and run correctly unless this patch is applied. I'm not sure if the
value "solaris" for platform is correct. You may also need to change
libraries to ['rt'].


Index: setup.py
===================================================================
--- setup.py    (revision 70478)
+++ setup.py    (working copy)
@@ -1280,6 +1280,14 @@
                 )
             libraries = []

+        elif platform == 'solaris':
+            macros = dict(
+                HAVE_SEM_OPEN=1,
+                HAVE_SEM_TIMEDWAIT=0,
+                HAVE_FD_TRANSFER=0,
+                )
+            libraries = []
+
         else:                                   # Linux and other unices
             macros = dict(
                 HAVE_SEM_OPEN=1,

From jnoller at gmail.com  Sat Mar 21 01:58:16 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 20 Mar 2009 20:58:16 -0400
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <49C439E1.4050403@cheimes.de>
References: <gq1bo2$uov$1@ger.gmane.org> <49C43864.6050808@v.loewis.de>
	<49C439E1.4050403@cheimes.de>
Message-ID: <4222a8490903201758m99339f1h8515bddfe09d570@mail.gmail.com>

On Fri, Mar 20, 2009 at 8:50 PM, Christian Heimes <lists at cheimes.de> wrote:
> Martin v. L?wis schrieb:
>>> Today I was in contact with a Python user who tried to compile
>>> pyprocessing - the ancestor of multiprocessing - on Solaris. It failed
>>> to run because Solaris is missing two features (HAVE_FD_TRANSFER and
>>> HAVE_SEM_TIMEDWAIT). Does anybody have a Solaris box at his disposal to
>>> test the settings? Neither Python 2.6 nor my backup have the correct
>>> settings for Solaris.
>>
>> I don't quite understand what it is that you want tested - what
>> "settings"?
>>
>> Most likely, the answer is yes, I can test stuff on Solaris (both SPARC
>> and x86/amd64).
>
> According to the user's experience multiprocessing should not compile
> and run correctly unless this patch is applied. I'm not sure if the
> value "solaris" for platform is correct. You may also need to change
> libraries to ['rt'].
>
>
> Index: setup.py
> ===================================================================
> --- setup.py ? ?(revision 70478)
> +++ setup.py ? ?(working copy)
> @@ -1280,6 +1280,14 @@
> ? ? ? ? ? ? ? ? )
> ? ? ? ? ? ? libraries = []
>
> + ? ? ? ?elif platform == 'solaris':
> + ? ? ? ? ? ?macros = dict(
> + ? ? ? ? ? ? ? ?HAVE_SEM_OPEN=1,
> + ? ? ? ? ? ? ? ?HAVE_SEM_TIMEDWAIT=0,
> + ? ? ? ? ? ? ? ?HAVE_FD_TRANSFER=0,
> + ? ? ? ? ? ? ? ?)
> + ? ? ? ? ? ?libraries = []
> +
> ? ? ? ? else: ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? # Linux and other unices
> ? ? ? ? ? ? macros = dict(
> ? ? ? ? ? ? ? ? HAVE_SEM_OPEN=1,

If this should be addressed in trunk/3k, we need to track this in the
tracker in the bug I cited in the other email. I can't speak for the
original pyprocessing code.

-jesse

From scott+python-dev at scottdial.com  Sat Mar 21 02:50:34 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Fri, 20 Mar 2009 21:50:34 -0400
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <4222a8490903201758m99339f1h8515bddfe09d570@mail.gmail.com>
References: <gq1bo2$uov$1@ger.gmane.org>
	<49C43864.6050808@v.loewis.de>	<49C439E1.4050403@cheimes.de>
	<4222a8490903201758m99339f1h8515bddfe09d570@mail.gmail.com>
Message-ID: <49C447EA.7000402@scottdial.com>

Jesse Noller wrote:
> On Fri, Mar 20, 2009 at 8:50 PM, Christian Heimes <lists at cheimes.de> wrote:
>> Martin v. L?wis schrieb:
>>>> Today I was in contact with a Python user who tried to compile
>>>> pyprocessing - the ancestor of multiprocessing - on Solaris. It failed
>>>> to run because Solaris is missing two features (HAVE_FD_TRANSFER and
>>>> HAVE_SEM_TIMEDWAIT). Does anybody have a Solaris box at his disposal to
>>>> test the settings? Neither Python 2.6 nor my backup have the correct
>>>> settings for Solaris.
>>> I don't quite understand what it is that you want tested - what
>>> "settings"?
>>>
>>> Most likely, the answer is yes, I can test stuff on Solaris (both SPARC
>>> and x86/amd64).
>> According to the user's experience multiprocessing should not compile
>> and run correctly unless this patch is applied. I'm not sure if the
>> value "solaris" for platform is correct. You may also need to change
>> libraries to ['rt'].
>>
>>
>> Index: setup.py
>> ===================================================================
>> --- setup.py    (revision 70478)
>> +++ setup.py    (working copy)
>> @@ -1280,6 +1280,14 @@
>>                 )
>>             libraries = []
>>
>> +        elif platform == 'solaris':
>> +            macros = dict(
>> +                HAVE_SEM_OPEN=1,
>> +                HAVE_SEM_TIMEDWAIT=0,
>> +                HAVE_FD_TRANSFER=0,
>> +                )
>> +            libraries = []
>> +
>>         else:                                   # Linux and other unices
>>             macros = dict(
>>                 HAVE_SEM_OPEN=1,
> 
> If this should be addressed in trunk/3k, we need to track this in the
> tracker in the bug I cited in the other email. I can't speak for the
> original pyprocessing code.
> 

I just checked out the trunk on a Sparc Solaris 8 box, and on the trunk,
those defines are specified differently:

building '_multiprocessing' extension
gcc -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC
-DHAVE_SEM_OPEN=1 -DHAVE_FD_TRANSFER=1 -DHAVE_SEM_TIMEDWAIT=1
-IModules/_multiprocessing -I. -I./Include -I/usr/local/include
-IInclude -I/nfs/nfs2/home/scratch/scodial/python-trunk -c
trunk/Modules/_multiprocessing/multiprocessing.c -o
build/temp.solaris-2.8-sun4u-2.7/trunk/Modules/_multiprocessing/multiprocessing.o

However, the build is still without issue:

trunk/Modules/_multiprocessing/multiprocessing.c: In function
`multiprocessing_sendfd':
trunk/Modules/_multiprocessing/multiprocessing.c:100: warning: implicit
declaration of function `CMSG_SPACE'
trunk/Modules/_multiprocessing/multiprocessing.c:117: warning: implicit
declaration of function `CMSG_LEN'

trunk/Modules/_multiprocessing/connection.h: In function `connection_new':
trunk/Modules/_multiprocessing/connection.h:51: warning: unknown
conversion type character `z' in format
trunk/Modules/_multiprocessing/connection.h:51: warning: too many
arguments for format
trunk/Modules/_multiprocessing/connection.h: In function `connection_repr':
trunk/Modules/_multiprocessing/connection.h:401: warning: unknown
conversion type character `z' in format

trunk/Modules/_multiprocessing/connection.h: In function `connection_new':
trunk/Modules/_multiprocessing/connection.h:51: warning: unknown
conversion type character `z' in format
trunk/Modules/_multiprocessing/connection.h:51: warning: too many
arguments for format
trunk/Modules/_multiprocessing/connection.h: In function `connection_repr':
trunk/Modules/_multiprocessing/connection.h:401: warning: unknown
conversion type character `z' in format

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From skip at pobox.com  Sat Mar 21 02:51:11 2009
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 20 Mar 2009 20:51:11 -0500
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <4222a8490903201749h5565bb96q52c90ed6edb57a64@mail.gmail.com>
References: <gq1bo2$uov$1@ger.gmane.org>
	<4222a8490903201749h5565bb96q52c90ed6edb57a64@mail.gmail.com>
Message-ID: <18884.18447.666826.52383@montanaro.dyndns.org>

    Jesse> Known issue:

    Jesse> http://bugs.python.org/issue3110

    Jesse> I haven't had time to look into it, I was planning on working on
    Jesse> many of the mp bugs during the sprint at pycon.

Jesse, I will be at the sprints for a couple days and should be able to test
things out on Solaris or let you look over my shoulder as we poke around the
machines at work if you need.

Skip

From jnoller at gmail.com  Sat Mar 21 03:43:06 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 20 Mar 2009 22:43:06 -0400
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <18884.18447.666826.52383@montanaro.dyndns.org>
References: <gq1bo2$uov$1@ger.gmane.org>
	<4222a8490903201749h5565bb96q52c90ed6edb57a64@mail.gmail.com>
	<18884.18447.666826.52383@montanaro.dyndns.org>
Message-ID: <4222a8490903201943y37d35ab5r3c9a275a0c938c64@mail.gmail.com>

On Fri, Mar 20, 2009 at 9:51 PM,  <skip at pobox.com> wrote:
> ? ?Jesse> Known issue:
>
> ? ?Jesse> http://bugs.python.org/issue3110
>
> ? ?Jesse> I haven't had time to look into it, I was planning on working on
> ? ?Jesse> many of the mp bugs during the sprint at pycon.
>
> Jesse, I will be at the sprints for a couple days and should be able to test
> things out on Solaris or let you look over my shoulder as we poke around the
> machines at work if you need.
>
> Skip
>

Sweet, do you think a 64 bit Opensolaris VM would work too?

From brett at python.org  Sat Mar 21 04:17:27 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 20 Mar 2009 20:17:27 -0700
Subject: [Python-Dev] What level of detail wanted for import and the
	language reference?
In-Reply-To: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
References: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
Message-ID: <bbaeab100903202017p7063306en829d469926e510b6@mail.gmail.com>

Doc changes are now checked in. Someone who has not been starting at import
for over two years should probably go in and clean it up as it is probably
not clear to a newbie (but then again newbies should not be reading the
language ref; more worried about the docs in sys).

On Mon, Mar 16, 2009 at 15:39, Brett Cannon <brett at python.org> wrote:

> At this point importlib is done for its public API for Python 3.1. That
> means it's time to turn my attention to making sure the semantics of import
> are well documented. But where to put all of the details? The language
> reference for import (
> http://docs.python.org/dev/py3k/reference/simple_stmts.html#the-import-statement)
> explains the basics, but is lacking all of the details of PEP 302 and other
> stuff like __path__ that have existed for ages.
>
> My question is if I should flesh out the details in the language reference
> or do it in importlib's intro docs. The main reason I could see not doing it
> in the langauge reference (or at least duplicating it) is it would be
> somewhat easier to reference specific objects in importlib but I am not sure
> if the language reference should try to stay away from stdlib references.
>
> -Brett
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090320/57b7c32d/attachment.htm>

From benjamin at python.org  Sat Mar 21 04:18:57 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 20 Mar 2009 22:18:57 -0500
Subject: [Python-Dev] What level of detail wanted for import and the
	language reference?
In-Reply-To: <bbaeab100903202017p7063306en829d469926e510b6@mail.gmail.com>
References: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com>
	<bbaeab100903202017p7063306en829d469926e510b6@mail.gmail.com>
Message-ID: <1afaf6160903202018w6a8885cfkdb4d94c67fffeb71@mail.gmail.com>

2009/3/20 Brett Cannon <brett at python.org>:
> Doc changes are now checked in. Someone who has not been starting at import
> for over two years should probably go in and clean it up as it is probably
> not clear to a newbie (but then again newbies should not be reading the
> language ref; more worried about the docs in sys).

It would be nice to have at least the sys docs backported to the trunk.



-- 
Regards,
Benjamin

From brett at python.org  Sat Mar 21 04:20:52 2009
From: brett at python.org (Brett Cannon)
Date: Fri, 20 Mar 2009 20:20:52 -0700
Subject: [Python-Dev] What level of detail wanted for import and the
	language reference?
In-Reply-To: <1afaf6160903202018w6a8885cfkdb4d94c67fffeb71@mail.gmail.com>
References: <bbaeab100903161539l3d73dc82pcbf204403067d856@mail.gmail.com> 
	<bbaeab100903202017p7063306en829d469926e510b6@mail.gmail.com> 
	<1afaf6160903202018w6a8885cfkdb4d94c67fffeb71@mail.gmail.com>
Message-ID: <bbaeab100903202020g126c55a5q3ba4e51015bd7943@mail.gmail.com>

On Fri, Mar 20, 2009 at 20:18, Benjamin Peterson <benjamin at python.org>wrote:

> 2009/3/20 Brett Cannon <brett at python.org>:
> > Doc changes are now checked in. Someone who has not been starting at
> import
> > for over two years should probably go in and clean it up as it is
> probably
> > not clear to a newbie (but then again newbies should not be reading the
> > language ref; more worried about the docs in sys).
>
> It would be nice to have at least the sys docs backported to the trunk.


That would also require backporting stuff from the glossary. In other words
I ain't doing it now, but you might be able to convince me at PyCon. I will
at least create a bug about it, though.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090320/067da17c/attachment-0001.htm>

From ncoghlan at gmail.com  Sat Mar 21 05:04:58 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 21 Mar 2009 14:04:58 +1000
Subject: [Python-Dev] Proposal: new list function: pack
In-Reply-To: <49C3B829.9060908@avl.com>
References: <fa7d4c4f0903200635w3bbc3c45m9a4202fd966ea769@mail.gmail.com>	<1900589.3551164.1237558955033.JavaMail.xicrypt@atgrzls001>
	<49C3B829.9060908@avl.com>
Message-ID: <49C4676A.9000903@gmail.com>

Hrvoje Niksic wrote:
> Looking at izip(*[iter(l)]*n), I tend to agree.

Note that the itertools recipes page in the docs includes the following:

def pairwise(iterable):
    "s -> (s0,s1), (s1,s2), (s2, s3), ..."
    a, b = tee(iterable)
    next(b, None)
    return izip(a, b)

There are a couple of other variants here:
http://code.activestate.com/recipes/439095/

And a different take on providing similar functionality here:
http://code.activestate.com/recipes/544296/

However, the idea of providing a general windowing function in itertools
has been considered in the past and emphatically rejected:
http://mail.python.org/pipermail/python-dev/2006-May/065305.html

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Sat Mar 21 07:45:23 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 21 Mar 2009 16:45:23 +1000
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
Message-ID: <49C48D03.7030709@gmail.com>

I really like the PEP - it's a solid extension of the ideas introduced
by PEP 342.

The two changes I would suggest is that the PEP be made more explicit
regarding the fact that the try/finally block only enclose the yield
expression itself (i.e. no other parts of the containing statement) and
that the caching comment be updated with a list of specific semantic
elements that the caching should not affect.

For the first part, I would prefer if the example was changed to use
capitals for the variant non-keyword parts of the statement:

  RESULT = yield from EXPR

And that it formally expanded to:

  _i = iter(EXPR)
  try:
      _u = _i.next()
      while 1:
          try:
              _v = yield _u
          except Exception, _e:
              _m = getattr(_i, 'throw', None)
              if _m is not None:
                  _u = _m(_e)
              else:
                  raise
          else:
              if _v is None:
                  _u = _i.next()
              else:
                  _u = _i.send(_v)
  except StopIteration, _e:
      _expr_result = _e.value
  finally:
      _m = getattr(_i, 'close', None)
      if _m is not None:
          _m()
  RESULT = _expr_result

I believe writing it that way would make it clearer that the scope of
the try/finally block doesn't include the assignment part of the statement.

For the second part, the specific semantics that I believe should be
noted as not changing even if an implementation chooses to cache the
bound methods are these:

- The "send" and "throw" methods of the subiterator should not be
retrieved if those methods are never called on the delegating generator

- If "send" is called on the delegating generator and the subiterator
has no "send" method, then an appropriate "AttributeError" should be
raised in the delegating generator

- If retrieving the "next", "send" or "throw" methods from the
subiterator results in an exception then the subiterator's "close"
method (if it has one) should still be called

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Sat Mar 21 10:13:06 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 21 Mar 2009 19:13:06 +1000
Subject: [Python-Dev] PEP 377 - allow __enter__() methods to skip the
 statement body
In-Reply-To: <A58DCFB4-C0EC-418C-A8A1-2BEB5AE06EC1@jwp.name>
References: <49BCFAEE.2070603@gmail.com> <49BD8317.3090704@voidspace.org.uk>	
	<49BE3B4F.3060300@gmail.com>	
	<ca471dc20903161001q38772b93wd7773c8a5fa6bd1e@mail.gmail.com>	
	<ca471dc20903161106h7440e2ecr70f1c0b852aac697@mail.gmail.com>	
	<d11dcfba0903161126y6fc234f9pb387f65cdcd7b605@mail.gmail.com>	
	<ca471dc20903161133g3afcae32q5500c94f6c924def@mail.gmail.com>	
	<49BEC6A1.3050504@gmail.com>	
	<ca471dc20903161457w2be669deud7b41abf57ecb56e@mail.gmail.com>	
	<49BED05B.1020504@gmail.com>
	<ca471dc20903161527m3252cf09u64d44ac20f73f4e9@mail.gmail.com>
	<49BED571.6070002@gmail.com>
	<A58DCFB4-C0EC-418C-A8A1-2BEB5AE06EC1@jwp.name>
Message-ID: <49C4AFA2.4010008@gmail.com>

James Pye wrote:
> The identification of this issue came from an *experiment* attempting to
> create a *single* "daemonized()" CM that would execute the
> with-statement's block in a new child process and, of course, not
> execute it in the parent. At first, I ran into the RuntimeError in the
> parent process, and then after rewriting the CMs as classes, I realized
> the futility.
> 
> with daemonized():
>  run_some_subprocess()
> 
> Of course it was all possible if I used the component CMs directly:
> 
> with parent_trap():
>  with fork_but_raise_in_parent():
>   run_some_subprocess()

When updating the PEP with the rejection notice, it occurred to me that
it is fairly easy to handle specific use cases like this reasonably
cleanly by including a callable in the design that is always used inline
in the body of the outermost with statement. For example:

  @contextmanager
  def make_daemon()
    class SkipInParent(Exception): pass
    def startd():
       # Fork process, then raise SkipInParent
       # in the parent process. The child process
       # continues running as a daemon.
    try:
      yield startd
    except SkipInParent:
      pass


  with make_daemon() as startd:
    startd()
    # Daemon code goes here

With that approach, since it is startd() that raises the exception
rather than __enter__() then __exit__() will always be given the chance
to suppress it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Sat Mar 21 13:27:46 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 21 Mar 2009 12:27:46 +0000 (UTC)
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
References: <49C48D03.7030709@gmail.com>
Message-ID: <loom.20090321T122452-862@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:
> 
> And that it formally expanded to:
> 
> <snip 20 lines of code with multiple try/except/finally clauses and various
conditionals>

Do we really want to add a syntactic feature which has such a complicated
expansion? I fear it will make code using "yield from" much more difficult to
understand and audit.



From ncoghlan at gmail.com  Sat Mar 21 14:49:29 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 21 Mar 2009 23:49:29 +1000
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <loom.20090321T122452-862@post.gmane.org>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
Message-ID: <49C4F069.9060606@gmail.com>

Antoine Pitrou wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>> And that it formally expanded to:
>>
>> <snip 20 lines of code with multiple try/except/finally clauses and various
> conditionals>
> 
> Do we really want to add a syntactic feature which has such a complicated
> expansion? I fear it will make code using "yield from" much more difficult to
> understand and audit.

Yes, I think we do. The previous argument against explicit syntactic
support for invoking subiterators was that it was trivial to do so by
iterating over the subiterator and yielding each item in turn.

With the additional generator features introduced by PEP 342, that is no
longer the case: as described in Greg's PEP, simple iteration doesn't
support send() and throw() correctly. The gymnastics needed to support
send() and throw() actually aren't that complex when you break them
down, but they aren't trivial either.

Whether or not different people will find code using "yield from"
difficult to understand or not will have more to do with their grasp of
the concepts of cooperative multitasking in general more so than the
underlying trickery involved in allowing truly nested generators.

Here's an annotated version of the expansion that will hopefully make
things clearer:

  # Create the subiterator
  _i = iter(EXPR)
  # Outer try block serves two purposes:
  #  - retrieve expression result from StopIteration instance
  #  - ensure _i.close() is called if it exists
  try:
      # Get first value to be yielded
      _u = _i.next()
      while 1:
          # Inner try block allows exceptions passed in via
          # the generator's throw() method to be passed to
          # the subiterator
          try:
              _v = yield _u
          except Exception, _e:
              # An exception was thrown into this
              # generator. If the subiterator has
              # a throw() method, then we pass the
              # exception down. Otherwise, we
              # propagate the exception in the
              # current generator
              # Note that SystemExit and
              # GeneratorExit are never passed down.
              # For those, we rely on the close()
              # call in the outer finally block
              _m = getattr(_i, 'throw', None)
              if _m is not None:
                  # throw() will either yield
                  # a new value, raise StopIteration
                  # or reraise the original exception
                  _u = _m(_e)
              else:
                  raise
          else:
              if _v is None:
                  # Get the next subiterator value
                  _u = _i.next()
              else:
                  # A value was passed in using
                  # send(), so attempt to pass it
                  # down to the subiterator.
                  # AttributeError will be raised
                  # if the subiterator doesn't
                  # provide a send() method
                  _u = _i.send(_v)
  except StopIteration, _e:
      # Subiterator ended, get the expression result
      _expr_result = _e.value
  finally:
      # Ensure close() is called if it exists
      _m = getattr(_i, 'close', None)
      if _m is not None:
          _m()
  RESULT = _expr_result


On further reflection (and after reading a couple more posts on
python-ideas relating to this PEP), I have two more questions/concerns:

1. The inner try/except is completely pointless if the subiterator
doesn't have a throw() method. Would it make sense to have two versions
of the inner loop (with and without the try block) and choose which one
to use based on whether or not the subiterator has a throw() method?
(Probably not, since this PEP is mainly about generators as cooperative
pseudo-threads and in such situations all iterators involved are likely
to be generators and hence have throw() methods. However, I think the
question is at least worth thinking about.)

2. Due to a couple of bug reports against 2.5,
contextlib.GeneratorContextManager now takes extra care when handling
exceptions to avoid accidentally suppressing explicitly thrown in
StopIteration instances. However, the current expansion in PEP 380
doesn't check if the StopIteration caught by the outer try statement was
one that was originally thrown into the generator rather than an
indicator that the subiterator naturally reached the end of its
execution. That isn't a difficult behaviour to eliminate, but it does
require a slight change to the semantic definition of the new expression:

  _i = iter(EXPR)
  _thrown_exc = None
  try:
      _u = _i.next()
      while 1:
          try:
              _v = yield _u
          except Exception, _e:
              _thrown_exc = _e
              _m = getattr(_i, 'throw', None)
              if _m is not None:
                  _u = _m(_e)
              else:
                  raise
          else:
              if _v is None:
                  _u = _i.next()
              else:
                  _u = _i.send(_v)
  except StopIteration, _e:
      if _e is _thrown_exc:
          # Don't suppress StopIteration if it
          # was thrown in from outside the
          # generator
          raise
      _expr_result = _e.value
  finally:
      _m = getattr(_i, 'close', None)
      if _m is not None:
          _m()
  RESULT = _expr_result

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Sat Mar 21 15:18:03 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 21 Mar 2009 14:18:03 +0000 (UTC)
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C4F069.9060606@gmail.com>
Message-ID: <loom.20090321T135228-436@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:
> 
> Whether or not different people will find code using "yield from"
> difficult to understand or not will have more to do with their grasp of
> the concepts of cooperative multitasking in general more so than the
> underlying trickery involved in allowing truly nested generators.

I don't agree. Cooperative multitasking looks quite orthogonal to me to the
complexity brought by this new statement. You can perfectly well "grasp the
concepts of cooperative multitasking" without finding the semantics of this new
statement easy to understand and remember. Hiding so many special cases behind a
one-line statement does not help, IMO. And providing a commented version of the
expansion does not really help either: it does not make the expansion easier to
remember and replay in the case you have to debug something involving such a
"yield from" statement.

(remember, by the way, that a third-party package like greenlets already
provides cooperative multitasking without any syntax addition, and that
libraries like Twisted already have their own generator-based solution for
cooperative multitasking, which AFAIR no one demonstrated would be improved by
the new statement. I'm not sure where the urgency is, and I don't see any
compelling use case.)



From pje at telecommunity.com  Sat Mar 21 17:25:05 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 21 Mar 2009 12:25:05 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49C48D03.7030709@gmail.com>
References: <49C48D03.7030709@gmail.com>
Message-ID: <20090321162244.ED2423A4062@sparrow.telecommunity.com>

At 04:45 PM 3/21/2009 +1000, Nick Coghlan wrote:
>I really like the PEP - it's a solid extension of the ideas introduced
>by PEP 342.

(Replying to you since I haven't seen any other thread on this)

My concern is that allowing 'return value' in generators is going to 
be confusing, since it effectively causes the return value to 
"disappear" if you're not using it in this special way with some 
framework that takes advantage.

However, if you *do* have some framework that takes advantage of 
generators to do microthreads, then it is most likely already written 
so as to have things like 'yield Return(value)' to signal a return, 
and to handle 'yield subgenerator()' without the use of additional syntax.

So, I don't really see the point of the PEP.  'yield from' seems 
marginally useful, but I really dislike making it an expression, 
rather than a statement.  The difference seems just a little too 
subtle, considering how radically different the behavior 
is.   Overall, it has the feel of jamming a framework into the 
language, when doing the same thing in a library is pretty trivial.

I'd almost rather see a standard or "reference" trampoline added to 
the stdlib (preferably with a way to register handling for 
specialized yielded types IO/scheduling hooks), than try to cram half 
a trampoline into the language itself.


From greg.ewing at canterbury.ac.nz  Sat Mar 21 22:54:18 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 22 Mar 2009 09:54:18 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <loom.20090321T122452-862@post.gmane.org>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
Message-ID: <49C5620A.4080908@canterbury.ac.nz>

Antoine Pitrou wrote:

> Do we really want to add a syntactic feature which has such a complicated
> expansion? I fear it will make code using "yield from" much more difficult to
> understand and audit.

As I've said before, I don't think the feature itself is
difficult to understand. You're not meant to learn about
it by reading the expansion -- that's only there to pin
down all the details for language lawyers.

For humans, almost all the important information is
contained in one paragraph near the top:

"When the iterator is another generator, the effect is the same as if
the body of the subgenerator were inlined at the point of the ``yield
from`` expression. Furthermore, the subgenerator is allowed to execute
a ``return`` statement with a value, and that value becomes the value of
the ``yield from`` expression."

Armed with this perspective, do you still think there will
be difficulty in understanding or auditing code?

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sat Mar 21 23:21:38 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 22 Mar 2009 10:21:38 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090321162244.ED2423A4062@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>
	<20090321162244.ED2423A4062@sparrow.telecommunity.com>
Message-ID: <49C56872.9020609@canterbury.ac.nz>

P.J. Eby wrote:

> My concern is that allowing 'return value' in generators is going to be 
> confusing, since it effectively causes the return value to "disappear" 
> if you're not using it in this special way with some framework that 
> takes advantage.

But part of all this is that you *don't* need a special
framework to get the return value -- all you need is a
caller that uses a yield-from statement. There are uses
for that besides threading systems.

-- 
Greg

From p.f.moore at gmail.com  Sat Mar 21 23:59:47 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 21 Mar 2009 22:59:47 +0000
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49C56872.9020609@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<20090321162244.ED2423A4062@sparrow.telecommunity.com>
	<49C56872.9020609@canterbury.ac.nz>
Message-ID: <79990c6b0903211559t5cf32894tf3d56f262b556b4@mail.gmail.com>

2009/3/21 Greg Ewing <greg.ewing at canterbury.ac.nz>:
> P.J. Eby wrote:
>
>> My concern is that allowing 'return value' in generators is going to be
>> confusing, since it effectively causes the return value to "disappear" if
>> you're not using it in this special way with some framework that takes
>> advantage.
>
> But part of all this is that you *don't* need a special
> framework to get the return value -- all you need is a
> caller that uses a yield-from statement. There are uses
> for that besides threading systems.

Can they be added to the PEP? Personally, I find the proposal
appealing, and I don't find the semantics hard to understand (although
certainly the expansion given in the "formal semantics" section makes
my head hurt ;-)) but I don't see many actual reasons why it's useful.
(My own use would most likely to be the trivial "for v in g: yield v"
case). More motivating examples would help a lot.

Paul.

From pje at telecommunity.com  Sun Mar 22 01:52:10 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 21 Mar 2009 20:52:10 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49C56872.9020609@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<20090321162244.ED2423A4062@sparrow.telecommunity.com>
	<49C56872.9020609@canterbury.ac.nz>
Message-ID: <20090322004950.A56AC3A4062@sparrow.telecommunity.com>

At 10:21 AM 3/22/2009 +1200, Greg Ewing wrote:
>P.J. Eby wrote:
>
>>My concern is that allowing 'return value' in generators is going 
>>to be confusing, since it effectively causes the return value to 
>>"disappear" if you're not using it in this special way with some 
>>framework that takes advantage.
>
>But part of all this is that you *don't* need a special
>framework to get the return value -- all you need is a
>caller that uses a yield-from statement. There are uses
>for that besides threading systems.

Such as?  I've been wracking my brain trying to come up with any 
*other* occasion where I'd need -- or even find it useful -- to have 
one generator yield the contents of another generator to its caller, 
and then use a separate return value in itself.  (I'm thus finding it 
hard to believe there's a non-contrived example that's not doing I/O, 
scheduling, or some other form of co-operative multitasking.)

In any case, you didn't address the confusion issue: the inability of 
generators to return a value is there for a good reason, and adding a 
return value that doesn't actually return anywhere unless you use it 
in a yield-from expression -- an expression that both looks like a 
statement and has control-flow side-effects -- seems both 
over-complex and an invitation to confusion.

This is different from plain yield expressions, in that plain yield 
expressions are *symmetric*: the value returned from the yield 
expression comes from the place where control flow is passed by the 
yield.  That is, 'x = yield y' takes value y, passes control flow to 
the caller, and then returns a result from the caller.  It's like an 
inverse function call.  'x = yield from y', on the other hand, first 
passes control to y, then the caller, then y, then the caller, an 
arbitrary number of times, and then finally returns a value from y, 
not the caller.

This is an awful lot of difference in control flow for only a slight 
change in syntax -- much more of a difference than the difference 
between yield statements and yield expressions.

So at present (for whatever those opinions are worth), I'd say -0 on 
a yield-from *statement* (somewhat useful but maybe not worth 
bothering with), +0 on a reference trampoline in the stdlib (slightly 
better than doing nothing at all, but not by much), and -1 on 
yield-from expressions and return values (confusing complication with 
very narrowly focused benefit, reasonably doable with library code).


From python-dev-list at trentnelson.com  Sun Mar 22 06:36:50 2009
From: python-dev-list at trentnelson.com (Trent Nelson)
Date: Sun, 22 Mar 2009 05:36:50 +0000
Subject: [Python-Dev] Py_ssize_t support for ctypes arrays and pointers
In-Reply-To: <gq0p4u$8rf$1@ger.gmane.org>
References: <gq0p4u$8rf$1@ger.gmane.org>
Message-ID: <20090322053650.GA30571@wind.teleri.net>

On Fri, Mar 20, 2009 at 08:00:46PM +0100, Thomas Heller wrote:
> Since I do not have a machine with so much memory: Does one
> of the buildbots allow to run tests for this feature, or
> do I have to wait for the snakebite farm?

    Will you be at PyCon?  The wait might not be as bad as you think ;-)

        Trent.

From martin at v.loewis.de  Sun Mar 22 08:00:44 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 22 Mar 2009 08:00:44 +0100
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <49C439E1.4050403@cheimes.de>
References: <gq1bo2$uov$1@ger.gmane.org> <49C43864.6050808@v.loewis.de>
	<49C439E1.4050403@cheimes.de>
Message-ID: <49C5E21C.5040607@v.loewis.de>

> According to the user's experience multiprocessing should not compile
> and run correctly unless this patch is applied.

Can this please be more qualified? I can confirm Scott's observation:
for the trunk, it compiles just fine, using SunPro CC on Solaris 10,
on SPARC. Also, test_multiprocessing passes.

IIUC, incorrect setting of HAVE_SEM_TIMEDWAIT to 1 should cause a
compile failure, as sem_timedwait would be called but not defined.
However, sem_timedwait *is* defined on Solaris.

Likewise, for HAVE_FD_TRANSFER=1 to work, SCM_RIGHTS must be defined
(and implemented); it is defined in sys/socket.h, and documented in
socket.h(3HEAD).

So there must be going on something else at the user's machine.

Regards,
Martin

From greg.ewing at canterbury.ac.nz  Sun Mar 22 09:11:29 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 22 Mar 2009 20:11:29 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090322004950.A56AC3A4062@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>
	<20090321162244.ED2423A4062@sparrow.telecommunity.com>
	<49C56872.9020609@canterbury.ac.nz>
	<20090322004950.A56AC3A4062@sparrow.telecommunity.com>
Message-ID: <49C5F2B1.4020503@canterbury.ac.nz>

P.J. Eby wrote:

> (I'm thus finding it hard 
> to believe there's a non-contrived example that's not doing I/O, 
> scheduling, or some other form of co-operative multitasking.)

Have you seen my xml parser example?

http://www.cosc.canterbury.ac.nz/greg.ewing/python/yield-from/

Whether you'll consider it contrived or not I don't know
(contrivedness being such a subjective property) but it
illustrates the style of programming I'm trying to support
with the return-value feature.

> In any case, you didn't address the confusion issue: the inability of 
> generators to return a value is there for a good reason,

It's there because formerly there was nowhere for the
return value to go. If there is somewhere for it to go,
the restriction will no longer be needed.

Things like this have happened before. It used to be
forbidden to put a yield in a try-finally block, because
there was no way to ensure that the finally would be
executed. Once a way was found to do that, the restriction
was lifted.

As for confusion, we ignore the return values of function
calls all the time, without worrying that someone might be
confused by the fact that their return value doesn't go
anywhere. And that's the right way to think of a yield-from
expression -- as a kind of function call, not a kind of yield.

If there's anything confusing, it's the presence of the
word 'yield'. Its only virtue is that it gives a clue that
the construct has something to do with generators, but
you'll have to RTM to find out exactly what. Nobody has
thus far suggested any better name, however.

-- 
Greg

From rdmurray at bitdance.com  Sun Mar 22 15:59:00 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 22 Mar 2009 10:59:00 -0400 (EDT)
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
Message-ID: <Pine.LNX.4.64.0903221051280.31981@kimball.webabinitio.net>

On Sun Mar 22 09:11:29 CET 2009 Greg Ewing wrote:
> As for confusion, we ignore the return values of function
> calls all the time, without worrying that someone might be
> confused by the fact that their return value doesn't go
> anywhere. And that's the right way to think of a yield-from
> expression -- as a kind of function call, not a kind of yield.
> 
> If there's anything confusing, it's the presence of the
> word 'yield'. Its only virtue is that it gives a clue that
> the construct has something to do with generators, but
> you'll have to RTM to find out exactly what. Nobody has
> thus far suggested any better name, however.

The PEP doesn't seem to contain a list of unacceptable names and reasons
in the section where the problem with the name is mentioned.  So I wonder
why the obvious 'delegate to' is not acceptable?  (Because adding
a keyword would make it less likely to be accepted?)

--
R. David Murray           http://www.bitdance.com

From solipsis at pitrou.net  Sun Mar 22 16:09:27 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 22 Mar 2009 15:09:27 +0000 (UTC)
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
Message-ID: <loom.20090322T150537-254@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> 
> "When the iterator is another generator, the effect is the same as if
> the body of the subgenerator were inlined at the point of the ``yield
> from`` expression. Furthermore, the subgenerator is allowed to execute
> a ``return`` statement with a value, and that value becomes the value of
> the ``yield from`` expression."

If it's really enough to understand and debug all corner cases of using "yield
from", then fair enough.
(I still don't like the PEP and feel it's much too specialized for a new
syntactic feature. The language should try to be obvious rather than clever, IMO)



From tismer at stackless.com  Sun Mar 22 17:38:37 2009
From: tismer at stackless.com (Christian Tismer)
Date: Sun, 22 Mar 2009 17:38:37 +0100
Subject: [Python-Dev] an unimportant question, ...
Message-ID: <49C6698D.4060700@stackless.com>

... but I'm curious.

Hi Guido,

while working on Psyco, I stumbled over a log entry in modsupport.h:

>
>    19-Aug-2002  GvR	1012	Changes to string object struct for
>    				interning changes, saving 3 bytes.
>

The change to stringobject was this  (rev. 28308):

Before:
> typedef struct {
>     PyObject_VAR_HEAD
>     long ob_shash;
>     PyObject *ob_sinterned;
>     char ob_sval[1];
> } PyStringObject;

After:
> typedef struct {
>     PyObject_VAR_HEAD
>     long ob_shash;
>     int ob_sstate;
>     char ob_sval[1];
> } PyStringObject;

Now, the internals are very clear to me. What I don't understand
is where the three saved bytes should be.

Thinking of the time where this change was made, I cannot imagine
that this comment was about the size diff between pointer and int,
and if this was meant, I still don't get how this could save three
bytes?

With unaligned ob_sval, structure packing and ob_sstate being
unsigned char one could save 3 bytes, but we don't do that.

Well, as said, this is no important question. I am just asking
myself what I don't see here, or if the comment is just sub-optimal :-)

all the best -- chris


p.s.: won't make it to PyCon this time, see you soon at the piggies
-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/

From pje at telecommunity.com  Sun Mar 22 18:24:34 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sun, 22 Mar 2009 13:24:34 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49C5F2B1.4020503@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<20090321162244.ED2423A4062@sparrow.telecommunity.com>
	<49C56872.9020609@canterbury.ac.nz>
	<20090322004950.A56AC3A4062@sparrow.telecommunity.com>
	<49C5F2B1.4020503@canterbury.ac.nz>
Message-ID: <20090322172216.B01713A4097@sparrow.telecommunity.com>

At 08:11 PM 3/22/2009 +1200, Greg Ewing wrote:
>P.J. Eby wrote:
>
>>(I'm thus finding it hard to believe there's a non-contrived 
>>example that's not doing I/O, scheduling, or some other form of 
>>co-operative multitasking.)
>
>Have you seen my xml parser example?
>
>http://www.cosc.canterbury.ac.nz/greg.ewing/python/yield-from/
>
>Whether you'll consider it contrived or not I don't know
>(contrivedness being such a subjective property) but it
>illustrates the style of programming I'm trying to support
>with the return-value feature.

I find the parser *without* yield-from to be much easier to follow 
what's going on, actually...  and don't see what benefit was obtained 
by the additional complication of using send().


>>In any case, you didn't address the confusion issue: the inability 
>>of generators to return a value is there for a good reason,
>
>It's there because formerly there was nowhere for the
>return value to go. If there is somewhere for it to go,
>the restriction will no longer be needed.

But that's begging the question (in the original meaning of the 
phrase) of why we *want* to have two ways to return data from a generator.


>As for confusion, we ignore the return values of function
>calls all the time, without worrying that someone might be
>confused by the fact that their return value doesn't go
>anywhere. And that's the right way to think of a yield-from
>expression -- as a kind of function call, not a kind of yield.

But it's not a function call -- it's multiple *inverted* function 
calls, followed by special handling of the last iteration of the 
iterator it takes.

The control flow is also hard to explain, as is the implementation.


>If there's anything confusing, it's the presence of the
>word 'yield'. Its only virtue is that it gives a clue that
>the construct has something to do with generators, but
>you'll have to RTM to find out exactly what. Nobody has
>thus far suggested any better name, however.

Perhaps this is because it's not that interesting of a feature.  As I 
said, I wouldn't fight a yield-from statement without all this 
return-value stuff, although it still seems like too much trouble to me.


From fuzzyman at voidspace.org.uk  Sun Mar 22 18:38:54 2009
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Sun, 22 Mar 2009 17:38:54 +0000
Subject: [Python-Dev] an unimportant question, ...
In-Reply-To: <49C6698D.4060700@stackless.com>
References: <49C6698D.4060700@stackless.com>
Message-ID: <49C677AE.1010802@voidspace.org.uk>

Christian Tismer wrote:
> ... but I'm curious.
>
> Hi Guido,
>
> while working on Psyco, I stumbled over a log entry in modsupport.h:
>
>>
>>    19-Aug-2002  GvR    1012    Changes to string object struct for
>>                    interning changes, saving 3 bytes.
>>
>
> The change to stringobject was this  (rev. 28308):
>
> Before:
>> typedef struct {
>>     PyObject_VAR_HEAD
>>     long ob_shash;
>>     PyObject *ob_sinterned;
>>     char ob_sval[1];
>> } PyStringObject;
>
> After:
>> typedef struct {
>>     PyObject_VAR_HEAD
>>     long ob_shash;
>>     int ob_sstate;
>>     char ob_sval[1];
>> } PyStringObject;
>
> Now, the internals are very clear to me. What I don't understand
> is where the three saved bytes should be.
>
> Thinking of the time where this change was made, I cannot imagine
> that this comment was about the size diff between pointer and int,
> and if this was meant, I still don't get how this could save three
> bytes?
>
> With unaligned ob_sval, structure packing and ob_sstate being
> unsigned char one could save 3 bytes, but we don't do that.
>
> Well, as said, this is no important question. I am just asking
> myself what I don't see here, or if the comment is just sub-optimal :-)
>

At Resolver we've found it useful to short-circuit any doubt and just 
refer to comments in code as 'lies'. :-)

Michael

> all the best -- chris
>
>
> p.s.: won't make it to PyCon this time, see you soon at the piggies


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From martin at v.loewis.de  Sun Mar 22 20:01:36 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 22 Mar 2009 20:01:36 +0100
Subject: [Python-Dev] an unimportant question, ...
In-Reply-To: <49C6698D.4060700@stackless.com>
References: <49C6698D.4060700@stackless.com>
Message-ID: <49C68B10.4040107@v.loewis.de>

> Now, the internals are very clear to me. What I don't understand
> is where the three saved bytes should be.

If you look at the various patches in

http://bugs.python.org/issue576101

then there is a three-byte saving in all versions from 1 to 6.
Consequentially, the API was changed in those versions (but only
starting from version 5, i.e. the first version created by Guido).

For some reason, the saving was then removed from the patch that
got actually committed (#7). I guess the comment just stayed.

Regards,
Martin

From tismer at stackless.com  Sun Mar 22 20:23:27 2009
From: tismer at stackless.com (Christian Tismer)
Date: Sun, 22 Mar 2009 20:23:27 +0100
Subject: [Python-Dev] an unimportant question, ...
In-Reply-To: <49C68B10.4040107@v.loewis.de>
References: <49C6698D.4060700@stackless.com> <49C68B10.4040107@v.loewis.de>
Message-ID: <49C6902F.9070601@stackless.com>

On 3/22/09 8:01 PM, "Martin v. L?wis" wrote:
>> Now, the internals are very clear to me. What I don't understand
>> is where the three saved bytes should be.
>
> If you look at the various patches in
>
> http://bugs.python.org/issue576101
>
> then there is a three-byte saving in all versions from 1 to 6.
> Consequentially, the API was changed in those versions (but only
> starting from version 5, i.e. the first version created by Guido).
>
> For some reason, the saving was then removed from the patch that
> got actually committed (#7). I guess the comment just stayed.

That's very impressive!
Thank you very much for the enlightment.
Whow!

cheers - chris
-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/

From lists at cheimes.de  Sun Mar 22 20:36:19 2009
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 22 Mar 2009 20:36:19 +0100
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <49C5E21C.5040607@v.loewis.de>
References: <gq1bo2$uov$1@ger.gmane.org> <49C43864.6050808@v.loewis.de>
	<49C439E1.4050403@cheimes.de> <49C5E21C.5040607@v.loewis.de>
Message-ID: <49C69333.9000806@cheimes.de>

Martin v. L?wis schrieb:
>> According to the user's experience multiprocessing should not compile
>> and run correctly unless this patch is applied.
> 
> Can this please be more qualified? I can confirm Scott's observation:
> for the trunk, it compiles just fine, using SunPro CC on Solaris 10,
> on SPARC. Also, test_multiprocessing passes.
> 
> IIUC, incorrect setting of HAVE_SEM_TIMEDWAIT to 1 should cause a
> compile failure, as sem_timedwait would be called but not defined.
> However, sem_timedwait *is* defined on Solaris.
> 
> Likewise, for HAVE_FD_TRANSFER=1 to work, SCM_RIGHTS must be defined
> (and implemented); it is defined in sys/socket.h, and documented in
> socket.h(3HEAD).
> 
> So there must be going on something else at the user's machine.

The user doesn't respond to my inquiries anymore. According to his
initial message he is using Solaris 5.8 with GCC 3.4.6 on a Sun Fire
machine. Here is a link to his mesage:
http://permalink.gmane.org/gmane.comp.python.general/615802

Christian


From tismer at stackless.com  Sun Mar 22 20:38:29 2009
From: tismer at stackless.com (Christian Tismer)
Date: Sun, 22 Mar 2009 20:38:29 +0100
Subject: [Python-Dev] an unimportant question, ...
In-Reply-To: <49C68B10.4040107@v.loewis.de>
References: <49C6698D.4060700@stackless.com> <49C68B10.4040107@v.loewis.de>
Message-ID: <49C693B5.8070309@stackless.com>

On 3/22/09 8:01 PM, "Martin v. L?wis" wrote:
>> Now, the internals are very clear to me. What I don't understand
>> is where the three saved bytes should be.
>
> If you look at the various patches in
>
> http://bugs.python.org/issue576101
>
> then there is a three-byte saving in all versions from 1 to 6.
> Consequentially, the API was changed in those versions (but only
> starting from version 5, i.e. the first version created by Guido).
>
> For some reason, the saving was then removed from the patch that
> got actually committed (#7). I guess the comment just stayed.

Yes, funny, actually. At least, I don't find any comment why
the char was turned into an int, after all.
Are char pointers not on a word boundary problematic on some
platforms?

Or was it maybe to just keep the string layout on many
common platforms compatible, in order to save rebuilding
so many windows extension modules?

If the latter is true and the only reason, I vote for reclaiming
the three bytes. Maybe it saves a tree or two. Maybe it hurts
very little if done for Python 3000.

In any case, use the version that saves the most energy. :-)

not kidding - ciao -- chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/

From thomas at python.org  Sun Mar 22 20:48:46 2009
From: thomas at python.org (Thomas Wouters)
Date: Sun, 22 Mar 2009 20:48:46 +0100
Subject: [Python-Dev] an unimportant question, ...
In-Reply-To: <49C693B5.8070309@stackless.com>
References: <49C6698D.4060700@stackless.com> <49C68B10.4040107@v.loewis.de>
	<49C693B5.8070309@stackless.com>
Message-ID: <9e804ac0903221248r6cb753d6i7ffb2c02ca4e587d@mail.gmail.com>

On Sun, Mar 22, 2009 at 20:38, Christian Tismer <tismer at stackless.com>wrote:

> On 3/22/09 8:01 PM, "Martin v. L?wis" wrote:
>
>> Now, the internals are very clear to me. What I don't understand
>>> is where the three saved bytes should be.
>>>
>>
>> If you look at the various patches in
>>
>> http://bugs.python.org/issue576101
>>
>> then there is a three-byte saving in all versions from 1 to 6.
>> Consequentially, the API was changed in those versions (but only
>> starting from version 5, i.e. the first version created by Guido).
>>
>> For some reason, the saving was then removed from the patch that
>> got actually committed (#7). I guess the comment just stayed.
>>
>
> Yes, funny, actually. At least, I don't find any comment why
> the char was turned into an int, after all.
> Are char pointers not on a word boundary problematic on some
> platforms?
>
> Or was it maybe to just keep the string layout on many
> common platforms compatible, in order to save rebuilding
> so many windows extension modules?
>
> If the latter is true and the only reason, I vote for reclaiming
> the three bytes. Maybe it saves a tree or two. Maybe it hurts
> very little if done for Python 3000.
>
> In any case, use the version that saves the most energy. :-)
>
> not kidding - ciao -- chris


Actually, we should use the version that breaks the ABI the least. The
change you're referring to actually (comparatively silently!) broke the ABI
for 64-bit Python builds (at least, on LP64 and LLP64 systems, which is most
of them.)  You can normally relatively safely use an extension module built
for Python 2.2 in Python 2.4, but not so for 64-bit builds. Let's try to
keep the ABI compatible, and at least make it an error (not a warning) to
load a truly incompatible ABI version.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090322/637c8201/attachment.htm>

From solipsis at pitrou.net  Sun Mar 22 20:56:40 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 22 Mar 2009 19:56:40 +0000 (UTC)
Subject: [Python-Dev] an unimportant question, ...
References: <49C6698D.4060700@stackless.com> <49C68B10.4040107@v.loewis.de>
	<49C693B5.8070309@stackless.com>
Message-ID: <loom.20090322T195414-22@post.gmane.org>

Christian Tismer <tismer <at> stackless.com> writes:
> 
> Or was it maybe to just keep the string layout on many
> common platforms compatible, in order to save rebuilding
> so many windows extension modules?
> 
> If the latter is true and the only reason, I vote for reclaiming
> the three bytes. Maybe it saves a tree or two. Maybe it hurts
> very little if done for Python 3000.
> 
> In any case, use the version that saves the most energy. 

Well, if you want to make the str type in py3k smaller, there is a more massive
saving to be done by making it a PyVarObject, rather than allocating the storage
separately.

A patch has existed for that for a long time now, it probably needs updating if
anyone is interested:
http://bugs.python.org/issue1943




From ajaksu at gmail.com  Sun Mar 22 23:30:01 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Sun, 22 Mar 2009 19:30:01 -0300
Subject: [Python-Dev] GSoC: Core Python development tools
Message-ID: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>

Hi,
I'd like to bring up the general idea of using a PSF slot in GSoC2009
to improve the Python development infrastructure. I also happen to
have two concrete proposals for that (such a coincidence!). But I
assure you the general idea is more important than my proposals :)

General:
Solving issues that get in core devs' way when they work on Python
development could be a nice PSF GSoC project.

I believe there are enough code related tasks that would greatly
improve Python development workflow but lack manpower to complete.
E.g., ISTM that a student working on svnmerge in past SoC editions
would've been able to mitigate some painful shortcomings. If the core
developers could come up with a list of peeves that would be solvable
in code (in existing tools), that would allow for a very useful GSoC
proposal.

Proposals:
These might fit the description above, and they're both tracker
related (yep, one-trick-pony here). The upside is that at least one
potential mentor is available for each, and I'd be able to offer
support to the student.

First, the PSF could invest a slot on integrating different tools of
the development workflow. Variations of 'file issue at bug tracker,
submit code for review' or 'branch locally to virtualenv, upload
failing testcase to tracker, upload patch to tracker' command line
utils. If these could be developed as general tools (i.e., not deeply
tied to Python dev infrastructure), Ali Afshar from PIDA is willing to
mentor it. I'd be available to help with Roundup and Rietveld
(integration in progress), but would like to see it work with
Launchpad, Bugzilla, Google Code and Review Board :)

The other proposal is to use a slot in Roundup proper and the Python
tracker instance. This could have a core Roundup developer as mentor,
under the condition it's focused on Roundup itself. IMO, are some
missing/broken core features in Roundup that would have a positive
impact on our tracker, mostly those relating to searches, security and
UI. I'd have a lot to contribute to the Python tracker part, based on
ongoing work.

Even if neither is considered worthy, I'll keep them on my to-do list
and hope to slowly and hackishly work towards both proposals' goals.
Barring feedback saying that they're out of scope, stupid and
downright offensive, I'll post details on each to this thread very
soon.

So, if the PSF was to use a slot on improving the way you work on
Python development, what would you like to see fixed or implemented?

Best regards,
Daniel

From ctb at msu.edu  Mon Mar 23 00:13:00 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Sun, 22 Mar 2009 16:13:00 -0700
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
Message-ID: <20090322231259.GA29189@idyll.org>

On Sun, Mar 22, 2009 at 07:30:01PM -0300, Daniel (ajax) Diniz wrote:
-> Even if neither is considered worthy, I'll keep them on my to-do list
-> and hope to slowly and hackishly work towards both proposals' goals.
-> Barring feedback saying that they're out of scope, stupid and
-> downright offensive, I'll post details on each to this thread very
-> soon.

Given the relative paucity of core Python GSoC ideas, I think you should
go for both of these, *especially* if you have a mentor up front.  So,
write 'em up, add 'em to the GSoC page, and let's see who we get...
If we get good applications for both, then I think we can "fund" both of
them.

I do think you should be prepared for pushback from python-dev on any
such ideas ;).  Don't get too ambitious about writing up *your* way of
fixing things, but rather make sure you and the students are prepared to
adapt to what people on python-dev think.  Mind you, ultimately the
people doing the work should make the decisions, but input from
python-dev is usually pretty useful even when it's contradictory!

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From brett at python.org  Mon Mar 23 00:20:32 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 22 Mar 2009 16:20:32 -0700
Subject: [Python-Dev] Issue workflow doc is now live
Message-ID: <bbaeab100903221620q64dd24ccrfb171e9314309c8b@mail.gmail.com>

I sent this out as a Google Doc a while back, but I just did a
proof-reading, converted it, and pushed it live to the python.org:
http://www.python.org/dev/workflow/ . So now people who ever triage issues
have a guide to follow if they are not sure how to set things.

-Brett

P.S.: Doing this doc has made me realize that our issue workflow still feels
off and SF-influenced. If people want to talk at PyCon about it just let me
know. I might try to get a discussion going during the language summit or at
the sprints as I have some ideas on how to simplify thing but make it usable
for more people (preview: I like how
http://code.djangoproject.com/querydoes their tickets).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090322/a43697c6/attachment.htm>

From greg.ewing at canterbury.ac.nz  Mon Mar 23 01:34:26 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 23 Mar 2009 13:34:26 +1300
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <loom.20090322T150537-254@post.gmane.org>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<loom.20090322T150537-254@post.gmane.org>
Message-ID: <49C6D912.9020008@canterbury.ac.nz>

Antoine Pitrou wrote:

> If it's really enough to understand and debug all corner cases of using "yield
> from", then fair enough.

In the case where the subiterator is another generator and
isn't shared, it's intended to be a precise and complete
specification. That covers the vast majority of the use
cases I have in mind.

Most of the complexities arise from trying to pin down
what happens when the subiterator isn't a generator, or
is being shared by other code. I don't know how the
specification could be made any simpler for those cases
while still being complete.

Even so, the intention is that if you understand the
semantics in the generator case, the behaviour in the
other cases should be something reasonable and
unsurprising. I certainly don't expect users to memorize
either the expansion or the full text of the English
explanation.

-- 
Greg

From rdmurray at bitdance.com  Mon Mar 23 02:49:01 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 22 Mar 2009 21:49:01 -0400 (EDT)
Subject: [Python-Dev] issue 2170 review
Message-ID: <Pine.LNX.4.64.0903222134310.31981@kimball.webabinitio.net>

Instead of looking at a bunch of issues last week the way I'd originally
intended, I wound up doing a review of a particular patch submission,
issue 2170.  This is a refactoring of the 'normalize' method of
xml.dom.minidom.  I wound up redoing the patch with a different
refactoring after finding out that the submitted patch was slower than
the original code.  The revised patch is faster than the original code
(completes in 1/2 the time of the original code when run on a decent
sized document that needed no normalization done).

I think the issue stage should be set to 'patch review', as in, it is ready
for developer review.  The author of the original patch has reviewed my
revised patch and concurs.  The patch includes additional tests.

Now that I've learned more about how the tracker is organized, and read
the most-helpful http://www.python.org/dev/workflow/ document, I would
be happy to go through and triage issues for which that hasn't yet been
done, if I can be given permission to do so.

--
R. David Murray           http://www.bitdance.com

From ctb at msu.edu  Mon Mar 23 03:01:10 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Sun, 22 Mar 2009 19:01:10 -0700
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <913f9f570903201518l208576e2ndc4ffd75ce2eb16c@mail.gmail.com>
References: <913f9f570903201518l208576e2ndc4ffd75ce2eb16c@mail.gmail.com>
Message-ID: <20090323020110.GJ2444@idyll.org>

On Fri, Mar 20, 2009 at 03:18:00PM -0700, average wrote:
-> > Summer of Code is ramping up. ?Every year the common complaint is that not
-> > enough Python core projects get proposed by students, and of course a big
-> > reason for that is often the only encouragement we offer prospective
-> > students is a link to the PEP index.
-> >
-> > The challenge is finding project ideas for them that could reasonably occupy
-> > them for the entire Summer and which the results of their work can be
-> > demonstrated. ?They're being paid for specific projects so "Spend the Summer
-> > fixing bugs on the tracker" is a no-go, and Google has outlined that Summer
-> > of Code is about code, not documentation.
-> 
-> Improve doctest by allowing it to be aware of nested test scopes such
-> that a variable defined at "class-level scope" (i.e. the variable b
-> defined at the class-level doctest """>>> b=Bag("abacab")""") can be
-> used in "method-level scopes" without re-defining it every time for
-> each method's doctest (each method would reset the given variable (if
-> used) to its original state rather than live mutated between
-> equal-level scopes).
-> 
-> Would be a great improvement for doctest in my opinion--both in
-> ease-of-use, and reduction of redundant, error-prone ("did you define
-> your test variable the same in each method?") code)--as well as other
-> benefits.
-> 
-> Appreciate any consideration...

Hi Marcos,

my primary concern here would be that the student would do all this work
and then python-dev would reject it for incorporation into core!  Plus
it's probably not a summer-long project.

If, however, you wanted to suggest a general "gather disparate doctest
features and integrate them, for consideration for the core" project, I
would definitely recommend posting that as a possible project on the
Python GSoC site.  I know that zope has done some good doctest stuff,
for example; the 'testing-in-python' list would be a good place to go
for finding out more.

Note, you don't have to offer to be the mentor to post it, but it would
help ;)

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From tjreedy at udel.edu  Mon Mar 23 03:01:43 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 22 Mar 2009 22:01:43 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49C5F2B1.4020503@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>	<20090321162244.ED2423A4062@sparrow.telecommunity.com>	<49C56872.9020609@canterbury.ac.nz>	<20090322004950.A56AC3A4062@sparrow.telecommunity.com>
	<49C5F2B1.4020503@canterbury.ac.nz>
Message-ID: <gq6qi3$6b7$1@ger.gmane.org>

Greg Ewing wrote:

> As for confusion, we ignore the return values of function
> calls all the time, without worrying that someone might be
> confused by the fact that their return value doesn't go
> anywhere. And that's the right way to think of a yield-from
> expression -- as a kind of function call, not a kind of yield.
> 
> If there's anything confusing, it's the presence of the
> word 'yield'. Its only virtue is that it gives a clue that
> the construct has something to do with generators, but
> you'll have to RTM to find out exactly what. Nobody has
> thus far suggested any better name, however.

If the yield in 'yield from' does not make the function a generator, 
then perhaps 'return from' would be clearer.



From brett at python.org  Mon Mar 23 03:21:26 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 22 Mar 2009 19:21:26 -0700
Subject: [Python-Dev] issue 2170 review
In-Reply-To: <Pine.LNX.4.64.0903222134310.31981@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0903222134310.31981@kimball.webabinitio.net>
Message-ID: <bbaeab100903221921r45145989saf9aa03db69ff0da@mail.gmail.com>

On Sun, Mar 22, 2009 at 18:49, R. David Murray <rdmurray at bitdance.com>wrote:

> Instead of looking at a bunch of issues last week the way I'd originally
> intended, I wound up doing a review of a particular patch submission,
> issue 2170.  This is a refactoring of the 'normalize' method of
> xml.dom.minidom.  I wound up redoing the patch with a different
> refactoring after finding out that the submitted patch was slower than
> the original code.  The revised patch is faster than the original code
> (completes in 1/2 the time of the original code when run on a decent
> sized document that needed no normalization done).
>
> I think the issue stage should be set to 'patch review', as in, it is ready
> for developer review.  The author of the original patch has reviewed my
> revised patch and concurs.  The patch includes additional tests.
>

I have set the stage.


>
> Now that I've learned more about how the tracker is organized, and read
> the most-helpful http://www.python.org/dev/workflow/ document, I would
> be happy to go through and triage issues for which that hasn't yet been
> done, if I can be given permission to do so.


Done!

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090322/3801593e/attachment.htm>

From scott+python-dev at scottdial.com  Mon Mar 23 04:31:00 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Sun, 22 Mar 2009 23:31:00 -0400
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <49C69333.9000806@cheimes.de>
References: <gq1bo2$uov$1@ger.gmane.org>
	<49C43864.6050808@v.loewis.de>	<49C439E1.4050403@cheimes.de>
	<49C5E21C.5040607@v.loewis.de> <49C69333.9000806@cheimes.de>
Message-ID: <49C70274.5010902@scottdial.com>

Christian Heimes wrote:
> Martin v. L?wis schrieb:
>>> According to the user's experience multiprocessing should not compile
>>> and run correctly unless this patch is applied.
>> Can this please be more qualified? I can confirm Scott's observation:
>> for the trunk, it compiles just fine, using SunPro CC on Solaris 10,
>> on SPARC. Also, test_multiprocessing passes.

Sorry, I mistakenly said "without issue" and then copied the issues
below. I meant to say "not without issues." _multiprocessing does *not*
build on Solaris 8.

>> IIUC, incorrect setting of HAVE_SEM_TIMEDWAIT to 1 should cause a
>> compile failure, as sem_timedwait would be called but not defined.
>> However, sem_timedwait *is* defined on Solaris.
>>
>> Likewise, for HAVE_FD_TRANSFER=1 to work, SCM_RIGHTS must be defined
>> (and implemented); it is defined in sys/socket.h, and documented in
>> socket.h(3HEAD).
>>
>> So there must be going on something else at the user's machine.
> 
> The user doesn't respond to my inquiries anymore. According to his
> initial message he is using Solaris 5.8 with GCC 3.4.6 on a Sun Fire
> machine. Here is a link to his mesage:
> http://permalink.gmane.org/gmane.comp.python.general/615802

I can confirm his build issues on:

$ uname -srvmpi
SunOS 5.8 Generic_117350-51 sun4u sparc SUNW,Sun-Fire-280R
$ gcc -v
Reading specs from
/usr/local/gnu/lib/gcc-lib/sparc-sun-solaris2.8/2.95.2/specs
gcc version 2.95.2 19991024 (release)

My build output from within the trunk is the same as his modulo the
details of being part of a trunk build instead.

Is Solaris 8 really a supported platform? If so, I can investigate the
changes he suggested.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From tjreedy at udel.edu  Mon Mar 23 04:48:22 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 22 Mar 2009 23:48:22 -0400
Subject: [Python-Dev] GSoC: Replace MS Windows Console with Unicode UI
In-Reply-To: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
Message-ID: <gq70q2$hul$1@ger.gmane.org>

One of the disappointments of CPython 3.0 on Windows is that the switch 
to unicode for text (str), coupled with the continued use of a 
unicode-oblivious (obtuse) user interface (MS 'Command Prompt'), means 
that print can no longer print all str strings, or all legal Python code 
(as in a traceback).

Different people have tried and failed fix this bug by fiddling with 
'Command Prompt' configeration.  This would make a useful summer 
project, though I don't know if it would involve enough coding.  Call 
the project something like 3.0 Unicode UI Improvement.

I see two possibilities.

1) Find an C-coded open-source C-P replacement whose author will license 
to PSF and, as needed, modify or integrate it with CPython.

2) IDLE does much better but its support seems to still be imcomplete. 
Upgrade tk/tkinter/IDLE (wherever the problems lie) and make IDLE's 
shell an alternate UI.

If Windows (or other OSes) (to be investigated) does not reliably come 
with a full unicode font (at least current BMP), is there a public 
domain or open license font that we can include?

Terry Jan Reedy


From v+python at g.nevcal.com  Mon Mar 23 05:13:30 2009
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sun, 22 Mar 2009 21:13:30 -0700
Subject: [Python-Dev] GSoC: Replace MS Windows Console with Unicode UI
In-Reply-To: <gq70q2$hul$1@ger.gmane.org>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
	<gq70q2$hul$1@ger.gmane.org>
Message-ID: <49C70C6A.7070106@g.nevcal.com>

On approximately 3/22/2009 8:48 PM, came the following characters from 
the keyboard of Terry Reedy:
> One of the disappointments of CPython 3.0 on Windows is that the switch 
> to unicode for text (str), coupled with the continued use of a 
> unicode-oblivious (obtuse) user interface (MS 'Command Prompt'), means 
> that print can no longer print all str strings, or all legal Python code 
> (as in a traceback).
> 
> Different people have tried and failed fix this bug by fiddling with 
> 'Command Prompt' configeration.  This would make a useful summer 
> project, though I don't know if it would involve enough coding.  Call 
> the project something like 3.0 Unicode UI Improvement.
> 
> I see two possibilities.
> 
> 1) Find an C-coded open-source C-P replacement whose author will license 
> to PSF and, as needed, modify or integrate it with CPython.
> 
> 2) IDLE does much better but its support seems to still be imcomplete. 
> Upgrade tk/tkinter/IDLE (wherever the problems lie) and make IDLE's 
> shell an alternate UI.
> 
> If Windows (or other OSes) (to be investigated) does not reliably come 
> with a full unicode font (at least current BMP), is there a public 
> domain or open license font that we can include?


One can, of course, set CMD into Latin-1 mode (chcp 1252)... not sure 
how Python reacts to that, as I've only used it with Perl, until I gave 
up on Perl's Unicode support (which someone finally seems to be fixing, 
but then there is CPAN to improve).  But that doesn't solve the font 
problem, nor characters above 255.

One can set CMD into Unicode mode (chcp 65001)... not sure how Python 
reacts to that either.  But even then...

CMD will only use fixed-width fonts, and none of the standard XP ones 
seem to contain all of Unicode.  Not sure if that has improved on Vista 
or 7, as they don't run here.

It _would_ be nice to get this resolved, somehow.


-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

From brett at python.org  Mon Mar 23 05:20:41 2009
From: brett at python.org (Brett Cannon)
Date: Sun, 22 Mar 2009 21:20:41 -0700
Subject: [Python-Dev] "How to Contribute to Python" doc now online
Message-ID: <bbaeab100903222120l6399cc8csb688a0b2a790f17a@mail.gmail.com>

In preparation for Pycon and the sprints I quickly pulled together a doc
explaining how people can help out with Python's development:
http://www.python.org/dev/contributing/ .

-Brett

P.S.: Just so people know, I will be taking a month or two off from Python
development (i.e. heavy coding) after PyCon to redo my father's web site and
simply to take a slight breather after all the importlib stuff. Plan to keep
plugging away at the DVCS PEP, though.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090322/154dceb5/attachment.htm>

From tleeuwenburg at gmail.com  Mon Mar 23 06:20:07 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Mon, 23 Mar 2009 16:20:07 +1100
Subject: [Python-Dev] tracker status options
In-Reply-To: <2d75d7660903191653v65640c90pc4df8ddcf60343f1@mail.gmail.com>
References: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>
	<49C1EA9E.3040603@v.loewis.de>
	<bbaeab100903191414o1710bd87x3d4c9a713c63c832@mail.gmail.com>
	<49C2D040.4080609@v.loewis.de>
	<2d75d7660903191653v65640c90pc4df8ddcf60343f1@mail.gmail.com>
Message-ID: <43c8685c0903222220h6ecc841ar27ce66adf3c3d064@mail.gmail.com>

Hi Daniel,

That would be great. It occurs to me that if we wanted to use "Stage"
settings, it would be easy to search for issues which are not closed by
literally searching for 'not closed' rather than 'open'. I think it's also
unclear whether the 'pending' stage means 'suspended pending additional user
feedback' or 'resolution of this issue is impending'. My understanding was
that it meant 'pending additional feedback' in the sense of awaiting, rather
than imminent.

In any case, let me know when the tracker is online and I will suggest the
tag alterations that I think would be appropriate.

Regards,
-Tennessee

On Fri, Mar 20, 2009 at 10:53 AM, Daniel (ajax) Diniz <ajaksu at gmail.com>wrote:

> Martin v. L?wis wrote:
> > In addition, I would like to see a specification of the exact labels to
> > be used, plus a one-line description that should be attached to the
> > labels.
>
> Tennessee,
> If you'd like to test those additional status options, I'm setting a
> test instance of the Python tracker up at
> http://bot.bio.br/python-dev/ . It might be frequently offline for a
> while, but once things are stable it'll be reliable enough to perform
> such experiments.
>
> If it's easy on resource usage, I might have one instance following
> the Python tracker code closely and a more bleeding-edge one :)
>
> Regards,
> Daniel
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/tleeuwenburg%40gmail.com
>



-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090323/b400b947/attachment.htm>

From bignose+hates-spam at benfinney.id.au  Mon Mar 23 06:25:39 2009
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Mon, 23 Mar 2009 16:25:39 +1100
Subject: [Python-Dev] GSoC: Replace MS Windows Console with Unicode UI
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
	<gq70q2$hul$1@ger.gmane.org>
Message-ID: <87r60obyjw.fsf@benfinney.id.au>

Terry Reedy <tjreedy at udel.edu> writes:

> If Windows (or other OSes) (to be investigated) does not reliably
> come with a full unicode font (at least current BMP), is there a
> public domain or open license font that we can include?

The GNU Unifont at Unifoundry <URL:http://unifoundry.com/> is designed
for this purpose.

-- 
 \      ?For my birthday I got a humidifier and a de-humidifier. I put |
  `\  them in the same room and let them fight it out.? ?Steven Wright |
_o__)                                                                  |
Ben Finney


From theller at ctypes.org  Mon Mar 23 07:58:29 2009
From: theller at ctypes.org (Thomas Heller)
Date: Mon, 23 Mar 2009 07:58:29 +0100
Subject: [Python-Dev] Py_ssize_t support for ctypes arrays and pointers
In-Reply-To: <20090322053650.GA30571@wind.teleri.net>
References: <gq0p4u$8rf$1@ger.gmane.org>
	<20090322053650.GA30571@wind.teleri.net>
Message-ID: <49C73315.3070706@ctypes.org>

Trent Nelson schrieb:
> On Fri, Mar 20, 2009 at 08:00:46PM +0100, Thomas Heller wrote:
>> Since I do not have a machine with so much memory: Does one
>> of the buildbots allow to run tests for this feature, or
>> do I have to wait for the snakebite farm?
> 
>     Will you be at PyCon?  The wait might not be as bad as you think ;-)

No, I will not.  But I've heard some rumors...

-- 
Thanks,
Thomas

From martin at v.loewis.de  Mon Mar 23 08:03:26 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 23 Mar 2009 08:03:26 +0100
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <49C70274.5010902@scottdial.com>
References: <gq1bo2$uov$1@ger.gmane.org>
	<49C43864.6050808@v.loewis.de>	<49C439E1.4050403@cheimes.de>
	<49C5E21C.5040607@v.loewis.de> <49C69333.9000806@cheimes.de>
	<49C70274.5010902@scottdial.com>
Message-ID: <49C7343E.3040500@v.loewis.de>

> Sorry, I mistakenly said "without issue" and then copied the issues
> below. I meant to say "not without issues." _multiprocessing does *not*
> build on Solaris 8.

Hmm. They are all warnings - did you omit the actual error message?

The lack of CMSG_LEN seems to suggest that control messages are not
supported on Solaris 8, or that you need to include an additional
header file to get them.

> Is Solaris 8 really a supported platform? If so, I can investigate the
> changes he suggested.

The concept of "supported platform" doesn't really exist - there is no
way to obtain "support". If it works, it works, if it doesn't, it
doesn't. So if you want it fixed, provide a patch - else we can drop
the issue.

I think multiprocessing is misguided in hard-coding these settings
into setup.py. As we can see, the necessary features are available
on some versions of Solaris, but not on others. It would be better
if autoconf tests were written, and the entire configuration removed
from setup.py.

Regards,
Martin



From martin at v.loewis.de  Mon Mar 23 08:58:49 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 23 Mar 2009 08:58:49 +0100
Subject: [Python-Dev] GSoC: Replace MS Windows Console with Unicode UI
In-Reply-To: <gq70q2$hul$1@ger.gmane.org>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
	<gq70q2$hul$1@ger.gmane.org>
Message-ID: <49C74139.7030004@v.loewis.de>

> 2) IDLE does much better but its support seems to still be imcomplete.
> Upgrade tk/tkinter/IDLE (wherever the problems lie) and make IDLE's
> shell an alternate UI.

That is certainly *no* good SoC project. Instead, just report it as
a *specific* bug report (rather than saying "it seems incomplete").

Regards,
Martin

From scott+python-dev at scottdial.com  Mon Mar 23 09:02:38 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Mon, 23 Mar 2009 04:02:38 -0400
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <49C7343E.3040500@v.loewis.de>
References: <gq1bo2$uov$1@ger.gmane.org>	<49C43864.6050808@v.loewis.de>	<49C439E1.4050403@cheimes.de>	<49C5E21C.5040607@v.loewis.de>
	<49C69333.9000806@cheimes.de>	<49C70274.5010902@scottdial.com>
	<49C7343E.3040500@v.loewis.de>
Message-ID: <49C7421E.3080605@scottdial.com>

Martin v. L?wis wrote:
>> Sorry, I mistakenly said "without issue" and then copied the issues
>> below. I meant to say "not without issues." _multiprocessing does *not*
>> build on Solaris 8.
> 
> Hmm. They are all warnings - did you omit the actual error message?
> 
> The lack of CMSG_LEN seems to suggest that control messages are not
> supported on Solaris 8, or that you need to include an additional
> header file to get them.

Sorry, I was trying to keep the amount of noise to a minimum, but those
particular implicit function declarations ultimately lead to linking
errors for lack of those same symbols:

*** WARNING: renaming "_multiprocessing" since importing it failed:
ld.so.1: python: fatal: relocation error: file
build/lib.solaris-2.8-sun4u-2.7/_multiprocessing.so: symbol CMSG_SPACE:
referenced symbol not found

>> Is Solaris 8 really a supported platform? If so, I can investigate the
>> changes he suggested.
> 
> The concept of "supported platform" doesn't really exist - there is no
> way to obtain "support". If it works, it works, if it doesn't, it
> doesn't. So if you want it fixed, provide a patch - else we can drop
> the issue.

I have no personal interest in the matter. I just happened to have
access to an older Solaris just for this sort of testing. If someone has
a patch, then I would be glad to test it, but otherwise, I am not going
to invest any time on this. The patch given by the OP is clearly
undesirable since status quo works just fine for a modern Solaris install.

> I think multiprocessing is misguided in hard-coding these settings
> into setup.py. As we can see, the necessary features are available
> on some versions of Solaris, but not on others. It would be better
> if autoconf tests were written, and the entire configuration removed
> from setup.py.

I agree, but I am not familiar enough with the set of features that
multiprocessing is implicitly depending on and how to test for them on
all of the platforms.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From ncoghlan at gmail.com  Mon Mar 23 13:26:54 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 23 Mar 2009 22:26:54 +1000
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <20090322231259.GA29189@idyll.org>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
	<20090322231259.GA29189@idyll.org>
Message-ID: <49C7800E.5020502@gmail.com>

C. Titus Brown wrote:
> On Sun, Mar 22, 2009 at 07:30:01PM -0300, Daniel (ajax) Diniz wrote:
> I do think you should be prepared for pushback from python-dev on any
> such ideas ;).  Don't get too ambitious about writing up *your* way of
> fixing things, but rather make sure you and the students are prepared to
> adapt to what people on python-dev think.  Mind you, ultimately the
> people doing the work should make the decisions, but input from
> python-dev is usually pretty useful even when it's contradictory!

Everything I've seen from Daniel so far seems to be about either making
things we already do more efficient, or else providing additional
features in ways that don't make the tools any more confusing for others
already used to a particular way of doing things. So he seems to be
navigating this particular minefield pretty well so far.

Then again, I may be a little biased - some of the recent bug tracker
changes are things I had wished for in the past, but never chipped in
any code to help make them happen :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Mon Mar 23 13:31:06 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 23 Mar 2009 22:31:06 +1000
Subject: [Python-Dev] Issue workflow doc is now live
In-Reply-To: <bbaeab100903221620q64dd24ccrfb171e9314309c8b@mail.gmail.com>
References: <bbaeab100903221620q64dd24ccrfb171e9314309c8b@mail.gmail.com>
Message-ID: <49C7810A.8060908@gmail.com>

Brett Cannon wrote:
> I sent this out as a Google Doc a while back, but I just did a
> proof-reading, converted it, and pushed it live to the python.org
> <http://python.org>: http://www.python.org/dev/workflow/ . So now people
> who ever triage issues have a guide to follow if they are not sure how
> to set things.

Does that mean PEP 3 should be Withdrawn or does it mean it should be
replaced?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From barry at python.org  Mon Mar 23 13:36:53 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 23 Mar 2009 08:36:53 -0400
Subject: [Python-Dev] Issue workflow doc is now live
In-Reply-To: <49C7810A.8060908@gmail.com>
References: <bbaeab100903221620q64dd24ccrfb171e9314309c8b@mail.gmail.com>
	<49C7810A.8060908@gmail.com>
Message-ID: <9143467B-3151-4758-8DD9-EB3A1E5D89A5@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 23, 2009, at 8:31 AM, Nick Coghlan wrote:

> Brett Cannon wrote:
>> I sent this out as a Google Doc a while back, but I just did a
>> proof-reading, converted it, and pushed it live to the python.org
>> <http://python.org>: http://www.python.org/dev/workflow/ . So now  
>> people
>> who ever triage issues have a guide to follow if they are not sure  
>> how
>> to set things.
>
> Does that mean PEP 3 should be Withdrawn or does it mean it should be
> replaced?

It should probably be replaced with Brett's document.
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSceCZXEjvBPtnXfVAQKMoQQApOowWiXtfS9p1+barzrEwbFJ89tE3Tmy
LW14lbgDxMXc+nx/Y1z+0L8r8eRThrbuiLr8qtM7VWiEmlUz8msHovNL6AR+oppy
lyAk8ex9sc/GAnZXQ6i3b2FBo1evHr3zbQzUgcbNlVW1iyLsjGPUiG0Ay2A+AmyM
ZWJXGCbJ89g=
=PuKR
-----END PGP SIGNATURE-----

From chris at simplistix.co.uk  Mon Mar 23 14:55:26 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 23 Mar 2009 13:55:26 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BA7531.8000806@palladion.com>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>	<49BA6009.5070605@palladion.com>	<gpdr4k$sbl$1@ger.gmane.org>
	<49BA7531.8000806@palladion.com>
Message-ID: <49C794CE.6050707@simplistix.co.uk>

Tres Seaver wrote:
> Given that the out-of-the-box Python install already has facilities for
> retrieving text over the net and executing that text, the notion of
> "locking down" a machine to include only the bits installed in the stock
> Python install is just "security theatre;"  such a machine shouldn't
> have Python installed at all (nor a C compiler, etc.)

Indeed, in the real world this locking down is done at the firewall level.

As for packaging in this scenario, that's what private package servers 
are for...

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Mon Mar 23 15:00:12 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 23 Mar 2009 14:00:12 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <gpdu3h$8ct$1@ger.gmane.org>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>	<49BA6009.5070605@palladion.com>	<gpdr4k$sbl$1@ger.gmane.org>	<49BA7531.8000806@palladion.com>
	<gpdu3h$8ct$1@ger.gmane.org>
Message-ID: <49C795EC.6020603@simplistix.co.uk>

Lie Ryan wrote:
> Some companies have /very/ strict policies on running anything on live 
> server, including scripts you write yourself. The problem is if the 
> script goes awry, it might disturb the stability or even security of the 
> server.

Yes, "we" as a profession right software and have responsibilities. Get 
over it. It's what dev servers, uat, backups and dr are for...
I see no relation between this and packaging other than that any 
packaging story needs to support privtae distribution servers.

> When the server administrator is already freaked out about adding an 
> script developed by in-house employee, what about adding an external 
> module?

Then he's a muppet, plain and simple. If he's not, then he will have 
tested the whole setup before hand and got signoff from the developers 
and users who are responsbile for doing so.

All of this has very little to do with packaging.

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Mon Mar 23 15:04:38 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 23 Mar 2009 14:04:38 +0000
Subject: [Python-Dev] packaging (was Integrate BeautifulSoup into
	stdlib?)
In-Reply-To: <49BB1EB6.90702@gmail.com>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>	<49BA6009.5070605@palladion.com>
	<49BB1EB6.90702@gmail.com>
Message-ID: <49C796F6.1040502@simplistix.co.uk>

Nick Coghlan wrote:
> Network connectivity isn't a given, even today. 

Indeed, now that is an important consideration.
Packaging systems need to support "offline" modes. Buildout already does...

> If someone else decides to create a MinimalPython which consists solely
> of something like easy_install and whatever is needed to run it (i.e.
> the opposite of the "fat" bundles from folks like ActiveState and
> Enthought), then more power to them. But I don't believe the official
> releases from python.org should go that way.

My frustration is that some of the big standard libraries are locked to 
python releases meaning they carry around bugs for longer and are harder 
to contribute to than necessary...

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Mon Mar 23 15:08:56 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 23 Mar 2009 14:08:56 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <79990c6b0903131542hdfad7fbi1964cba097d497b7@mail.gmail.com>
References: <49ABCFC9.1070508@gmail.com>	
	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	
	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	
	<gooa4o$rrh$1@ger.gmane.org>	
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	
	<49B3B2E8.4050605@simplistix.co.uk> <49B3CD53.5010906@v.loewis.de>	
	<49BA2E61.7070104@simplistix.co.uk>	
	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>	
	<49BA6009.5070605@palladion.com>
	<79990c6b0903131542hdfad7fbi1964cba097d497b7@mail.gmail.com>
Message-ID: <49C797F8.2030208@simplistix.co.uk>

Paul Moore wrote:
> I am not. What I *am* doing is saying (obliquely, I admit) is that for
> a package management system to be "decent enough" for stripping down
> the stdlib to not be an issue, it has to address these points (which
> clearly it can't). 

Sure it can, either by supporting "offline bundles" or by having sets of 
packages that are marked as "Python Approved!" or some such and so all 
have the same license.

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Mon Mar 23 15:24:04 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 23 Mar 2009 14:24:04 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <Pine.LNX.4.64.0903130739170.2596@kimball.webabinitio.net>
References: <49ABCFC9.1070508@gmail.com>
	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>
	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>
	<Pine.LNX.4.64.0903130739170.2596@kimball.webabinitio.net>
Message-ID: <49C79B84.9050703@simplistix.co.uk>

R. David Murray wrote:
> I disagree.  One of the jobs I've had is release management for
> internal software projects that depend on various external pieces.
> Release integration tested against specific versions of those external
> packages, and those were the packages that needed to wind up on the system
> when the release was installed.  I've done systems depending on both perl
> and python, and let me tell you, python is way, _way_ easier to manage.
> With python, I have a dependency on a particular python version, and then
> maybe one or two add on packages.  

Well, python already has tools available to do exactly this.:
buildout from a private egg repository will do exactly what you're after.

However, its built on top of setuptools, which is flawed, and it's not 
blessed as "official core python", so there's lots of room for improvement!

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From steve at holdenweb.com  Mon Mar 23 15:25:48 2009
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 23 Mar 2009 10:25:48 -0400
Subject: [Python-Dev] packaging (was Integrate BeautifulSoup into
	stdlib?)
In-Reply-To: <49C796F6.1040502@simplistix.co.uk>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>	<49BA6009.5070605@palladion.com>	<49BB1EB6.90702@gmail.com>
	<49C796F6.1040502@simplistix.co.uk>
Message-ID: <49C79BEC.2080601@holdenweb.com>

Chris Withers wrote:
> Nick Coghlan wrote:
>> Network connectivity isn't a given, even today. 
> 
> Indeed, now that is an important consideration.
> Packaging systems need to support "offline" modes. Buildout already does...
> 
>> If someone else decides to create a MinimalPython which consists solely
>> of something like easy_install and whatever is needed to run it (i.e.
>> the opposite of the "fat" bundles from folks like ActiveState and
>> Enthought), then more power to them. But I don't believe the official
>> releases from python.org should go that way.
> 
> My frustration is that some of the big standard libraries are locked to
> python releases meaning they carry around bugs for longer and are harder
> to contribute to than necessary...
> 
Possibly so, but there are conflicting requirements and Python can't
satisfy them all without getting more complex.

Some people want an "all batteries and kitchen sink included" distro
that they can treat as a single component for configuration control
purposes. Others, like you, want the libraries to be separated out to
allow separate fixes.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Mon Mar 23 15:25:48 2009
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 23 Mar 2009 10:25:48 -0400
Subject: [Python-Dev] packaging (was Integrate BeautifulSoup into
	stdlib?)
In-Reply-To: <49C796F6.1040502@simplistix.co.uk>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>	<49BA6009.5070605@palladion.com>	<49BB1EB6.90702@gmail.com>
	<49C796F6.1040502@simplistix.co.uk>
Message-ID: <49C79BEC.2080601@holdenweb.com>

Chris Withers wrote:
> Nick Coghlan wrote:
>> Network connectivity isn't a given, even today. 
> 
> Indeed, now that is an important consideration.
> Packaging systems need to support "offline" modes. Buildout already does...
> 
>> If someone else decides to create a MinimalPython which consists solely
>> of something like easy_install and whatever is needed to run it (i.e.
>> the opposite of the "fat" bundles from folks like ActiveState and
>> Enthought), then more power to them. But I don't believe the official
>> releases from python.org should go that way.
> 
> My frustration is that some of the big standard libraries are locked to
> python releases meaning they carry around bugs for longer and are harder
> to contribute to than necessary...
> 
Possibly so, but there are conflicting requirements and Python can't
satisfy them all without getting more complex.

Some people want an "all batteries and kitchen sink included" distro
that they can treat as a single component for configuration control
purposes. Others, like you, want the libraries to be separated out to
allow separate fixes.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From chris at simplistix.co.uk  Mon Mar 23 15:26:34 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 23 Mar 2009 14:26:34 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BAA596.5020106@v.loewis.de>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BA3154.8080408@simplistix.co.uk> <49BAA596.5020106@v.loewis.de>
Message-ID: <49C79C1A.8040301@simplistix.co.uk>

Martin v. L?wis wrote:
>>> In some environments, each new component must be approved.  Once
>>> python is approved, the standard library is OK, but adding 7 packages
>>> from pypi requires 7 more sets of approvals.
>> True, but as I mentioend elsewhere, I myself haven't done a python
>> project where I only needed python and the standard lib for many years...
> 
> I was always able to get what I need through aptitude.

Yes, well known for its ability to run on Windows and Mac OS...

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From chris at simplistix.co.uk  Mon Mar 23 15:27:44 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 23 Mar 2009 14:27:44 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49BAA693.3080709@activestate.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BA3154.8080408@simplistix.co.uk>
	<49BAA693.3080709@activestate.com>
Message-ID: <49C79C60.9000905@simplistix.co.uk>

Trent Mick wrote:
> Apologies for jumping in mid-thread here. FYI: We're (where "we" == 
> ActiveState here) looking at spending more effort on Python of late. 
> Some of our thoughts are on add modules: whether added to the 
> ActivePython core or easily addable via an equivalent to ActivePerl's 
> ppm (package manager) is still be batted around.

Buildout is the closest thing python has to sanity at the moment.

> I'm curious as to people's thoughts. I'll also be at PyCon in Chicago 
> getting thoughts.

I'll be there, look forward to it.

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From lists at cheimes.de  Mon Mar 23 15:31:47 2009
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 23 Mar 2009 15:31:47 +0100
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <49C7343E.3040500@v.loewis.de>
References: <gq1bo2$uov$1@ger.gmane.org>
	<49C43864.6050808@v.loewis.de>	<49C439E1.4050403@cheimes.de>
	<49C5E21C.5040607@v.loewis.de> <49C69333.9000806@cheimes.de>
	<49C70274.5010902@scottdial.com> <49C7343E.3040500@v.loewis.de>
Message-ID: <49C79D53.8040902@cheimes.de>

Martin v. L?wis wrote:
> I think multiprocessing is misguided in hard-coding these settings
> into setup.py. As we can see, the necessary features are available
> on some versions of Solaris, but not on others. It would be better
> if autoconf tests were written, and the entire configuration removed
> from setup.py.

I guess multiprocessing doesn't use autoconf tests for historical
reasons. It's ancestor -- the pyprocessing package -- was using hard
coded values, too.

I've started with a autoconf file for the multiprocessing backport
package. The tests should produce the same set of definitions but I'm
not able to test it on different platforms expect Linux.
http://python-multiprocessing.googlecode.com/svn/trunk/configure.ac

Christian

From chris at simplistix.co.uk  Mon Mar 23 15:32:40 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 23 Mar 2009 14:32:40 +0000
Subject: [Python-Dev] packaging (was Integrate BeautifulSoup
	into	stdlib?)
In-Reply-To: <49C79BEC.2080601@holdenweb.com>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>	<49BA6009.5070605@palladion.com>	<49BB1EB6.90702@gmail.com>	<49C796F6.1040502@simplistix.co.uk>
	<49C79BEC.2080601@holdenweb.com>
Message-ID: <49C79D88.9000404@simplistix.co.uk>

Steve Holden wrote:
> Some people want an "all batteries and kitchen sink included" distro
> that they can treat as a single component for configuration control
> purposes. Others, like you, want the libraries to be separated out to
> allow separate fixes.

Yes, but while the "batteries included" option can be rolled from the 
"no batteries" version, the reverse is not true.

The current package management systems can't even figure out that a 
version of a standard lib library is "what came with Python 2.x.y" 
rather than a bugfixed version that's been later installed (pyunit 
springs to mind as a potential candidate here...)

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From jnoller at gmail.com  Mon Mar 23 15:39:46 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 23 Mar 2009 10:39:46 -0400
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <49C79D53.8040902@cheimes.de>
References: <gq1bo2$uov$1@ger.gmane.org> <49C43864.6050808@v.loewis.de>
	<49C439E1.4050403@cheimes.de> <49C5E21C.5040607@v.loewis.de>
	<49C69333.9000806@cheimes.de> <49C70274.5010902@scottdial.com>
	<49C7343E.3040500@v.loewis.de> <49C79D53.8040902@cheimes.de>
Message-ID: <4222a8490903230739v32b8a066sde586c7cc4841855@mail.gmail.com>

On Mon, Mar 23, 2009 at 10:31 AM, Christian Heimes <lists at cheimes.de> wrote:
> Martin v. L?wis wrote:
>> I think multiprocessing is misguided in hard-coding these settings
>> into setup.py. As we can see, the necessary features are available
>> on some versions of Solaris, but not on others. It would be better
>> if autoconf tests were written, and the entire configuration removed
>> from setup.py.
>
> I guess multiprocessing doesn't use autoconf tests for historical
> reasons. It's ancestor -- the pyprocessing package -- was using hard
> coded values, too.
>
> I've started with a autoconf file for the multiprocessing backport
> package. The tests should produce the same set of definitions but I'm
> not able to test it on different platforms expect Linux.
> http://python-multiprocessing.googlecode.com/svn/trunk/configure.ac
>
> Christian
>

Christian is correct, it's purely for historical reasons. I have no
particular stake in keeping it as is other than the fact that there
are higher priorities in the bug queue (and improving the tests) than
the build functionality.

Christian - I would appreciate it if we could coordinate/track this on
the tracker too - I had no idea you were doing this in the back port,
and I don't think we want the two code bases to diverge that much.

This is definitely something which could be hacked on during the
sprints though. There's been a lot of movement in the *BSD space
around defining these values and providing the faculties needed for mp
to work.

-jesse

From ctb at msu.edu  Mon Mar 23 15:40:39 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Mon, 23 Mar 2009 07:40:39 -0700
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <49C7800E.5020502@gmail.com>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
	<20090322231259.GA29189@idyll.org> <49C7800E.5020502@gmail.com>
Message-ID: <20090323144039.GE11027@idyll.org>

On Mon, Mar 23, 2009 at 10:26:54PM +1000, Nick Coghlan wrote:
-> C. Titus Brown wrote:
-> > On Sun, Mar 22, 2009 at 07:30:01PM -0300, Daniel (ajax) Diniz wrote:
-> > I do think you should be prepared for pushback from python-dev on any
-> > such ideas ;).  Don't get too ambitious about writing up *your* way of
-> > fixing things, but rather make sure you and the students are prepared to
-> > adapt to what people on python-dev think.  Mind you, ultimately the
-> > people doing the work should make the decisions, but input from
-> > python-dev is usually pretty useful even when it's contradictory!
-> 
-> Everything I've seen from Daniel so far seems to be about either making
-> things we already do more efficient, or else providing additional
-> features in ways that don't make the tools any more confusing for others
-> already used to a particular way of doing things. So he seems to be
-> navigating this particular minefield pretty well so far.
-> 
-> Then again, I may be a little biased - some of the recent bug tracker
-> changes are things I had wished for in the past, but never chipped in
-> any code to help make them happen :)

Oh, I heartily endorse his suggestions! I just want to make sure that we
make maximum use of students (and their code doesn't get tossed at the
end of the summer, which has serious morale consequences ;)

--t
-- 
C. Titus Brown, ctb at msu.edu

From arcriley at gmail.com  Mon Mar 23 15:59:51 2009
From: arcriley at gmail.com (Arc Riley)
Date: Mon, 23 Mar 2009 10:59:51 -0400
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <20090323144039.GE11027@idyll.org>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
	<20090322231259.GA29189@idyll.org> <49C7800E.5020502@gmail.com>
	<20090323144039.GE11027@idyll.org>
Message-ID: <bad82a80903230759l532fb30cmbb0dc3ef6d8416ce@mail.gmail.com>

> Oh, I heartily endorse his suggestions! I just want to make sure that we
> make maximum use of students (and their code doesn't get tossed at the
> end of the summer, which has serious morale consequences ;)


This is my concern as well.

One of my past students pitched a core project and ended up spending most of
his SoC time in the PEP process (http://www.python.org/dev/peps/pep-0368/).
Given how frustrating the experience was for him, I'd rather have future SoC
students be able to focus on code.

Let's at least have rough consensus on brainstormed ideas before they hit
the ideas page.  Existing PEPs that have had a time to air and evolve are
much better for this reason.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090323/2ac94ddb/attachment.htm>

From lists at cheimes.de  Mon Mar 23 16:03:13 2009
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 23 Mar 2009 16:03:13 +0100
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <4222a8490903230739v32b8a066sde586c7cc4841855@mail.gmail.com>
References: <gq1bo2$uov$1@ger.gmane.org> <49C43864.6050808@v.loewis.de>	
	<49C439E1.4050403@cheimes.de> <49C5E21C.5040607@v.loewis.de>	
	<49C69333.9000806@cheimes.de> <49C70274.5010902@scottdial.com>	
	<49C7343E.3040500@v.loewis.de> <49C79D53.8040902@cheimes.de>
	<4222a8490903230739v32b8a066sde586c7cc4841855@mail.gmail.com>
Message-ID: <49C7A4B1.3080509@cheimes.de>

Jesse Noller wrote:
> Christian - I would appreciate it if we could coordinate/track this on
> the tracker too - I had no idea you were doing this in the back port,
> and I don't think we want the two code bases to diverge that much.

You could not have known about the idea because I started the experiment
about an hour ago. ;) I needed a playground to test and save the files
of the experiment. The multiprocessing backport sounded like a good place.

Christian

From p.f.moore at gmail.com  Mon Mar 23 16:04:39 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 23 Mar 2009 15:04:39 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C797F8.2030208@simplistix.co.uk>
References: <49ABCFC9.1070508@gmail.com> <gooa4o$rrh$1@ger.gmane.org>
	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>
	<49B3B2E8.4050605@simplistix.co.uk> <49B3CD53.5010906@v.loewis.de>
	<49BA2E61.7070104@simplistix.co.uk>
	<79990c6b0903130347h6fbbaba5sd07976b2af1eabbf@mail.gmail.com>
	<49BA6009.5070605@palladion.com>
	<79990c6b0903131542hdfad7fbi1964cba097d497b7@mail.gmail.com>
	<49C797F8.2030208@simplistix.co.uk>
Message-ID: <79990c6b0903230804s481158fw2466c35029348a1f@mail.gmail.com>

2009/3/23 Chris Withers <chris at simplistix.co.uk>:
> Paul Moore wrote:
>>
>> I am not. What I *am* doing is saying (obliquely, I admit) is that for
>> a package management system to be "decent enough" for stripping down
>> the stdlib to not be an issue, it has to address these points (which
>> clearly it can't).
>
> Sure it can, either by supporting "offline bundles" or by having sets of
> packages that are marked as "Python Approved!" or some such and so all have
> the same license.

OK, I'll drop out of the discussion at this point. We clearly have
such different experience that we aren't understanding each others'
points - and the misunderstandings aren't moving the discussion
forwards.

Paul

From daniel at stutzbachenterprises.com  Mon Mar 23 17:23:43 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 23 Mar 2009 11:23:43 -0500
Subject: [Python-Dev] speeding up PyObject_GetItem
Message-ID: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com>

In Python 2.5, list objects were special-cased to skip PyObject_GetItem and
go straight to PyList_GET_ITEM.  That special case gave made other sequences
20% slower than lists for getitem operations.  The special case was removed
in Python 3 (haven't checked 2.6).

Today I was tracing through how PyObject_GetItem works, trying to figure it
why it's so much slower, to see if we can get back some of that speed
without special-casing just one type.  Below is a rough sketch of what
PyObject_GetItem does for the common case of a sequence with a valid
positive integer parameter, and my observations on where there is room for
improvement.  I'm willing to write patches and test them, but I wanted to
get some general feedback first.  After all, it may be the way it is for A
Very Good Reason that I'm not aware of. ;-)

The code paths for PyObject_SetItem and PyObject_DelItem are essentially the
same, so any speed improvments to one could easily be applied to the other
two operations.

PyObject_GetItem ob i
    ob==NULL                      # can't be null anyway
    i==NULL                         # can't be null anyway
    is mapping? ob
    is sequence? ob
    PyIndex_Check i
        tp_as_number i
        tp_flags i
        tp_as_number->nb_index i
    PyNumber_AsSsize_t i
        PyNumber_Index i
            i==NULL                 # still can't be null
            PyLong_Check i
        i==NULL                   # still can't be null
        PyLong_Check i                 # redundant
        => v
    v == -1
    PySequence_GetItem ob v
        ob==NULL                  # still can't be null
        is sequence? ob                # redundant
        sq_item? ob
        sq_item ob v

I think there are two avenues for speed improvement, both of which involve
removing redundancy:

1) Assume the index is a PyLong until proven otherwise

The PyIndex_Check in PyObject_GetItem looks pretty useless.  If it returns
false, PyObject_GetItem throws a type error.  If we skipped the
PyIndex_Check when it would have returned false, PyNumber_AsSsize_t would
later call PyIndex_Check and throw a type error.  Unless I'm missing
something, this is a clear win and makes the code simpler.

In PyNumber_AsSsize_t, we could speed up the common case by trying
PyLong_Check first, and if it fails then calling PyNumber_Index.  This
change would make the common case faster, but make the code a few lines
longer.

2) Remove some of the exessive checking for NULL unless Py_DEBUG is defined

Many of the routines in abstract.c check their parameters for NULL, as a
sanity check, and throw a SystemError if NULL is detected.  I'm tempted to
suggest making these checks only when Py_DEBUG is defined, but I suspect if
you wanted it that way, you would have changed it already. ;)

Assuming you really want the NULL checks in production Python, there are 5
checks for NULL even though there are only two parameters.  Seems like
overkill?

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090323/25ab8055/attachment-0001.htm>

From solipsis at pitrou.net  Mon Mar 23 17:38:24 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 23 Mar 2009 16:38:24 +0000 (UTC)
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
References: <49ABCFC9.1070508@gmail.com>
	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>
	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>
	<Pine.LNX.4.64.0903130739170.2596@kimball.webabinitio.net>
	<49C79B84.9050703@simplistix.co.uk>
Message-ID: <loom.20090323T163659-779@post.gmane.org>

Chris Withers <chris <at> simplistix.co.uk> writes:
> 
> Well, python already has tools available to do exactly this.:
> buildout from a private egg repository will do exactly what you're after.
> 
> However, its built on top of setuptools, which is flawed, and it's not 
> blessed as "official core python", so there's lots of room for improvement!

Could you explain how buildout is an improvement over other systems?
Its documentation seems full of generic wording ("parts" etc.) that I can't make
sense of.

Regards

Antoine.



From ggpolo at gmail.com  Mon Mar 23 20:07:46 2009
From: ggpolo at gmail.com (Guilherme Polo)
Date: Mon, 23 Mar 2009 16:07:46 -0300
Subject: [Python-Dev] Non-Core project: IDLE
In-Reply-To: <gps1in$e1i$1@ger.gmane.org>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com> 
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net> 
	<gprp3t$icr$2@ger.gmane.org> <gps1in$e1i$1@ger.gmane.org>
Message-ID: <ac2200130903231207x16d30091g2347abb95f72efce@mail.gmail.com>

On Wed, Mar 18, 2009 at 8:53 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> IDLE needs lots of attention -- more than any one experienced person is
> likely to have
>

I'm willing to step up as a student for this but I still have to write
a good proposal for it.
My actual concern is about mentor availability, is someone around
interested on being an IDLE mentor ?

Regards,

-- 
-- Guilherme H. Polo Goncalves

From tjreedy at udel.edu  Mon Mar 23 20:12:07 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 23 Mar 2009 15:12:07 -0400
Subject: [Python-Dev] GSoC: Replace MS Windows Console with Unicode UI
In-Reply-To: <49C70C6A.7070106@g.nevcal.com>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>	<gq70q2$hul$1@ger.gmane.org>
	<49C70C6A.7070106@g.nevcal.com>
Message-ID: <gq8mu2$mfl$1@ger.gmane.org>

Glenn Linderman wrote:

> 
> One can set CMD into Unicode mode (chcp 65001)... not sure how Python 
> reacts to that either.  But even then...

I tried that and others have reported doing so on python-list but no one 
has gotten that to work.

> CMD will only use fixed-width fonts, and none of the standard XP ones 
> seem to contain all of Unicode.  Not sure if that has improved on Vista 
> or 7, as they don't run here.
> 
> It _would_ be nice to get this resolved, somehow.

Definitely.

tjr




From tseaver at palladion.com  Mon Mar 23 20:14:50 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 23 Mar 2009 15:14:50 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <loom.20090323T163659-779@post.gmane.org>
References: <49ABCFC9.1070508@gmail.com>	<49AE9698.1020806@simplistix.co.uk>	<464DE11D-FFD4-48BB-BAEF-D5D724D4A71D@fuhm.net>	<784B6E74-AD3E-4C33-879F-7A523AC715FA@solarsail.hcs.harvard.edu>	<gooa4o$rrh$1@ger.gmane.org>	<ca471dc20903050932i774f83f0ob9794d851b9f3042@mail.gmail.com>	<49B3B2E8.4050605@simplistix.co.uk>	<49B3CD53.5010906@v.loewis.de>	<49BA2E61.7070104@simplistix.co.uk>	<Pine.LNX.4.64.0903130739170.2596@kimball.webabinitio.net>	<49C79B84.9050703@simplistix.co.uk>
	<loom.20090323T163659-779@post.gmane.org>
Message-ID: <gq8n3b$nfn$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Antoine Pitrou wrote:
> Chris Withers <chris <at> simplistix.co.uk> writes:
>> Well, python already has tools available to do exactly this.:
>> buildout from a private egg repository will do exactly what you're after.
>>
>> However, its built on top of setuptools, which is flawed, and it's not 
>> blessed as "official core python", so there's lots of room for improvement!
> 
> Could you explain how buildout is an improvement over other systems?
> Its documentation seems full of generic wording ("parts" etc.) that I can't make
> sense of.

It has a couple of differentiators from a "stock" distutils or
setuptools-based installation:

 - Distributions are compiled and installed as eggs, but in a directory
   which is neither on the sys.path nor one of those marked as a 'site'
   directory.  zc.buildout *does* use the dependency information, if
   present in setup.py, to fetch dependent distributions (like
   easy_install with the --multi-version option).

 - Scripts generated from the eggs get a generated prologue which sets
   up the sys.path expressing the requirements spelled out for that
   script.

 - It externalizes much of the "how to build it" information out of
   'setup.py' into a separate "INI-style" configuration file,

 - It uses "recipes" as extensions, which enable a lot of tasks which
   are unsupported or poorly supported by distutils / setuptools (e.g.,
   installing non-Python software using "configure-make-make install",
   generating config files, etc.)


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJx9+q+gerLs4ltQ4RAlheAJ0Wq98Q3+SwgeaZthN2FrVYLyle2wCg3NiG
QX6ojgLlSmBJY1g48gMLimM=
=pkMq
-----END PGP SIGNATURE-----


From ajaksu at gmail.com  Mon Mar 23 20:14:44 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Mon, 23 Mar 2009 16:14:44 -0300
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <20090322231259.GA29189@idyll.org>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com> 
	<20090322231259.GA29189@idyll.org>
Message-ID: <2d75d7660903231214l2404a8d6j2f6903e948b89edc@mail.gmail.com>

C. Titus Brown wrote:
> Given the relative paucity of core Python GSoC ideas, I think you should
> go for both of these, *especially* if you have a mentor up front. ?So,
> write 'em up, add 'em to the GSoC page, and let's see who we get...
> If we get good applications for both, then I think we can "fund" both of
> them.

Great, thanks a lot for the feedback :)

> I do think you should be prepared for pushback from python-dev on any
> such ideas ;). ?Don't get too ambitious about writing up *your* way of
> fixing things, but rather make sure you and the students are prepared to
> adapt to what people on python-dev think. ?Mind you, ultimately the
> people doing the work should make the decisions, but input from
> python-dev is usually pretty useful even when it's contradictory!

Pushback? Luxury! :)
Thanks for the great advice, but I'd actually love negative input on
this. Better to find out early that there's no way out of the
bikeshedding tar pit than to waste a slot (and the student's time).

I think many people don't speak up for or against GSoC proposals
because they don't have time to mentor / discuss details. This
particular proposal is doomed if nobody voices the itches to be
scratched.

So I'm kinda taunting python-dev with a huge proposal that goes around
generalities and tries to make the case for joining efforts with PIDA
;)

Skipping to "Suggestions" and the "Food for thought example" might
help in deciding whether the wall of text is worth a look...

Best regards,
Daniel

==
Helper Python core development tools.

  There's some amount of repetitive, required steps to work on Python
development. This proposal is about improving the workflow of core
developers, probably with small glue scripts.  As each developer has
his own personal workflow, these helper scripts should be optional,
easy to extend and aimed at the most common tasks.
  Of course, there may be no real demand for optimizing the workflow.
Avoiding the use of a GSoC slot for unnecessary projects is very
important, so if you think it's a wast of resources, please speak up
:)

Justification
  Sometimes, non-obvious bits like the right sequence of svnmerge
commands, the right way to link a Rietveld code review to a given
issue or checking for correct autoconf version might get in the way of
real development.
  Other obvious steps, like handling issue properties (e.g.
Resolution), posting a message that tells in which revs the issue was
fixed of or even checking for changes in tabs versus spaces, also
require a bit of time.
  Regardless of obviousness, forgetting one item (or getting it wrong)
in the development checklist happens from time to time.
  If there are ways to factor these repetitive, required tasks out
from a core dev's concerns, it can only help the development process.
Non-code developers could also benefit from this kind of tool, and
could contribute in a more efficient way to Python development, with
higher code/ticket quality. Quoting the tracker cleanup proposal:
    Optimizing the application of main/core developers' time to Python issues
    is a no-brainer. Being able to add volunteers to the productive time pool
    is also very desirable.

Pre-Requisites
  Identifying which tasks and steps should be optimized cannot be left
to the student nor to the mentor, it must be a collective effort.
Clear goals must be set before someone try to implement them. Sure,
there are many descriptions of workflows in past python-dev (and
python-list) threads, but the GSoC is about code.

Methods
  It's up to the mentors/student team. I suggest offering early
releases for the scripts and maybe test repositories, trackers,
Rietveld instance, etc., as ways of making sure the resulting code
will be useful and used by the target public.

  I believe most tools should be developed in a generic way, so that
they could fit in other projects and workflows. IMHO, this would make
the resulting scripts much more valuable.

Student
  Besides some experience with Python and tools of the trade (VCSs,
bug trackers, etc.), the most important requisite is being able to
listen to the community and taking criticism well.

Mentors
  Ali Afshar from PIDA is willing to mentor if the 'generic tools' way
is accepted. Since it's  about reusing development tools inside an
IDE, any of these helper scripts could be integrated into PIDA, the
only pre-requisite being that they'd not be too deeply linked to the
Python infrastructure. With a small bit of concern about this, it'd be
easy to extend/fork the resulting tools to use with other trackers,
VCSs, test runners and code review tools.
  I am also available to help with the Python workflow part, and
there's an early effort to integrate Rietveld and our bug tracker
under way.
  At least one core Python developer should mentor, preferably one
that knows how to interact well with python-dev. Any project focused
on how people work is prone to flamefests, so diverting most heat away
from the student might be a necessary skill.
  Bikeshedding is also very likely to occur, but the proposal's goals
is to provide a couple of brushes and some paint cans of different
colo(u)rs :)

Deliverables
  Should include a couple of helper scripts, maybe some patches to
currently used tools. Other ideas?

Suggestions
  Wrappers for working with tracker issues, managing patches, running
tests, code reviews, committing (including code checking hooks),
merging, etc. E.g., I'm working on a script that allows one to submit
a Roundup issue and a Rietveld issue in a way that links them
together.

Food for thought example (sorry, not quite how a core dev works):
ajaksu at Belkar:~/py3k$ pyfix 5434 --export ../month_delta --no-assign
PyFixing...
Issue 5434: datetime.MonthDelta
Type:  feature request   Stage: not selected
Versions: 3.1            Nosy list: jess.austin,lemburg,tim_one,+ajaksu2
Assigned to: nobody (--no-assign)

SVN exporting current working copy to: ../month_delta
Export complete

Downloading files:
    monthdelta.py -> ../month_delta/patch/monthdelta.py OK
    monthdelta.diff -> ../month_delta/patch/monthdelta.diff OK
Downloaded 2 files:
monthdelta.py jess.austin, 2009-03-11 18:39 prototype implementation in python
monthdelta.diff jess.austin, 2009-03-12 17:54 updated patch, unified
diff against py3k

Downloaded 4 messages:
    ../month_delta/msgs/msg83272.txt Jess Austin: datetime is a wonderful (...)
    ../month_delta/msgs/msg83273.txt Jess Austin: This is my first try at (...)
    ../month_delta/msgs/msg83275.txt Jess Austin: Rietveld link: (...)
    ../month_delta/msgs/msg83480.txt Jess Austin: This prototype python (...)

Rietveld link found: http://codereview.appspot.com/25079

ajaksu at Belkar:~/py3k$ cd ../month_delta
ajaksu at Belkar:~/month_delta$ pyfix patch
PyFixing issue 5434...
One patch found: patch/monthdelta.diff
    jess.austin, 2009-03-12 17:54 updated patch, unified diff against py3k
Applying... OK
ajaksu at Belkar:~/month_delta$ pyfix set stage "patch review"
ajaksu at Belkar:~/month_delta$ pyfix status
PyFixing issue 5434...
Stage: not selected -> "patch review"
D    M Doc/c-api/datetime.rst                 2 chunks     34 lines
D    M Doc/library/datetime.rst              10 chunks    354 lines
B    M Include/datetime.h                     7 chunks     82 lines
T    M Lib/test/test_datetime.py              2 chunks    231 lines
D    M Misc/NEWS                              1 chunk      14 lines
B    M Modules/datetimemodule.c              20 chunks    781 lines
B: build required - T: tests - D: docs - G: backtrace

ajaksu at Belkar:~/month_delta$ pyfix test
Error: build required, try to test anyway? [n]
Test aborted
ajaksu at Belkar:~/month_delta$ pyfix build
./configure --quiet
make -s
Python build finished, but the necessary bits to build these modules
were not found:
bsddb185           sunaudiodev
ajaksu at Belkar:~/month_delta$ pyfix test
./python -E -tt ./Lib/test/regrtest.py -l  -uall -rw
344 tests OK.
25 tests skipped.
ajaksu at Belkar:~/month_delta$ pyfix up -m"LGTM, needs testing on other
platforms."
PyFixing issue 5434...
No differences found.
Setting stage: not selected -> patch review
Adding message:
    Author: Daniel Diniz
    LGTM, needs testing on other platforms.
ajaksu at Belkar:~/month_delta$

From solipsis at pitrou.net  Mon Mar 23 20:32:11 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 23 Mar 2009 19:32:11 +0000 (UTC)
Subject: [Python-Dev] GSoC: Core Python development tools
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
	<20090322231259.GA29189@idyll.org>
	<2d75d7660903231214l2404a8d6j2f6903e948b89edc@mail.gmail.com>
Message-ID: <loom.20090323T192523-285@post.gmane.org>

Daniel (ajax) Diniz <ajaksu <at> gmail.com> writes:
> 
>   Sometimes, non-obvious bits like the right sequence of svnmerge
> commands, the right way to link a Rietveld code review to a given
> issue or checking for correct autoconf version might get in the way of
> real development.

Well, really, rather than trying to improve svnmerge, we should try to find a
way forward to switch to Merc... oops, I mean what will turn out to be the best
DVCS for our needs :-)

>   I am also available to help with the Python workflow part, and
> there's an early effort to integrate Rietveld and our bug tracker
> under way.

> Food for thought example (sorry, not quite how a core dev works):
[...]
> 
> SVN exporting current working copy to: ../month_delta

Does it work when using an hg/bzr/git mirror? There's already hg and git support
in Rietveld's upload.py, so this should be possible.



From ajaksu at gmail.com  Mon Mar 23 20:56:00 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Mon, 23 Mar 2009 16:56:00 -0300
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <loom.20090323T192523-285@post.gmane.org>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com> 
	<20090322231259.GA29189@idyll.org>
	<2d75d7660903231214l2404a8d6j2f6903e948b89edc@mail.gmail.com> 
	<loom.20090323T192523-285@post.gmane.org>
Message-ID: <2d75d7660903231256yccb400etfedc98a369ce6612@mail.gmail.com>

Thanks for the feedback, Antoine!

Antoine Pitrou wrote:
> Daniel (ajax) Diniz <ajaksu <at> gmail.com> writes:
>>
>> ? Sometimes, non-obvious bits like the right sequence of svnmerge
>> commands, the right way to link a Rietveld code review to a given
>> issue or checking for correct autoconf version might get in the way of
>> real development.
>
> Well, really, rather than trying to improve svnmerge, we should try to find a
> way forward to switch to Merc... oops, I mean what will turn out to be the best
> DVCS for our needs :-)

That was a little bait for input ;)

But the real point is that, regardless of underlying VCS, there is a
choice between "having all core developers know by heart the right
switches and order of steps to correctly checkout/update ->( branch
locally) -> fix -> diff -> commit -> merge -> solve conflicts" and
"offering a little, well-documented script that takes care of the
switches and sequence of steps".

Maybe a 'untangle svnmerge-created history' tool would help too? :)

>> ? I am also available to help with the Python workflow part, and
>> there's an early effort to integrate Rietveld and our bug tracker
>> under way.
>
>> Food for thought example (sorry, not quite how a core dev works):
> [...]
>>
>> SVN exporting current working copy to: ../month_delta
>
> Does it work when using an hg/bzr/git mirror? There's already hg and git support
> in Rietveld's upload.py, so this should be possible.

Given that the pyfix script is completely imaginary ATM, yes, it works
as well with hg/bzr/git/perforce/CVS/darcs as it does with SVN :)

In a more serious note, the PIDA offer puts anyvc[1] on the table. So
we could support different VCSs as upload.py does it, or aim for a
more pluggable way, probably based on anyvc. Either way, them scripts
should be simple and follow the Unix way, so e.g. pyfix --export would
actually call anyvc --export or pyvcs --export.

Cheers,
Daniel

[1] http://pypi.python.org/pypi/anyvc

From solipsis at pitrou.net  Mon Mar 23 21:03:41 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 23 Mar 2009 20:03:41 +0000 (UTC)
Subject: [Python-Dev] GSoC: Core Python development tools
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
	<20090322231259.GA29189@idyll.org>
	<2d75d7660903231214l2404a8d6j2f6903e948b89edc@mail.gmail.com>
	<loom.20090323T192523-285@post.gmane.org>
	<2d75d7660903231256yccb400etfedc98a369ce6612@mail.gmail.com>
Message-ID: <loom.20090323T195903-980@post.gmane.org>


Hi,

Daniel (ajax) Diniz <ajaksu <at> gmail.com> writes:
> 
> But the real point is that, regardless of underlying VCS, there is a
> choice between "having all core developers know by heart the right
> switches and order of steps to correctly checkout/update ->( branch
> locally) -> fix -> diff -> commit -> merge -> solve conflicts" and
> "offering a little, well-documented script that takes care of the
> switches and sequence of steps".

Well, it seems to me that most of these steps are separated by manual
intervention (e.g. compile and run the test suite to check that everything works
smoothly), so there's no real point in making a script out of them.

The real issues with svnmerge are its occasional bugs or failures (it forgot
some changesets when merging in the io-c branch!), its slowness, and its
limitations (which are really inherent to the SVN model: e.g., if someone
commits to the branch you have just started doing an svnmerge to, you have to
revert everything and start over with the latest updates).

Regards

Antoine.



From ggpolo at gmail.com  Mon Mar 23 21:17:43 2009
From: ggpolo at gmail.com (Guilherme Polo)
Date: Mon, 23 Mar 2009 17:17:43 -0300
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <loom.20090323T195903-980@post.gmane.org>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com> 
	<20090322231259.GA29189@idyll.org>
	<2d75d7660903231214l2404a8d6j2f6903e948b89edc@mail.gmail.com> 
	<loom.20090323T192523-285@post.gmane.org>
	<2d75d7660903231256yccb400etfedc98a369ce6612@mail.gmail.com> 
	<loom.20090323T195903-980@post.gmane.org>
Message-ID: <ac2200130903231317p91d1caagb7b2de9f65107927@mail.gmail.com>

On Mon, Mar 23, 2009 at 5:03 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Hi,
>
> Daniel (ajax) Diniz <ajaksu <at> gmail.com> writes:
>>
>> But the real point is that, regardless of underlying VCS, there is a
>> choice between "having all core developers know by heart the right
>> switches and order of steps to correctly checkout/update ->( branch
>> locally) -> fix -> diff -> commit -> merge -> solve conflicts" and
>> "offering a little, well-documented script that takes care of the
>> switches and sequence of steps".
>
> Well, it seems to me that most of these steps are separated by manual
> intervention (e.g. compile and run the test suite to check that everything works
> smoothly), so there's no real point in making a script out of them.
>
> The real issues with svnmerge are its occasional bugs or failures (it forgot
> some changesets when merging in the io-c branch!),

Any chance you were not using the latest svnmerge when you did that
merge ? I've had problems like this when using older versions.

> its slowness, and its
> limitations (which are really inherent to the SVN model: e.g., if someone
> commits to the branch you have just started doing an svnmerge to, you have to
> revert everything and start over with the latest updates).
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com
>



-- 
-- Guilherme H. Polo Goncalves

From solipsis at pitrou.net  Mon Mar 23 21:26:35 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 23 Mar 2009 20:26:35 +0000 (UTC)
Subject: [Python-Dev] GSoC: Core Python development tools
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
	<20090322231259.GA29189@idyll.org>
	<2d75d7660903231214l2404a8d6j2f6903e948b89edc@mail.gmail.com>
	<loom.20090323T192523-285@post.gmane.org>
	<2d75d7660903231256yccb400etfedc98a369ce6612@mail.gmail.com>
	<loom.20090323T195903-980@post.gmane.org>
	<ac2200130903231317p91d1caagb7b2de9f65107927@mail.gmail.com>
Message-ID: <loom.20090323T202559-143@post.gmane.org>

Guilherme Polo <ggpolo <at> gmail.com> writes:
> 
> Any chance you were not using the latest svnmerge when you did that
> merge ? I've had problems like this when using older versions.

Well, actually, Benjamin did the merge, so perhaps he can give more info.

Regards

Antoine.



From benjamin at python.org  Mon Mar 23 21:28:50 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 23 Mar 2009 15:28:50 -0500
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <loom.20090323T202559-143@post.gmane.org>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
	<20090322231259.GA29189@idyll.org>
	<2d75d7660903231214l2404a8d6j2f6903e948b89edc@mail.gmail.com>
	<loom.20090323T192523-285@post.gmane.org>
	<2d75d7660903231256yccb400etfedc98a369ce6612@mail.gmail.com>
	<loom.20090323T195903-980@post.gmane.org>
	<ac2200130903231317p91d1caagb7b2de9f65107927@mail.gmail.com>
	<loom.20090323T202559-143@post.gmane.org>
Message-ID: <1afaf6160903231328j3d1f6aaof35c3a11a3eb739a@mail.gmail.com>

2009/3/23 Antoine Pitrou <solipsis at pitrou.net>:
> Guilherme Polo <ggpolo <at> gmail.com> writes:
>>
>> Any chance you were not using the latest svnmerge when you did that
>> merge ? I've had problems like this when using older versions.
>
> Well, actually, Benjamin did the merge, so perhaps he can give more info.

I was using the latest svnmerge.py from the SVN trunk. (maybe it's broken?)



-- 
Regards,
Benjamin

From tjreedy at udel.edu  Mon Mar 23 21:39:41 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 23 Mar 2009 16:39:41 -0400
Subject: [Python-Dev] Non-Core project: IDLE
In-Reply-To: <ac2200130903231207x16d30091g2347abb95f72efce@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org> <gps1in$e1i$1@ger.gmane.org>
	<ac2200130903231207x16d30091g2347abb95f72efce@mail.gmail.com>
Message-ID: <gq8s28$95b$1@ger.gmane.org>

Guilherme Polo wrote:
> On Wed, Mar 18, 2009 at 8:53 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> IDLE needs lots of attention -- more than any one experienced person is
>> likely to have
>>
> 
> I'm willing to step up as a student for this but I still have to write
> a good proposal for it.
> My actual concern is about mentor availability, is someone around
> interested on being an IDLE mentor ?

If I could, I would, and would have said so.  But I have only read about 
tk and have not actually used it.  If I did decide to dive into it, 
you'd be mentoring me ;-).  What I can and would do is give ideas for 
changes, read and comment on a proposal, and user test patched versions.

Terry


From jaustin at post.harvard.edu  Mon Mar 23 22:00:13 2009
From: jaustin at post.harvard.edu (Jess Austin)
Date: Mon, 23 Mar 2009 16:00:13 -0500
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
	currently conform
In-Reply-To: <b8ad139e0903231352i66bc38f3v4f88b0914c760df7@mail.gmail.com>
References: <b8ad139e0903231352i66bc38f3v4f88b0914c760df7@mail.gmail.com>
Message-ID: <b8ad139e0903231400h570d771bt588ccbc71a295db4@mail.gmail.com>

On Tue, Mar 3 at 19:25:21 Guido van Rossum <guido at python.org> wrote:
> On Tue, Mar 3, 2009 at 5:15 PM, Brett Cannon <brett at python.org> wrote:
>>
>>
>> On Tue, Mar 3, 2009 at 05:13, <rdmurray at bitdance.com> wrote:
>>>
>>> On Tue, 3 Mar 2009 at 06:01, Ivan Krsti?~G wrote:
>>>>
>>>> On Mar 2, 2009, at 7:08 PM, Steve Holden wrote:
>>>>>
>>>>> > > ?PS.: so is datetime.datetime a builtin then? :)
>>>>> > > ?Another historic accident. Like socket.socket. :-(
>>>>> >
>>>>> ?A pity this stuff wasn't addressed for 3.0. Way too late now, though.
>
> A pity indeed.
>
>>>> It may be too late to rename the existing accidents, but why not add
>>>> consistently-named aliases (socket.Socket, datetime.DateTime, etc) and
>>>> strongly encourage their use in new code?
>>
>> Or make the old names aliases for the new names and start a
>> PendingDeprecationWarning on the old names so they can be switched in the
>> distant future?
>
> +1, if it's not done in a rush and only for high-visibility modules --
> let's start with socket and datetime.
>
> We need a really long lead time before we can remove these. I
> recommend starting with a *silent* deprecation in 3.1 combined with a
> PR offensive for the new names.

I've uploaded a patch for the datetime module with respect to this
issue at http://bugs.python.org/issue5530 . I would appreciate it if
experienced developers could take a look at it and provide some
feedback. ?Since I've only been hacking on CPython for about a month,
please be kind! ?I'm happy to make changes to this.

As it stands now, the patch changes the current objects to have
CapWords names, and subclasses these objects to provide objects with
the old names. Use of methods (including __new__) of the derived
objects causes PendingDeprecations (if one has warning filters
appropriately set).

A warning: this patch requires the patch to the test refactoring at
Issue 5520 to completely apply. ?It will fail one test without the
patch at Issue 5516. ?Both of these are (inexpertly) linked from the
roundup page for this issue.

I hope this will be helpful.

cheers,
Jess Austin

From martin at v.loewis.de  Mon Mar 23 22:17:57 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 23 Mar 2009 22:17:57 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C79C1A.8040301@simplistix.co.uk>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BA3154.8080408@simplistix.co.uk> <49BAA596.5020106@v.loewis.de>
	<49C79C1A.8040301@simplistix.co.uk>
Message-ID: <49C7FC85.5000809@v.loewis.de>

>>>> In some environments, each new component must be approved.  Once
>>>> python is approved, the standard library is OK, but adding 7 packages
>>>> from pypi requires 7 more sets of approvals.
>>> True, but as I mentioend elsewhere, I myself haven't done a python
>>> project where I only needed python and the standard lib for many
>>> years...
>>
>> I was always able to get what I need through aptitude.
> 
> Yes, well known for its ability to run on Windows and Mac OS...

Is it? I had never heard before that aptitude works well on
Windows (although it does work on OSX).

In any case, I rarely need to install Python packages on
Windows or Mac, so this causes no problem to me. If I need
something on Windows, I usually download the Windows installer
for it.

Regards,
Martin

From ncoghlan at gmail.com  Mon Mar 23 22:21:19 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Mar 2009 07:21:19 +1000
Subject: [Python-Dev] speeding up PyObject_GetItem
In-Reply-To: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com>
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com>
Message-ID: <49C7FD4F.4020103@gmail.com>

Daniel Stutzbach wrote:
> 1) Assume the index is a PyLong until proven otherwise
> 
> The PyIndex_Check in PyObject_GetItem looks pretty useless.  If it
> returns false, PyObject_GetItem throws a type error.  If we skipped the
> PyIndex_Check when it would have returned false, PyNumber_AsSsize_t
> would later call PyIndex_Check and throw a type error.  Unless I'm
> missing something, this is a clear win and makes the code simpler.
> 
> In PyNumber_AsSsize_t, we could speed up the common case by trying
> PyLong_Check first, and if it fails then calling PyNumber_Index.  This
> change would make the common case faster, but make the code a few lines
> longer.

If this part can be done without losing any of the information currently
in the error messages, then it sounds like a decent idea.

> 2) Remove some of the exessive checking for NULL unless Py_DEBUG is defined
> 
> Many of the routines in abstract.c check their parameters for NULL, as a
> sanity check, and throw a SystemError if NULL is detected.  I'm tempted
> to suggest making these checks only when Py_DEBUG is defined, but I
> suspect if you wanted it that way, you would have changed it already. ;)
> 
> Assuming you really want the NULL checks in production Python, there are
> 5 checks for NULL even though there are only two parameters.  Seems like
> overkill?

The main problem is that many of these methods are not only used
internally, but are *also* part of the public C API made available to
extension modules. We want misuse of the latter to trigger exceptions,
not segfault the interpreter.

In theory you *could* create dual versions of the APIs, one without the
NULL check for internal use and one with the NULL check for the public
API, but that would get unmaintainable pretty fast (not to mention
speeding up the internal calls at the expense of potentially slowing
down calls from extension modules).

Redundant NULL checks within a single function are more promising
candidates for removal, but I don't believe that is happening here.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From martin at v.loewis.de  Mon Mar 23 22:21:32 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 23 Mar 2009 22:21:32 +0100
Subject: [Python-Dev] Multiprocessing on Solaris
In-Reply-To: <49C79D53.8040902@cheimes.de>
References: <gq1bo2$uov$1@ger.gmane.org>
	<49C43864.6050808@v.loewis.de>	<49C439E1.4050403@cheimes.de>
	<49C5E21C.5040607@v.loewis.de> <49C69333.9000806@cheimes.de>
	<49C70274.5010902@scottdial.com> <49C7343E.3040500@v.loewis.de>
	<49C79D53.8040902@cheimes.de>
Message-ID: <49C7FD5C.30809@v.loewis.de>

> I've started with a autoconf file for the multiprocessing backport
> package. The tests should produce the same set of definitions but I'm
> not able to test it on different platforms expect Linux.
> http://python-multiprocessing.googlecode.com/svn/trunk/configure.ac

If you want to see how it fares on the various systems which we have
build slaves for, feel free to create a branch, and then manually
ask the slaves to build branches/yourbranch.

Or, just commit it into the trunk, and see how it does.

Regards,
Martin

From v+python at g.nevcal.com  Mon Mar 23 22:29:03 2009
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Mon, 23 Mar 2009 14:29:03 -0700
Subject: [Python-Dev] GSoC: Replace MS Windows Console with Unicode UI
In-Reply-To: <gq8mu2$mfl$1@ger.gmane.org>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>	<gq70q2$hul$1@ger.gmane.org>	<49C70C6A.7070106@g.nevcal.com>
	<gq8mu2$mfl$1@ger.gmane.org>
Message-ID: <49C7FF1F.6080707@g.nevcal.com>

On approximately 3/23/2009 12:12 PM, came the following characters from 
the keyboard of Terry Reedy:
> Glenn Linderman wrote:
> 
>>
>> One can set CMD into Unicode mode (chcp 65001)... not sure how Python 
>> reacts to that either.  But even then...
> 
> I tried that and others have reported doing so on python-list but no one 
> has gotten that to work.


http://support.microsoft.com/kb/247815
http://www.microsoft.com/downloads/details.aspx?familyid=22e69ae4-7e40-4807-8a86-b3d36fab68d3&displaylang=en

(python 3)

import ctypes
k=ctypes.WinDLL('kernel32')
x = k.SetConsoleOutputCP(65001)
if x!= 1:
     print("x was ", x )
     exit( 1 )
print (''.join(chr(i) for i in range(0x410, 0x430)).encode('utf-8'))

produces a nice b'\xd0\x90\d0\x91....' stream of hex representations of 
UTF-8 encoded Unicode characters...

The only thing that seems to be missing is that Python won't emit them 
to the screen that way.

So surely some python-dev that is smarter than me, could provide that 
magic incantation.  Will go search, but that isn't in my current 
knowledge banks.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

From martin at v.loewis.de  Mon Mar 23 22:32:34 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 23 Mar 2009 22:32:34 +0100
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <49C7800E.5020502@gmail.com>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>	<20090322231259.GA29189@idyll.org>
	<49C7800E.5020502@gmail.com>
Message-ID: <49C7FFF2.9000903@v.loewis.de>

> Everything I've seen from Daniel so far seems to be about either making
> things we already do more efficient, or else providing additional
> features in ways that don't make the tools any more confusing for others
> already used to a particular way of doing things. So he seems to be
> navigating this particular minefield pretty well so far.

Yes, I'm also quite grateful for the contributions I have received from
Daniel. I hope he'll stay around for some time without exhausting.

Regards,
Martin

From ncoghlan at gmail.com  Mon Mar 23 22:39:44 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Mar 2009 07:39:44 +1000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C7FC85.5000809@v.loewis.de>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>
	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>
	<49C7FC85.5000809@v.loewis.de>
Message-ID: <49C801A0.1030008@gmail.com>

Martin v. L?wis wrote:
>>>>> In some environments, each new component must be approved.  Once
>>>>> python is approved, the standard library is OK, but adding 7 packages
>>>>> from pypi requires 7 more sets of approvals.
>>>> True, but as I mentioend elsewhere, I myself haven't done a python
>>>> project where I only needed python and the standard lib for many
>>>> years...
>>> I was always able to get what I need through aptitude.
>> Yes, well known for its ability to run on Windows and Mac OS...
> 
> Is it? I had never heard before that aptitude works well on
> Windows (although it does work on OSX).

I'm actually not quite sure if that was a sarcastic response to Chris's
original sarcasm, or if the original sarcasm just didn't cross the
language barrier very well...

> In any case, I rarely need to install Python packages on
> Windows or Mac, so this causes no problem to me. If I need
> something on Windows, I usually download the Windows installer
> for it.

I believe that was Chris's point as well - unlike Linux, getting new
packages directly from third parties rather than from a distro provided
repository is still the norm on Windows and OS X.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From martin at v.loewis.de  Mon Mar 23 22:40:23 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 23 Mar 2009 22:40:23 +0100
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <loom.20090323T195903-980@post.gmane.org>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>	<20090322231259.GA29189@idyll.org>	<2d75d7660903231214l2404a8d6j2f6903e948b89edc@mail.gmail.com>	<loom.20090323T192523-285@post.gmane.org>	<2d75d7660903231256yccb400etfedc98a369ce6612@mail.gmail.com>
	<loom.20090323T195903-980@post.gmane.org>
Message-ID: <49C801C7.4040201@v.loewis.de>

> The real issues with svnmerge are its occasional bugs or failures (it forgot
> some changesets when merging in the io-c branch!), its slowness, and its
> limitations (which are really inherent to the SVN model: e.g., if someone
> commits to the branch you have just started doing an svnmerge to, you have to
> revert everything and start over with the latest updates).

I think addressing the slowness should surely be in scope for SoC: I
would hope that rewriting it to only use a single session should provide
some speedup (i.e. through the Python bindings, rather than the command
line).

Of course, such a project might better be mentored at subversion than
Python.

Regards,
Martin

P.S. I don't believe your claim that it forgot changesets. Could it be
that it simply forgot adding files, and that it did so because you
already had the files in the sandbox, so that the merging failed?

P.P.S. Are you sure you have to re-merge when somebody commits
something unrelated to the branch? Or just when somebody else merges
as well?

From martin at v.loewis.de  Mon Mar 23 22:47:36 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 23 Mar 2009 22:47:36 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C801A0.1030008@gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>
	<49C801A0.1030008@gmail.com>
Message-ID: <49C80378.7000408@v.loewis.de>

>>>> I was always able to get what I need through aptitude.
>>> Yes, well known for its ability to run on Windows and Mac OS...
>> Is it? I had never heard before that aptitude works well on
>> Windows (although it does work on OSX).
> 
> I'm actually not quite sure if that was a sarcastic response to Chris's
> original sarcasm, or if the original sarcasm just didn't cross the
> language barrier very well...

The former, in this case.

> I believe that was Chris's point as well - unlike Linux, getting new
> packages directly from third parties rather than from a distro provided
> repository is still the norm on Windows and OS X.

Sure. However, I don't think that Chris' proposed solution (whatever
that is) would improve my life - it would likely make things worse for
me (already the introduction of eggs made the life worse for Debian
package maintainers, at least initially - i.e. for a few years).

Regards,
Martin

From ncoghlan at gmail.com  Mon Mar 23 22:47:53 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Mar 2009 07:47:53 +1000
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <49C801C7.4040201@v.loewis.de>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>	<20090322231259.GA29189@idyll.org>	<2d75d7660903231214l2404a8d6j2f6903e948b89edc@mail.gmail.com>	<loom.20090323T192523-285@post.gmane.org>	<2d75d7660903231256yccb400etfedc98a369ce6612@mail.gmail.com>	<loom.20090323T195903-980@post.gmane.org>
	<49C801C7.4040201@v.loewis.de>
Message-ID: <49C80389.9020105@gmail.com>

Martin v. L?wis wrote:
> P.P.S. Are you sure you have to re-merge when somebody commits
> something unrelated to the branch? Or just when somebody else merges
> as well?

The latter is the only one I've ever had problems with (and that was due
to me forgetting to update before merging rather than someone else
actually doing a concurrent merge).

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Mon Mar 23 23:09:56 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 23 Mar 2009 22:09:56 +0000 (UTC)
Subject: [Python-Dev] GSoC: Core Python development tools
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>	<20090322231259.GA29189@idyll.org>	<2d75d7660903231214l2404a8d6j2f6903e948b89edc@mail.gmail.com>	<loom.20090323T192523-285@post.gmane.org>	<2d75d7660903231256yccb400etfedc98a369ce6612@mail.gmail.com>
	<loom.20090323T195903-980@post.gmane.org>
	<49C801C7.4040201@v.loewis.de>
Message-ID: <loom.20090323T220746-520@post.gmane.org>

Martin v. L?wis <martin <at> v.loewis.de> writes:
> 
> P.S. I don't believe your claim that it forgot changesets. Could it be
> that it simply forgot adding files, and that it did so because you
> already had the files in the sandbox, so that the merging failed?

It's more weird actually, it actively forgot some changes in some files but some
other changes in the /same/ files did get merged. 

> P.P.S. Are you sure you have to re-merge when somebody commits
> something unrelated to the branch? Or just when somebody else merges
> as well?

Perhaps only the latter, I'm not sure actually.
(but it can also happen that the "unrelated" commit modifies some files you were
merging changes in...)




From ggpolo at gmail.com  Mon Mar 23 23:17:33 2009
From: ggpolo at gmail.com (Guilherme Polo)
Date: Mon, 23 Mar 2009 19:17:33 -0300
Subject: [Python-Dev] Non-Core project: IDLE
In-Reply-To: <gq8s28$95b$1@ger.gmane.org>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com> 
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net> 
	<gprp3t$icr$2@ger.gmane.org> <gps1in$e1i$1@ger.gmane.org> 
	<ac2200130903231207x16d30091g2347abb95f72efce@mail.gmail.com> 
	<gq8s28$95b$1@ger.gmane.org>
Message-ID: <ac2200130903231517k4115b130ga6e04f675274e56d@mail.gmail.com>

On Mon, Mar 23, 2009 at 5:39 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> Guilherme Polo wrote:
>>
>> On Wed, Mar 18, 2009 at 8:53 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>>>
>>> IDLE needs lots of attention -- more than any one experienced person is
>>> likely to have
>>>
>>
>> I'm willing to step up as a student for this but I still have to write
>> a good proposal for it.
>> My actual concern is about mentor availability, is someone around
>> interested on being an IDLE mentor ?
>
> If I could, I would, and would have said so. ?But I have only read about tk
> and have not actually used it. ?If I did decide to dive into it, you'd be
> mentoring me ;-). ?What I can and would do is give ideas for changes, read
> and comment on a proposal, and user test patched versions.
>

That is very nice Terry. Do you have some specific ideas that you want
to share publicly (or in private) about IDLE ? Your expectations about
what should be addressed first, or areas that should be improved.. you
know, anything.

The proposal I'm planning will include IDLE but it will also include
some Tkinter, since it depends on it and bugs on the later can affect
the former as you know. I was planning to first target the lack of
tests of both IDLE and Tkinter, I believe that by adding tests (and
doing it nicely) may change how future changes are applied (I'm
thinking about having them getting new tests for new features, fixes,
etc as it happens for other areas in Python) and will make easier to
maintain them. My other target is to check the open tickets in the bug
tracker regarding IDLE and Tkinter too, I have been much more active
on the later so the former will take some more time to test/think/make
a decision.

I will be able to test these changes under Linux and on plain Windows
XP, Vista and the 7 but differences between different system
configurations may also affect IDLE, so any help you can provide will
be very much appreciated. Hopefully someone with a mac will be able to
provide some help here too.

> Terry
>



-- 
-- Guilherme H. Polo Goncalves

From martin at v.loewis.de  Mon Mar 23 23:36:24 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 23 Mar 2009 23:36:24 +0100
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <loom.20090323T220746-520@post.gmane.org>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>	<20090322231259.GA29189@idyll.org>	<2d75d7660903231214l2404a8d6j2f6903e948b89edc@mail.gmail.com>	<loom.20090323T192523-285@post.gmane.org>	<2d75d7660903231256yccb400etfedc98a369ce6612@mail.gmail.com>	<loom.20090323T195903-980@post.gmane.org>	<49C801C7.4040201@v.loewis.de>
	<loom.20090323T220746-520@post.gmane.org>
Message-ID: <49C80EE8.2080702@v.loewis.de>

>> P.S. I don't believe your claim that it forgot changesets. Could it be
>> that it simply forgot adding files, and that it did so because you
>> already had the files in the sandbox, so that the merging failed?
> 
> It's more weird actually, it actively forgot some changes in some files but some
> other changes in the /same/ files did get merged. 

I see. Provided there is somebody willing to work on this, it might be
interesting to reproduce it.

>> P.P.S. Are you sure you have to re-merge when somebody commits
>> something unrelated to the branch? Or just when somebody else merges
>> as well?
> 
> Perhaps only the latter, I'm not sure actually.
> (but it can also happen that the "unrelated" commit modifies some files you were
> merging changes in...)

That should not be a problem, unless the to-be-merged changes directly
conflict. Just svn-update, then try committing again.

It's actually also possible to recover from overlapping merges also:
just re-run svnmerge with --record-only (-M); this assumes nobody else
has merged the very same changes concurrently.

Regards,
Martin

From chris at simplistix.co.uk  Mon Mar 23 23:39:28 2009
From: chris at simplistix.co.uk (Chris Withers)
Date: Mon, 23 Mar 2009 22:39:28 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C7FC85.5000809@v.loewis.de>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BA3154.8080408@simplistix.co.uk> <49BAA596.5020106@v.loewis.de>
	<49C79C1A.8040301@simplistix.co.uk> <49C7FC85.5000809@v.loewis.de>
Message-ID: <49C80FA0.4020800@simplistix.co.uk>

Martin v. L?wis wrote:
>>>>> In some environments, each new component must be approved.  Once
>>>>> python is approved, the standard library is OK, but adding 7 packages
>>>>> from pypi requires 7 more sets of approvals.
>>>> True, but as I mentioend elsewhere, I myself haven't done a python
>>>> project where I only needed python and the standard lib for many
>>>> years...
>>> I was always able to get what I need through aptitude.
>> Yes, well known for its ability to run on Windows and Mac OS...
> 
> Is it? I had never heard before that aptitude works well on
> Windows (although it does work on OSX).

Hmm, sorry, let me clarify:

<sarcasm>
Yes, well known for its ability to run on Windows and Mac OS...
</sarcasm>

;-)

aptitude also won't help when:
- your customer is deploying onto managed machines running RHEL
- debian has an outdated and/or broken version of your package.

The latter is a standard problem with Zope/Apache/Postgres in debian and 
causes much gnashing of teeth by people trying to support them. The 
Debian guys love buggering around with other people's packaging, not 
caring that it makes supporting stuff so much harder.

> In any case, I rarely need to install Python packages on
> Windows or Mac, so this causes no problem to me. If I need
> something on Windows, I usually download the Windows installer
> for it.

Yeah, 20 windows installers for different python packages whose versions 
may or may not conflict also seems a little silly ;-)

A cross-plaftorm, platform agnostic, python-centric package management 
system is what's needed. Setuptools comes close, but I wonder if it's 
impossible to get it to do the last bits of what's needed (uninstall 
being the big one, and knowing what version of what package you have 
installed!)

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk

From Scott.Daniels at Acm.Org  Mon Mar 23 23:52:50 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Mon, 23 Mar 2009 15:52:50 -0700
Subject: [Python-Dev] Non-Core project: IDLE
In-Reply-To: <ac2200130903231207x16d30091g2347abb95f72efce@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net>
	<gprp3t$icr$2@ger.gmane.org> <gps1in$e1i$1@ger.gmane.org>
	<ac2200130903231207x16d30091g2347abb95f72efce@mail.gmail.com>
Message-ID: <gq93lh$368$1@ger.gmane.org>

Guilherme Polo wrote:
> On Wed, Mar 18, 2009 at 8:53 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> IDLE needs lots of attention -- more than any one experienced person is
>> likely to have
> I'm willing to step up as a student for this but I still have to write
> a good proposal for it.
> My actual concern is about mentor availability, is someone around
> interested on being an IDLE mentor ?

You might want to talk to Bruce Sherwood, as VPython suffered through
a bunch of Idle problems.  I got the impression he had a person or two
who were his Idle experts (so I am thinking they might be the mentors
you are looking for).  In any case, even a charter of "unit tests to 50%
coverage" of Idle would be a huge improvement.

I've run after specific bugs in Idle, but don't really know the lay of
the land.


--Scott David Daniels
Scott.Daniels at Acm.Org


From greg.ewing at canterbury.ac.nz  Tue Mar 24 00:12:44 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 24 Mar 2009 11:12:44 +1200
Subject: [Python-Dev] speeding up PyObject_GetItem
In-Reply-To: <49C7FD4F.4020103@gmail.com>
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com>
	<49C7FD4F.4020103@gmail.com>
Message-ID: <49C8176C.10406@canterbury.ac.nz>

Nick Coghlan wrote:

> The main problem is that many of these methods are not only used
> internally, but are *also* part of the public C API made available to
> extension modules. We want misuse of the latter to trigger exceptions,
> not segfault the interpreter.

But is it worth slowing everything down to check for
this one particular error, among all the myriad ways
a malfunctioning extension module can cause a crash?

-- 
Greg

From martin at v.loewis.de  Tue Mar 24 00:13:35 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 24 Mar 2009 00:13:35 +0100
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <bad82a80903181537y3804623ax415265ab4ffa5664@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<1afaf6160903181421l5b1fcdd9ka36c30a68e9eabbb@mail.gmail.com>	<693bc9ab0903181459i707e3412w968bd78c8e617b50@mail.gmail.com>	<bad82a80903181512id3d9ebel4f6775a5f1e77873@mail.gmail.com>	<BAF8EF1295F44064855B6EC7A83F8B15@RaymondLaptop1>
	<bad82a80903181537y3804623ax415265ab4ffa5664@mail.gmail.com>
Message-ID: <49C8179F.1030509@v.loewis.de>

> I don't disagree, I just don't want to volunteer projects for something
> they don't want.

Right, there must be clear indication that they are willing to accept
the work when it's done.

> Note also that some of the largest Python-based projects, Django,

I have a working port of Django to Py3k, however, the Django authors
are not interested in integrating it. The same could easily happen to
other ports. I can accept that, and just wait a couple of years until
they are ready. The GSoC student can't really wait that long.

OTOH, it could be part of the student's application to get in
contact with a potential mentor, and we could prioritize porting
projects assuming the package authors indicate sufficient intent to
accept the results of the porting.

Regards,
Martin

From martin at v.loewis.de  Tue Mar 24 00:20:51 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Mar 2009 00:20:51 +0100
Subject: [Python-Dev] Core projects for Summer of Code
In-Reply-To: <fbe2e2100903181811v737a0cf6k9f8617ef9fbd1634@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>	<fbe2e2100903181523m25adb0a0g2ac79d8e1cd7977c@mail.gmail.com>	<ac2200130903181543r6a00af10s2c558b5d46611c9c@mail.gmail.com>	<fbe2e2100903181744x55d1b8e5s39a419c4b74854fc@mail.gmail.com>	<ac2200130903181755v2b720698ufec105892ab81dfa@mail.gmail.com>
	<fbe2e2100903181811v737a0cf6k9f8617ef9fbd1634@mail.gmail.com>
Message-ID: <49C81953.7060805@v.loewis.de>

> Maybe I'm misunderstanding you but I didn't mean to say that this
> version should work on both python 2.x and python 3.x. Ideally, there
> would be a PIL distribution for 2.x only and another one for 3.x only.

I don't know what Fredrik thinks, but I would not consider this ideal.
Ideally, there would be a single source that can be installed to both
2.x and 3.x. This is much better than two source trees, as the latter
would require porting of changes back and forth.

> The only thing is that people (myself included) will only be
> comfortable with the PIL for 3.x version if it comes with the
> blessings of Fredrik, i.e. if I were you I'd try pushing this with
> Fredrik. 

I don't think a GSoC project can possibly help with that.

Regards,
Martin

From brett at python.org  Tue Mar 24 00:47:21 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 23 Mar 2009 16:47:21 -0700
Subject: [Python-Dev] Issue workflow doc is now live
In-Reply-To: <9143467B-3151-4758-8DD9-EB3A1E5D89A5@python.org>
References: <bbaeab100903221620q64dd24ccrfb171e9314309c8b@mail.gmail.com> 
	<49C7810A.8060908@gmail.com>
	<9143467B-3151-4758-8DD9-EB3A1E5D89A5@python.org>
Message-ID: <bbaeab100903231647w300f1d3pae49acbb418c9842@mail.gmail.com>

On Mon, Mar 23, 2009 at 05:36, Barry Warsaw <barry at python.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Mar 23, 2009, at 8:31 AM, Nick Coghlan wrote:
>
>  Brett Cannon wrote:
>>
>>> I sent this out as a Google Doc a while back, but I just did a
>>> proof-reading, converted it, and pushed it live to the python.org
>>> <http://python.org>: http://www.python.org/dev/workflow/ . So now people
>>> who ever triage issues have a guide to follow if they are not sure how
>>> to set things.
>>>
>>
>> Does that mean PEP 3 should be Withdrawn or does it mean it should be
>> replaced?
>>
>
> It should probably be replaced with Brett's document.


I say replace as well.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090323/2cd6a8c9/attachment-0001.htm>

From brett at python.org  Tue Mar 24 00:53:17 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 23 Mar 2009 16:53:17 -0700
Subject: [Python-Dev] tracker status options
In-Reply-To: <43c8685c0903222220h6ecc841ar27ce66adf3c3d064@mail.gmail.com>
References: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com> 
	<49C1EA9E.3040603@v.loewis.de>
	<bbaeab100903191414o1710bd87x3d4c9a713c63c832@mail.gmail.com> 
	<49C2D040.4080609@v.loewis.de>
	<2d75d7660903191653v65640c90pc4df8ddcf60343f1@mail.gmail.com> 
	<43c8685c0903222220h6ecc841ar27ce66adf3c3d064@mail.gmail.com>
Message-ID: <bbaeab100903231653p2cee52e0k64a674c9ba6c948@mail.gmail.com>

2009/3/22 Tennessee Leeuwenburg <tleeuwenburg at gmail.com>

> Hi Daniel,
>
> That would be great. It occurs to me that if we wanted to use "Stage"
> settings, it would be easy to search for issues which are not closed by
> literally searching for 'not closed' rather than 'open'. I think it's also
> unclear whether the 'pending' stage means 'suspended pending additional user
> feedback' or 'resolution of this issue is impending'. My understanding was
> that it meant 'pending additional feedback' in the sense of awaiting, rather
> than imminent.
>

It's the latter; it's "pending feedback".

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090323/a35a1b16/attachment.htm>

From martin at v.loewis.de  Tue Mar 24 01:00:02 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 24 Mar 2009 01:00:02 +0100
Subject: [Python-Dev] tracker status options
In-Reply-To: <bbaeab100903231653p2cee52e0k64a674c9ba6c948@mail.gmail.com>
References: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>
	<49C1EA9E.3040603@v.loewis.de>
	<bbaeab100903191414o1710bd87x3d4c9a713c63c832@mail.gmail.com>
	<49C2D040.4080609@v.loewis.de>
	<2d75d7660903191653v65640c90pc4df8ddcf60343f1@mail.gmail.com>
	<43c8685c0903222220h6ecc841ar27ce66adf3c3d064@mail.gmail.com>
	<bbaeab100903231653p2cee52e0k64a674c9ba6c948@mail.gmail.com>
Message-ID: <49C82282.5080506@v.loewis.de>

>     That would be great. It occurs to me that if we wanted to use
>     "Stage" settings, it would be easy to search for issues which are
>     not closed by literally searching for 'not closed' rather than
>     'open'. I think it's also unclear whether the 'pending' stage means
>     'suspended pending additional user feedback' or 'resolution of this
>     issue is impending'. My understanding was that it meant 'pending
>     additional feedback' in the sense of awaiting, rather than imminent.
> 
> 
> It's the latter; it's "pending feedback".

Which "latter" (there seem to be multiple pairs in Tennessee's message)?

In any case, it's not really "pending feedback". Instead, it means "if
there is no feedback really soon, it will get closed". So closure is
impending and imminent.

Regards,
Martin

From martin at v.loewis.de  Tue Mar 24 01:01:41 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 24 Mar 2009 01:01:41 +0100
Subject: [Python-Dev] Issue workflow doc is now live
In-Reply-To: <bbaeab100903231647w300f1d3pae49acbb418c9842@mail.gmail.com>
References: <bbaeab100903221620q64dd24ccrfb171e9314309c8b@mail.gmail.com>
	<49C7810A.8060908@gmail.com>	<9143467B-3151-4758-8DD9-EB3A1E5D89A5@python.org>
	<bbaeab100903231647w300f1d3pae49acbb418c9842@mail.gmail.com>
Message-ID: <49C822E5.8090208@v.loewis.de>

>             I sent this out as a Google Doc a while back, but I just did a
>             proof-reading, converted it, and pushed it live to the
>             python.org <http://python.org>
>             <http://python.org>: http://www.python.org/dev/workflow/ .
>             So now people
>             who ever triage issues have a guide to follow if they are
>             not sure how
>             to set things.
> 
> 
>         Does that mean PEP 3 should be Withdrawn or does it mean it
>         should be
>         replaced?
> 
> 
>     It should probably be replaced with Brett's document.
> 
> 
> I say replace as well.

Will then dev/workflow be removed? I don't think we need two
copies (possibly inconsistent)? So if dev/workflow stays,
PEP 3 should be withdrawn.

Regards,
Martin

From brett at python.org  Tue Mar 24 01:03:41 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 23 Mar 2009 17:03:41 -0700
Subject: [Python-Dev] Issue workflow doc is now live
In-Reply-To: <49C822E5.8090208@v.loewis.de>
References: <bbaeab100903221620q64dd24ccrfb171e9314309c8b@mail.gmail.com> 
	<49C7810A.8060908@gmail.com>
	<9143467B-3151-4758-8DD9-EB3A1E5D89A5@python.org> 
	<bbaeab100903231647w300f1d3pae49acbb418c9842@mail.gmail.com> 
	<49C822E5.8090208@v.loewis.de>
Message-ID: <bbaeab100903231703l7ffb3c3ev1954fd382b0fa841@mail.gmail.com>

On Mon, Mar 23, 2009 at 17:01, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> >             I sent this out as a Google Doc a while back, but I just did
> a
> >             proof-reading, converted it, and pushed it live to the
> >             python.org <http://python.org>
> >             <http://python.org>: http://www.python.org/dev/workflow/ .
> >             So now people
> >             who ever triage issues have a guide to follow if they are
> >             not sure how
> >             to set things.
> >
> >
> >         Does that mean PEP 3 should be Withdrawn or does it mean it
> >         should be
> >         replaced?
> >
> >
> >     It should probably be replaced with Brett's document.
> >
> >
> > I say replace as well.
>
> Will then dev/workflow be removed? I don't think we need two
> copies (possibly inconsistent)? So if dev/workflow stays,
> PEP 3 should be withdrawn.


Sorry I wasn't clear; that's what I meant by replace; withdraw PEP 3 and
have a note pointing to /dev/workflow, not turn /dev/workflow into a new
version of PEP 3.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090323/b876e340/attachment.htm>

From stefan at codesourcery.com  Tue Mar 24 01:50:27 2009
From: stefan at codesourcery.com (Stefan Seefeld)
Date: Mon, 23 Mar 2009 20:50:27 -0400
Subject: [Python-Dev] [Roundup-devel] PSF GSoC slot for Roundup?
In-Reply-To: <2d75d7660903231517q2ed67806r40624b1db4d4418c@mail.gmail.com>
References: <2d75d7660903231517q2ed67806r40624b1db4d4418c@mail.gmail.com>
Message-ID: <49C82E53.5060509@codesourcery.com>

Daniel (ajax) Diniz wrote:
> Hi,
> I'm about to submit a proposal for the PSF to use one GSoC slot for
> the Python bug tracker. I've collected a few items I think might be
> interesting for Roundup itself, most of which are valid RFEs for our
> tracker. Feel free to add more ideas to the list below.
>   

I very much like the idea of Roundup participating in the GSoC. As I 
mentioned off-list, I'm likely to mentor another project on boost.org 
(specifically, boost.python). While I would certainly like to help, I'm 
not sure to what degree I would be able to. This may also depend on the 
proposal itself. Is there a place I need to sign up to be considered as 
a potential mentor ?

> I'd like to know what people think should be prioritized should
> Roundup get one or part of a GSoC slot. BTW, if anyone wants to submit
> a separate proposal for Roundup and is willing to mentor it, posts to
> python-dev are welcome.
>   

There are many interesting points on your wishlist. In particular, I 
like a common theme that emerges from some of those: Enhancing Roundup 
to be useful as a development tool. I have started to work on a new 
tracker template, specifically targeted at development (with a schema 
including 'milestone', and 'task'). It might be interesting to consider 
how this could be enhanced, for example to integrate with versioning 
tools such as subversion (for which Richard already worked out 
bindings), CVS, GIT, etc.

Or, integrate with automated build and test infrastructure (buildbot 
comes to mind).

Another topic for improvement is performance / optimization:

* Improve caching and instance sharing across multiple (web) requests
* Fuse multiple SQL queries to reduce the number of RDBMS calls
* ...

I'm sure there are a great many things to be worked on.
Thanks for proposing Roundup to be part in GSoC !

       Stefan


-- 
Stefan Seefeld
CodeSourcery
stefan at codesourcery.com
(650) 331-3385 x718


From tleeuwenburg at gmail.com  Tue Mar 24 02:06:29 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Tue, 24 Mar 2009 12:06:29 +1100
Subject: [Python-Dev] tracker status options
In-Reply-To: <49C82282.5080506@v.loewis.de>
References: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>
	<49C1EA9E.3040603@v.loewis.de>
	<bbaeab100903191414o1710bd87x3d4c9a713c63c832@mail.gmail.com>
	<49C2D040.4080609@v.loewis.de>
	<2d75d7660903191653v65640c90pc4df8ddcf60343f1@mail.gmail.com>
	<43c8685c0903222220h6ecc841ar27ce66adf3c3d064@mail.gmail.com>
	<bbaeab100903231653p2cee52e0k64a674c9ba6c948@mail.gmail.com>
	<49C82282.5080506@v.loewis.de>
Message-ID: <43c8685c0903231806o280a36b6r7ee007bf07e216ac@mail.gmail.com>

On Tue, Mar 24, 2009 at 11:00 AM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> >     That would be great. It occurs to me that if we wanted to use
> >     "Stage" settings, it would be easy to search for issues which are
> >     not closed by literally searching for 'not closed' rather than
> >     'open'. I think it's also unclear whether the 'pending' stage means
> >     'suspended pending additional user feedback' or 'resolution of this
> >     issue is impending'. My understanding was that it meant 'pending
> >     additional feedback' in the sense of awaiting, rather than imminent.
> >
> >
> > It's the latter; it's "pending feedback".
>
> Which "latter" (there seem to be multiple pairs in Tennessee's message)?
>
> In any case, it's not really "pending feedback". Instead, it means "if
> there is no feedback really soon, it will get closed". So closure is
> impending and imminent.


I think this indicates that the flag in not sufficiently self-descriptive.
My understanding, and I think the understanding of some others, is that this
flag indicates a suspension of development pending additional feedback,
rather the impending conclusion of development pending further feedback. In
some of my earlier emails to this list, other developers were happy to mark
issues that were being deferred as a result of requiring further
specification or clarification as "pending feedback", which is quite the
opposite of imminent closure.

While some may feel that the use of this flag is unambiguously used to
signify imminent closure, I respectfully point out the recent occasions
where confusion has occurred, and not just from a single individual.

Perhaps some developers have a well-established workflow and interpret these
flag in a particular, consistent fashion, however part of the purpose of the
issue tracker is to allow a diverse group to work on development as a group.
On that basis, I feel that more documentation, and clearer terminology, is
required in order to support the bug resolution process.

I have identified some gaps in the workflow process which impact me
personally in classifying issues. These issues will not impact on all
developers; clearly they will be entirely superfluous, perhaps even
confusing, for some individuals. However the impact still remains for some.
There appears to be a general agreement that ability to distinguish between
issues on the following bases would be useful:
   * Whether the nature of the requirement is still under debate or whether
it is well-understood
   * Whether the issue is "up for grabs" by any developer or whether
responsibility lies with an individual or group already
   * Whether the issue is ready for more serious consideration by more
experienced or core developers

Since these issues relate primarily to the long-standing, dubious or complex
issues which are not fully resolved, often of lower priority, it is apparent
that more experienced developers will not find a lot of use in the addition
of further flag, but I don't see that their workflow would be frustrated
either.

I'm happy to put my time an effort into classification of low-priority
issues, classification, and code development for areas which would probably
not otherwise receive much attention. However, to do this effectively, I
need to be able to set up additional parameters against the issues to
support the search requirements needed. Doing this on the development
tracker seems to be the best approach, seeing as there seems to be some
closely-held positions regarding the existing set of flags -- it would be
quite inappropriate to disrupt existing workflows for the more experienced
and core developers without demonstrating the value of new flags. However, I
do feel that adding flags is of very clear value to the development process
overall.

My suggestion remains to add two additional attributes, either as "stage" or
"status" options (personally I still feel 'status' is appropriate, but I
don't mind where they go):
   * Under discussion
   * Under development

This would flag "Open" issues as being up for grabs by any developer, "Under
discussion" as something which is not ready for a developer to start work on
a solution and which is still being worked out. "Under development"
similarly means that everyone who needs to be involved is already involved.
Issues that were "under discussion" or "under development" would drop back
to "Open" after a month of inactivity. Issues which could not be advanced
without further feedback would be marked as "suspended pending feedback",
and never drop back to Open. The current "pending feedback" which appears to
be used to indicate imminent closure should perhaps be altered to "pending
closure".

Thus, "Open" indicated triage is required. The default view on the issue
tracker should be "all issues not closed". The default view for a triage
nurse would be "show me everything that is open".

I'm all for more debate on options for these new flags, but I haven't heard
a whole lot of alternatives to date...

Cheers,
-Tennessee
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090324/5921bf7a/attachment-0001.htm>

From cournape at gmail.com  Tue Mar 24 02:24:33 2009
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 24 Mar 2009 10:24:33 +0900
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C80378.7000408@v.loewis.de>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BA3154.8080408@simplistix.co.uk> <49BAA596.5020106@v.loewis.de>
	<49C79C1A.8040301@simplistix.co.uk> <49C7FC85.5000809@v.loewis.de>
	<49C801A0.1030008@gmail.com> <49C80378.7000408@v.loewis.de>
Message-ID: <5b8d13220903231824s68c725d2hb8647b3255e4101e@mail.gmail.com>

On Tue, Mar 24, 2009 at 6:47 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:

>already the introduction of eggs made the life worse for Debian
> package maintainers, at least initially - i.e. for a few years.

It still is, FWIW,

David

From rdmurray at bitdance.com  Tue Mar 24 03:00:25 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 23 Mar 2009 22:00:25 -0400 (EDT)
Subject: [Python-Dev] tracker status options
In-Reply-To: <43c8685c0903222220h6ecc841ar27ce66adf3c3d064@mail.gmail.com>
References: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>
	<49C1EA9E.3040603@v.loewis.de>
	<bbaeab100903191414o1710bd87x3d4c9a713c63c832@mail.gmail.com>
	<49C2D040.4080609@v.loewis.de>
	<2d75d7660903191653v65640c90pc4df8ddcf60343f1@mail.gmail.com>
	<43c8685c0903222220h6ecc841ar27ce66adf3c3d064@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0903232156330.31981@kimball.webabinitio.net>

On Mon, 23 Mar 2009 at 16:20, Tennessee Leeuwenburg wrote:
> literally searching for 'not closed' rather than 'open'. I think it's also
> unclear whether the 'pending' stage means 'suspended pending additional user
> feedback' or 'resolution of this issue is impending'. My understanding was
> that it meant 'pending additional feedback' in the sense of awaiting, rather
> than imminent.

I understood from posts I saw go by earlier from Daniel that 'pending'
meant 'close pending unless there is feedback to the contrary' (and I
just used it that way).  It sounds like that is indeed correct but not
universally known, and thus I would suggest that at a minimum this status
be changed to 'close pending' to make it clearer.

--
R. David Murray           http://www.bitdance.com

From guido at python.org  Tue Mar 24 05:01:10 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Mar 2009 21:01:10 -0700
Subject: [Python-Dev] an unimportant question, ...
In-Reply-To: <49C693B5.8070309@stackless.com>
References: <49C6698D.4060700@stackless.com> <49C68B10.4040107@v.loewis.de>
	<49C693B5.8070309@stackless.com>
Message-ID: <ca471dc20903232101k1d8a4285y5671400d08fcc346@mail.gmail.com>

This is all over now, but I do think that the change from char to int
was made for alignment reasons. I don't know, but I could certainly
imagine that aligned memcpy calls are faster. I can't explain why I
still put the 3-byte savings comment in SVN -- perhaps I was in a
hurry.

On Sun, Mar 22, 2009 at 12:38 PM, Christian Tismer <tismer at stackless.com> wrote:
> On 3/22/09 8:01 PM, "Martin v. L?wis" wrote:
>>>
>>> Now, the internals are very clear to me. What I don't understand
>>> is where the three saved bytes should be.
>>
>> If you look at the various patches in
>>
>> http://bugs.python.org/issue576101
>>
>> then there is a three-byte saving in all versions from 1 to 6.
>> Consequentially, the API was changed in those versions (but only
>> starting from version 5, i.e. the first version created by Guido).
>>
>> For some reason, the saving was then removed from the patch that
>> got actually committed (#7). I guess the comment just stayed.
>
> Yes, funny, actually. At least, I don't find any comment why
> the char was turned into an int, after all.
> Are char pointers not on a word boundary problematic on some
> platforms?
>
> Or was it maybe to just keep the string layout on many
> common platforms compatible, in order to save rebuilding
> so many windows extension modules?
>
> If the latter is true and the only reason, I vote for reclaiming
> the three bytes. Maybe it saves a tree or two. Maybe it hurts
> very little if done for Python 3000.
>
> In any case, use the version that saves the most energy. :-)
>
> not kidding - ciao -- chris
>
> --
> Christian Tismer ? ? ? ? ? ? :^) ? <mailto:tismer at stackless.com>
> tismerysoft GmbH ? ? ? ? ? ? : ? ? Have a break! Take a ride on Python's
> Johannes-Niemeyer-Weg 9A ? ? : ? ?*Starship* http://starship.python.net/
> 14109 Berlin ? ? ? ? ? ? ? ? : ? ? PGP key -> http://wwwkeys.pgp.net/
> work +49 30 802 86 56 ?mobile +49 173 24 18 776 ?fax +49 30 80 90 57 05
> PGP 0x57F3BF04 ? ? ? 9064 F4E1 D754 C2FF 1619 ?305B C09C 5A3B 57F3 BF04
> ? ? ?whom do you want to sponsor today? ? http://www.stackless.com/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From stephen at xemacs.org  Tue Mar 24 06:10:32 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 24 Mar 2009 14:10:32 +0900
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C80FA0.4020800@simplistix.co.uk>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BA3154.8080408@simplistix.co.uk> <49BAA596.5020106@v.loewis.de>
	<49C79C1A.8040301@simplistix.co.uk> <49C7FC85.5000809@v.loewis.de>
	<49C80FA0.4020800@simplistix.co.uk>
Message-ID: <87ab7bh5fb.fsf@xemacs.org>

Chris Withers writes:

 > aptitude also won't help when:
 > - your customer is deploying onto managed machines running RHEL

True.

 > - debian has an outdated and/or broken version of your package.

True, but just as for the package system you are advocating, it's
quite easy to set up your apt to use third-party repositories of
Debian-style packages.  The question is whether those repositories
exist.  Introducing yet another, domain-specific package manager will
make it less likely that they do, and it will cause more work for
downstream distributors like Debian and RH.

 > The latter is a standard problem with Zope/Apache/Postgres in
 > debian and causes much gnashing of teeth by people trying to
 > support them. The Debian guys love buggering around with other
 > people's packaging, not caring that it makes supporting stuff so
 > much harder.

Well, I can't speak for the Debian Zope/Apache/Postgres maintainers,
but I assure you the Emacs maintainers do care.  But their hands are
full trying to implement what you keep proposing as a solution, viz.,
a "batteries-included distribution built on top of a package system".

If you don't like Debian, fine, as an upstream vendor, I don't either.
But an awful lot of my users *do* like[1] Debian (or Ubuntu or another
Debian-derived distro).  I see no alternative to cooperating with them
(though I sometimes complain loudly throughout the process<wink>).

If you want to see where the kind of thing you propose can lead, take
a look at the Debian Emacs Policy document, which is designed to deal
with one fork that takes the batteries-included approach, and another
that has gone a long way in the direction of unbundling packages.
Note that while Python doesn't have a political fork of the kind that
Emacs does, it *does* have multiple blessed technical forks, and your
suggestion involves the creation of yet more forks (ie, the
distributions with bundled packages, versus those without).  Whether
the technical differences among Python implementations and packaging
strategies will lead to the kind of issues that form the background of
the Debian Emacs Policy, I don't know.  But you don't know either.

 > A cross-plaftorm, platform agnostic, python-centric package
 > management system is what's needed.

That's what you (think you[2]) need, but that statement rudely ignores
the stated requirements of many other users.  What you are doing here
is divisive, not unifying.

 > Setuptools comes close, but I wonder if it's impossible to get it
 > to do the last bits of what's needed (uninstall being the big one,
 > and knowing what version of what package you have installed!)

Why wonder, when you can try an implementation and report the results?

I guess you mean you hope somebody else will do the work (not only of
development, but of maintaining the package repository).  Well,
"somebody else" has *already* done "the work", but unfortunately<wink>
they defined "the work" in their own way.  The result is the
batteries-included stdlib.


Footnotes: 
[1]  For values of "like" including but not limited to "see no
superior alternative to".

[2]  If "your users" include Debian and RHEL users, you may find that
they are not so happy with yet more PMS.



From steve at holdenweb.com  Tue Mar 24 06:30:44 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 24 Mar 2009 01:30:44 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <87ab7bh5fb.fsf@xemacs.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>
	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>
	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>
	<87ab7bh5fb.fsf@xemacs.org>
Message-ID: <49C87004.2030807@holdenweb.com>

Stephen J. Turnbull wrote:
> Chris Withers writes:
> 
>  > aptitude also won't help when:
>  > - your customer is deploying onto managed machines running RHEL
> 
> True.
> 
>  > - debian has an outdated and/or broken version of your package.
> 
> True, but just as for the package system you are advocating, it's
> quite easy to set up your apt to use third-party repositories of
> Debian-style packages.  The question is whether those repositories
> exist.  Introducing yet another, domain-specific package manager will
> make it less likely that they do, and it will cause more work for
> downstream distributors like Debian and RH.
> 
>  > The latter is a standard problem with Zope/Apache/Postgres in
>  > debian and causes much gnashing of teeth by people trying to
>  > support them. The Debian guys love buggering around with other
>  > people's packaging, not caring that it makes supporting stuff so
>  > much harder.
> 
> Well, I can't speak for the Debian Zope/Apache/Postgres maintainers,
> but I assure you the Emacs maintainers do care.  But their hands are
> full trying to implement what you keep proposing as a solution, viz.,
> a "batteries-included distribution built on top of a package system".
> 
> If you don't like Debian, fine, as an upstream vendor, I don't either.
> But an awful lot of my users *do* like[1] Debian (or Ubuntu or another
> Debian-derived distro).  I see no alternative to cooperating with them
> (though I sometimes complain loudly throughout the process<wink>).
> 
> If you want to see where the kind of thing you propose can lead, take
> a look at the Debian Emacs Policy document, which is designed to deal
> with one fork that takes the batteries-included approach, and another
> that has gone a long way in the direction of unbundling packages.
> Note that while Python doesn't have a political fork of the kind that
> Emacs does, it *does* have multiple blessed technical forks, and your
> suggestion involves the creation of yet more forks (ie, the
> distributions with bundled packages, versus those without).  Whether
> the technical differences among Python implementations and packaging
> strategies will lead to the kind of issues that form the background of
> the Debian Emacs Policy, I don't know.  But you don't know either.
> 
>  > A cross-plaftorm, platform agnostic, python-centric package
>  > management system is what's needed.
> 
> That's what you (think you[2]) need, but that statement rudely ignores
> the stated requirements of many other users.  What you are doing here
> is divisive, not unifying.
> 
>  > Setuptools comes close, but I wonder if it's impossible to get it
>  > to do the last bits of what's needed (uninstall being the big one,
>  > and knowing what version of what package you have installed!)
> 
> Why wonder, when you can try an implementation and report the results?
> 
> I guess you mean you hope somebody else will do the work (not only of
> development, but of maintaining the package repository).  Well,
> "somebody else" has *already* done "the work", but unfortunately<wink>
> they defined "the work" in their own way.  The result is the
> batteries-included stdlib.
> 
> 
> Footnotes: 
> [1]  For values of "like" including but not limited to "see no
> superior alternative to".
> 
> [2]  If "your users" include Debian and RHEL users, you may find that
> they are not so happy with yet more PMS.
> 
Seems to me that while all this is fine for developers and Python users
it's completely unsatisfactory for people who just want to use Python
applications. For them it's much easier if each application comes with
all dependencies including the interpreter.

This may seem wasteful, but it removes many of the version compatibility
issues that otherwise bog things down.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Tue Mar 24 06:30:44 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 24 Mar 2009 01:30:44 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <87ab7bh5fb.fsf@xemacs.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>
	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>
	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>
	<87ab7bh5fb.fsf@xemacs.org>
Message-ID: <49C87004.2030807@holdenweb.com>

Stephen J. Turnbull wrote:
> Chris Withers writes:
> 
>  > aptitude also won't help when:
>  > - your customer is deploying onto managed machines running RHEL
> 
> True.
> 
>  > - debian has an outdated and/or broken version of your package.
> 
> True, but just as for the package system you are advocating, it's
> quite easy to set up your apt to use third-party repositories of
> Debian-style packages.  The question is whether those repositories
> exist.  Introducing yet another, domain-specific package manager will
> make it less likely that they do, and it will cause more work for
> downstream distributors like Debian and RH.
> 
>  > The latter is a standard problem with Zope/Apache/Postgres in
>  > debian and causes much gnashing of teeth by people trying to
>  > support them. The Debian guys love buggering around with other
>  > people's packaging, not caring that it makes supporting stuff so
>  > much harder.
> 
> Well, I can't speak for the Debian Zope/Apache/Postgres maintainers,
> but I assure you the Emacs maintainers do care.  But their hands are
> full trying to implement what you keep proposing as a solution, viz.,
> a "batteries-included distribution built on top of a package system".
> 
> If you don't like Debian, fine, as an upstream vendor, I don't either.
> But an awful lot of my users *do* like[1] Debian (or Ubuntu or another
> Debian-derived distro).  I see no alternative to cooperating with them
> (though I sometimes complain loudly throughout the process<wink>).
> 
> If you want to see where the kind of thing you propose can lead, take
> a look at the Debian Emacs Policy document, which is designed to deal
> with one fork that takes the batteries-included approach, and another
> that has gone a long way in the direction of unbundling packages.
> Note that while Python doesn't have a political fork of the kind that
> Emacs does, it *does* have multiple blessed technical forks, and your
> suggestion involves the creation of yet more forks (ie, the
> distributions with bundled packages, versus those without).  Whether
> the technical differences among Python implementations and packaging
> strategies will lead to the kind of issues that form the background of
> the Debian Emacs Policy, I don't know.  But you don't know either.
> 
>  > A cross-plaftorm, platform agnostic, python-centric package
>  > management system is what's needed.
> 
> That's what you (think you[2]) need, but that statement rudely ignores
> the stated requirements of many other users.  What you are doing here
> is divisive, not unifying.
> 
>  > Setuptools comes close, but I wonder if it's impossible to get it
>  > to do the last bits of what's needed (uninstall being the big one,
>  > and knowing what version of what package you have installed!)
> 
> Why wonder, when you can try an implementation and report the results?
> 
> I guess you mean you hope somebody else will do the work (not only of
> development, but of maintaining the package repository).  Well,
> "somebody else" has *already* done "the work", but unfortunately<wink>
> they defined "the work" in their own way.  The result is the
> batteries-included stdlib.
> 
> 
> Footnotes: 
> [1]  For values of "like" including but not limited to "see no
> superior alternative to".
> 
> [2]  If "your users" include Debian and RHEL users, you may find that
> they are not so happy with yet more PMS.
> 
Seems to me that while all this is fine for developers and Python users
it's completely unsatisfactory for people who just want to use Python
applications. For them it's much easier if each application comes with
all dependencies including the interpreter.

This may seem wasteful, but it removes many of the version compatibility
issues that otherwise bog things down.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From martin at v.loewis.de  Tue Mar 24 08:00:19 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Mar 2009 08:00:19 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C87004.2030807@holdenweb.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>
	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>
	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>
	<87ab7bh5fb.fsf@xemacs.org> <49C87004.2030807@holdenweb.com>
Message-ID: <49C88503.2030902@v.loewis.de>

> Seems to me that while all this is fine for developers and Python users
> it's completely unsatisfactory for people who just want to use Python
> applications. For them it's much easier if each application comes with
> all dependencies including the interpreter.

I think it depends on your engineering principles. If you absolutely
have to use the latest version of any software package just because it
is there, you will certainly end up with a wobbly setup where only
a single combination of libraries can cooperate at all. OTOH, if you
develop with backwards compatibility in mind, it may be more painful
to develop, but much easier to deploy. Many of the larger libraries
and applications (Twisted, Roundup, MoinMoin) support a wide range
of Python releases, and an equally wide range of underlying libraries
(in case they need any) - so it can be done.

Of course, if you have just a single installation for your application,
developing it portably across various versions could be wasted effort.
However, in that case, I found it sufficient to just make it work on
(the Debian packages of) that single installation. Every time I upgrade
to a new Debian release, I have to see what breaks in my applications.
Most of the time, very little if any effort is needed.

So I disagree that application developers homogeneously prefer
packaging everything into a single stand-alone package. For some,
it's just not an option, for others, it's unnecessary.

I do develop applications myself, and had only once in ten years
the desire to package everything in a stand-alone way, and then ended
up using freeze. I'm genuinely curious what the scenarios are where
people desire such packaging - I did hear the desire often, but never
fully explained.

Regards,
Martin

From martin at v.loewis.de  Tue Mar 24 08:08:31 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Mar 2009 08:08:31 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C88503.2030902@v.loewis.de>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>
	<49C87004.2030807@holdenweb.com> <49C88503.2030902@v.loewis.de>
Message-ID: <49C886EF.80203@v.loewis.de>

> I do develop applications myself, and had only once in ten years
> the desire to package everything in a stand-alone way, and then ended
> up using freeze. I'm genuinely curious what the scenarios are where
> people desire such packaging - I did hear the desire often, but never
> fully explained.
> 

There is one case where I understand that desire: py2exe on Windows
(which is similar to my own case, except that I wanted to support
Solaris). It's the case where you produce an application to download
by end users, and you don't want to bother end users with installing
Python first.

However, I think that py2exe also solves this case sufficiently. One
might debate whether freeze would be better, or whether it would be
better to produce MSI instead of exe, but in essence, the result
will be always the same. I don't think a cross-platform packaging
solution can help here in any way.

Regards,
Martin

From list at qtrac.plus.com  Tue Mar 24 08:26:34 2009
From: list at qtrac.plus.com (Mark Summerfield)
Date: Tue, 24 Mar 2009 07:26:34 +0000
Subject: [Python-Dev] Non-Core project: IDLE
In-Reply-To: <ac2200130903231517k4115b130ga6e04f675274e56d@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<gq8s28$95b$1@ger.gmane.org>
	<ac2200130903231517k4115b130ga6e04f675274e56d@mail.gmail.com>
Message-ID: <200903240726.34577.list@qtrac.plus.com>

On 2009-03-23, Guilherme Polo wrote:
> On Mon, Mar 23, 2009 at 5:39 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> > Guilherme Polo wrote:
> >> On Wed, Mar 18, 2009 at 8:53 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> >>> IDLE needs lots of attention -- more than any one experienced person is
> >>> likely to have
> >>
> >> I'm willing to step up as a student for this but I still have to write
> >> a good proposal for it.
> >> My actual concern is about mentor availability, is someone around
> >> interested on being an IDLE mentor ?
> >
> > If I could, I would, and would have said so.  But I have only read about
> > tk and have not actually used it.  If I did decide to dive into it, you'd
> > be mentoring me ;-).  What I can and would do is give ideas for changes,
> > read and comment on a proposal, and user test patched versions.
>
> That is very nice Terry. Do you have some specific ideas that you want
> to share publicly (or in private) about IDLE ? Your expectations about
> what should be addressed first, or areas that should be improved.. you
> know, anything.

I have one suggestion that I think might be widely appreciated:

Add somewhere in the configuration dialog when users can enter a block
of Python code to be executed at startup and whenever Restart Shell is
executed.

Use case: for people who use IDLE for calculations/experiments they
might like to always have certain module imported. For me personally, it
would be:

    import os
    import re
    import sys
    from math import *

but of course the whole point is that people can write any code they
like. (Some people might want to do various from __future__ imports in
Python 2.6 to get various Python 3 features for example.)

I know that you can use the -c option, but that only works at startup,
not every time you Restart Shell.

[snip]

-- 
Mark Summerfield, Qtrac Ltd, www.qtrac.eu
    C++, Python, Qt, PyQt - training and consultancy
        "Rapid GUI Programming with Python and Qt" - ISBN 0132354187


From stephen at xemacs.org  Tue Mar 24 09:04:30 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 24 Mar 2009 17:04:30 +0900
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C87004.2030807@holdenweb.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BA3154.8080408@simplistix.co.uk> <49BAA596.5020106@v.loewis.de>
	<49C79C1A.8040301@simplistix.co.uk> <49C7FC85.5000809@v.loewis.de>
	<49C80FA0.4020800@simplistix.co.uk> <87ab7bh5fb.fsf@xemacs.org>
	<49C87004.2030807@holdenweb.com>
Message-ID: <873ad3gxdd.fsf@xemacs.org>

Steve Holden writes:

 > Seems to me that while all this is fine for developers and Python
 > users it's completely unsatisfactory for people who just want to
 > use Python applications.

?  I use lots of Python applications under Mac OS X, MacPorts, Debian,
and Gentoo without looking too closely at what the cat drags in.  In
fact, in many cases I don't even know and don't care that they're
Python apps.  As long as the cat stays in her corner, I'm happy.

 > For them it's much easier if each application comes with all
 > dependencies including the interpreter.  This may seem wasteful,
 > but it removes many of the version compatibility issues that
 > otherwise bog things down.

For that version of the application.  As soon as you start thinking
about version upgrades of the application, the package distribution
problem reappears (in the sense that somebody must manage a whole set
of packages in the application's installed tree, make sure they get
removed at upgrade, deal with multiple concurrent installations so
that beta versions can be tested and compared with one's current
production installation, etc).  This is really the same as the problem
that Python currently faces and solves with the batteries-included
stdlib, just specialized to a particular application.  But those
applications are similarly likely to experience bitrot in their
bundled dependencies, and *some* but not all users will find that
somewhere between annoying and disastrous, just like the
Python-with-stdlib distribution itself.  Degree of each kind of issue
will change, of course, but the issues do remain.

As far as I can *all* of these various solutions have their uses, and
each is going to be strongly favored by some groups of users, and more
or less unacceptable to others.  These preferences may or may not be
correlated with specific apps written in Python.  "Batteries included"
is the status quo for the (C) Python project, and I personally don't
see a strong reason to change that policy until those who favor a
package-manager-based solution have a package manager that satisfies
them!

From hrvoje.niksic at avl.com  Tue Mar 24 10:30:46 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Tue, 24 Mar 2009 10:30:46 +0100
Subject: [Python-Dev] speeding up PyObject_GetItem
In-Reply-To: <6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001>
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com>
	<6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001>
Message-ID: <49C8A846.4020507@avl.com>

Nick Coghlan wrote:
>> Many of the routines in abstract.c check their parameters for NULL, as a
>> sanity check, and throw a SystemError if NULL is detected.  I'm tempted
>> to suggest making these checks only when Py_DEBUG is defined, but I
>> suspect if you wanted it that way, you would have changed it already. ;)
>> 
>> Assuming you really want the NULL checks in production Python, there are
>> 5 checks for NULL even though there are only two parameters.  Seems like
>> overkill?
> 
> The main problem is that many of these methods are not only used
> internally, but are *also* part of the public C API made available to
> extension modules. We want misuse of the latter to trigger exceptions,
> not segfault the interpreter.

Agreed, and more importantly, I have yet to be convinced that those NULL 
checks introduce a measurable slowdown.  Daniel, have you tried 
measuring the performance difference with only the NULL checks removed?

From solipsis at pitrou.net  Tue Mar 24 10:39:19 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 24 Mar 2009 09:39:19 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?speeding_up_PyObject=5FGetItem?=
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com>
	<6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001>
	<49C8A846.4020507@avl.com>
Message-ID: <loom.20090324T093753-701@post.gmane.org>

Hrvoje Niksic <hrvoje.niksic <at> avl.com> writes:
> 
> Agreed, and more importantly, I have yet to be convinced that those NULL 
> checks introduce a measurable slowdown.  Daniel, have you tried 
> measuring the performance difference with only the NULL checks removed?

I think it would be fine to add a special case for lists (*) and dicts in
PyObject_GetItem(), provided it does make a significant difference in
performance for these two types.

(*) only for integers, not for slices

Regards

Antoine.



From python at rcn.com  Tue Mar 24 10:46:31 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 24 Mar 2009 02:46:31 -0700
Subject: [Python-Dev] speeding up PyObject_GetItem
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com><6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001><49C8A846.4020507@avl.com>
	<loom.20090324T093753-701@post.gmane.org>
Message-ID: <B75A6EBB17DB414AA6685A761A5B23A7@RaymondLaptop1>

>> Agreed, and more importantly, I have yet to be convinced that those NULL 
>> checks introduce a measurable slowdown.  Daniel, have you tried 
>> measuring the performance difference with only the NULL checks removed?

I think it highly unlikely that there is a performance difference.
These tend to branch the same way every time, so the 
processor's branch prediction will tend to reduce the check time
to near zero. 


> I think it would be fine to add a special case for lists (*) and dicts in
> PyObject_GetItem(), provided it does make a significant difference in
> performance for these two types.

-1

The API confusion and clutter isn't worth the micro-optimization.
Also, the checks probably do have some value in early detection
of programming errors; it would be ashamed to lose them in 
non-debug builds.  When we get bug reports that are due to 
problems with third-party extensions, it will be harder to know
whether the issue is with the extension or with us.


Raymond

From solipsis at pitrou.net  Tue Mar 24 10:52:29 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 24 Mar 2009 09:52:29 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?speeding_up_PyObject=5FGetItem?=
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com><6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001><49C8A846.4020507@avl.com>
	<loom.20090324T093753-701@post.gmane.org>
	<B75A6EBB17DB414AA6685A761A5B23A7@RaymondLaptop1>
Message-ID: <loom.20090324T095013-362@post.gmane.org>

Raymond Hettinger <python <at> rcn.com> writes:
> 
> -1
> 
> The API confusion and clutter isn't worth the micro-optimization.

The API wouldn't change, there would only be a short path for long-indexing of
lists, exactly as there is today in 2.x's BINARY_SUBSCR implementation.



From rdmurray at bitdance.com  Tue Mar 24 11:23:47 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 24 Mar 2009 06:23:47 -0400 (EDT)
Subject: [Python-Dev]  tracker status options
Message-ID: <Pine.LNX.4.64.0903240514490.31981@kimball.webabinitio.net>

On Tue Mar 24 02:06:29 CET 2009, Tennessee Leeuwenburg <tleeuwenburg at gmail.com> wrote:
> Perhaps some developers have a well-established workflow and interpret these
> flag in a particular, consistent fashion, however part of the purpose of the
> issue tracker is to allow a diverse group to work on development as a group.
> On that basis, I feel that more documentation, and clearer terminology, is
> required in order to support the bug resolution process.
> 
> I have identified some gaps in the workflow process which impact me
> personally in classifying issues. These issues will not impact on all
> developers; clearly they will be entirely superfluous, perhaps even
> confusing, for some individuals. However the impact still remains for some.
> There appears to be a general agreement that ability to distinguish between
> issues on the following bases would be useful:
>    * Whether the nature of the requirement is still under debate or whether
> it is well-understood

So, having triaged a few issues, here are my thoughts.

The current workflow is roughly:

     o test needed
     o patch needed
     o patch review
     o commit review

One can look at these and see what needs to be done "next".  I think
that in practice the above list actually expands something like this:

     o consensus needed
     o test needed
     o patch needed
     o patch needs work
     o patch review
     o commit review

The first of these additional items is equivalent to your bullet item
above.  I would propose that the issue, regardless of whether or not
it is a bug fix or a feature request, goes into 'consensus needed'
whenever there is significant debate as to the validity of the bug or
feature request.  It moves to an appropriate later stage when agreement
has been reached (which may be by fiat of a developer or the BDFL....so
the triage people would need to know who can "pronounce").

The second addition is not as clearly useful. 'patch needs work' could
be the result of a developer review, or of any other review.  That is, at
that stage we are trying to reach consensus that the patch is the correct
solution to the problem or feature request.  An issue could bounce back
and forth between 'patch review' and 'patch needs work' multiple times
(and it would probably be best if the patch submitter could request
'patch review').  But one could argue that the issue could just as easily
be bounced back and forth between 'patch review' and 'patch needed',
since one would need to read the comment stream to figure out what was
actually going on anyway.  I think it would be a useful addition, since
it would enable someone to search for 'patch needed' in order to look
for issues to pick up, whereas 'patch needs work' would indicate someone
was working on it.  (Of course, that someone could also stop working
on it...but if one wished to find such issues, one could simply sort
'patch needs work' issues by last activity date.)  Hmm.  Or perhaps it
should be "patch needs consensus", given the issue I'm looking at where
I want to set it to this stage :)

>    * Whether the issue is "up for grabs" by any developer or whether
> responsibility lies with an individual or group already

Isn't that covered by 'assigned to'?

>    * Whether the issue is ready for more serious consideration by more
> experienced or core developers

Hmm.  Theoretically that's covered by 'patch review'.  Given that
'commit review' is (currently?) reserved for issues being considered
for addition to a release candidate or final release, perhaps we need
an additional stage 'core review' that would come after 'patch review'.
Then triage could promote issues from 'patch review' to 'core review'
if triage thinks it is ready for review by someone with commit privileges.

This of course assumes that people other than core developers are actually
doing patch review.  I certainly intend to, but how many of us are there
really going to be?

--
R. David Murray           http://www.bitdance.com

From steve at holdenweb.com  Tue Mar 24 12:53:23 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 24 Mar 2009 07:53:23 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C886EF.80203@v.loewis.de>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>	<49C87004.2030807@holdenweb.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
Message-ID: <49C8C9B3.3070403@holdenweb.com>

Martin v. L?wis wrote:
>> I do develop applications myself, and had only once in ten years
>> the desire to package everything in a stand-alone way, and then ended
>> up using freeze. I'm genuinely curious what the scenarios are where
>> people desire such packaging - I did hear the desire often, but never
>> fully explained.
>>
> 
> There is one case where I understand that desire: py2exe on Windows
> (which is similar to my own case, except that I wanted to support
> Solaris). It's the case where you produce an application to download
> by end users, and you don't want to bother end users with installing
> Python first.
> 
> However, I think that py2exe also solves this case sufficiently. One
> might debate whether freeze would be better, or whether it would be
> better to produce MSI instead of exe, but in essence, the result
> will be always the same. I don't think a cross-platform packaging
> solution can help here in any way.
> 
I'm not convinced we do need a cross-platform packaging solution, so I
may have explained my views badly. I regard application developers as
Python users, so I did not intend to suggest that the requirement for
stand-alone installation came from them.

My main concern is that if Linux and Unix (Lunix) application
installation results, as is the case with setuptools, in the download
and/or installation of arbitrary support packages then we may end up
condemning Python app users to our own version of DLL hell (package
purgatory?).

As far as Lunix distributions are concerned I'd be happy enough to rely
on the distributed Python, and let people who replace it suffer the
consequences and maintain their own Python packages as required. So I
suppose that we *could* take the interpreter as a given.

But I don't think that a "normal" Lunix user should ever have to involve
themselves directly with distutils or setuptools (it's OK to invoke them
from the installation routine, but not to require the user to unpack the
application then run setup.py). I am afraid that distutils, and
setuptools, are not really the answer to the problem, since while they
may (as intended) guarantee that Python applications can be installed
uniformly across different platforms they also more or less guarantee
that Python applications are installed differently from all other
applications on the platform.

Mike Driscoll did some work providing Windows installers for various
Python packages and extension modules, and people were amused that he
provided executable installers for pure Python libraries. But I saw that
as a sensible decision, since it meant that Windows users (and even
Windows Python application developers) used the same mechanism to
install everything.

I don't claim to have all the answers, but distutils and setuptools are
for the savvy users - people who build distros and those who maintain
their own systems independent of their distribution's packaging
standard. As time goes by, however, and the Lunix installed base
continues to grow, the majority of users will expect to rely on the
standard installation mechanisms of their distribution, and that will
never be setuptools or distutils.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Tue Mar 24 12:53:23 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 24 Mar 2009 07:53:23 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C886EF.80203@v.loewis.de>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>	<49C87004.2030807@holdenweb.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
Message-ID: <49C8C9B3.3070403@holdenweb.com>

Martin v. L?wis wrote:
>> I do develop applications myself, and had only once in ten years
>> the desire to package everything in a stand-alone way, and then ended
>> up using freeze. I'm genuinely curious what the scenarios are where
>> people desire such packaging - I did hear the desire often, but never
>> fully explained.
>>
> 
> There is one case where I understand that desire: py2exe on Windows
> (which is similar to my own case, except that I wanted to support
> Solaris). It's the case where you produce an application to download
> by end users, and you don't want to bother end users with installing
> Python first.
> 
> However, I think that py2exe also solves this case sufficiently. One
> might debate whether freeze would be better, or whether it would be
> better to produce MSI instead of exe, but in essence, the result
> will be always the same. I don't think a cross-platform packaging
> solution can help here in any way.
> 
I'm not convinced we do need a cross-platform packaging solution, so I
may have explained my views badly. I regard application developers as
Python users, so I did not intend to suggest that the requirement for
stand-alone installation came from them.

My main concern is that if Linux and Unix (Lunix) application
installation results, as is the case with setuptools, in the download
and/or installation of arbitrary support packages then we may end up
condemning Python app users to our own version of DLL hell (package
purgatory?).

As far as Lunix distributions are concerned I'd be happy enough to rely
on the distributed Python, and let people who replace it suffer the
consequences and maintain their own Python packages as required. So I
suppose that we *could* take the interpreter as a given.

But I don't think that a "normal" Lunix user should ever have to involve
themselves directly with distutils or setuptools (it's OK to invoke them
from the installation routine, but not to require the user to unpack the
application then run setup.py). I am afraid that distutils, and
setuptools, are not really the answer to the problem, since while they
may (as intended) guarantee that Python applications can be installed
uniformly across different platforms they also more or less guarantee
that Python applications are installed differently from all other
applications on the platform.

Mike Driscoll did some work providing Windows installers for various
Python packages and extension modules, and people were amused that he
provided executable installers for pure Python libraries. But I saw that
as a sensible decision, since it meant that Windows users (and even
Windows Python application developers) used the same mechanism to
install everything.

I don't claim to have all the answers, but distutils and setuptools are
for the savvy users - people who build distros and those who maintain
their own systems independent of their distribution's packaging
standard. As time goes by, however, and the Lunix installed base
continues to grow, the majority of users will expect to rely on the
standard installation mechanisms of their distribution, and that will
never be setuptools or distutils.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From daniel at stutzbachenterprises.com  Tue Mar 24 13:30:03 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 24 Mar 2009 07:30:03 -0500
Subject: [Python-Dev] speeding up PyObject_GetItem
In-Reply-To: <49C8A846.4020507@avl.com>
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com>
	<6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001>
	<49C8A846.4020507@avl.com>
Message-ID: <eae285400903240530x2ecd50f7h53212bae422a8866@mail.gmail.com>

On Tue, Mar 24, 2009 at 4:30 AM, Hrvoje Niksic <hrvoje.niksic at avl.com>wrote:

> Agreed, and more importantly, I have yet to be convinced that those NULL
> checks introduce a measurable slowdown.  Daniel, have you tried measuring
> the performance difference with only the NULL checks removed?


I'll play around with different permutations and report back on their
impact.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090324/332d18da/attachment.htm>

From rdmurray at bitdance.com  Tue Mar 24 13:45:27 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 24 Mar 2009 08:45:27 -0400 (EDT)
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C8C9B3.3070403@holdenweb.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BA3154.8080408@simplistix.co.uk> <49BAA596.5020106@v.loewis.de>
	<49C79C1A.8040301@simplistix.co.uk> <49C7FC85.5000809@v.loewis.de>
	<49C80FA0.4020800@simplistix.co.uk> <87ab7bh5fb.fsf@xemacs.org>
	<49C87004.2030807@holdenweb.com> <49C88503.2030902@v.loewis.de>
	<49C886EF.80203@v.loewis.de> <49C8C9B3.3070403@holdenweb.com>
Message-ID: <Pine.LNX.4.64.0903240831390.31981@kimball.webabinitio.net>

On Tue, 24 Mar 2009 at 07:53, Steve Holden wrote:
> I don't claim to have all the answers, but distutils and setuptools are
> for the savvy users - people who build distros and those who maintain
> their own systems independent of their distribution's packaging
> standard. As time goes by, however, and the Lunix installed base
> continues to grow, the majority of users will expect to rely on the
> standard installation mechanisms of their distribution, and that will
> never be setuptools or distutils.

+1.

Or any other Python-only package manager.

And that goes at least double for Windows :)

I am (among other things) a software developer, and I am quite capable of
maintaining my own python installation outside the system one, and do so
for testing and development.  But for anything else, I rely on the system
package manager to install my Python packages.  If the extension hasn't
been packaged for my distribution, I typically will either not use it,
or I will install it somewhere other than the system default location.

--
R. David Murray           http://www.bitdance.com

From solipsis at pitrou.net  Tue Mar 24 14:49:28 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 24 Mar 2009 13:49:28 +0000 (UTC)
Subject: [Python-Dev] Test failures under Windows?
Message-ID: <loom.20090324T134451-902@post.gmane.org>

Hello,

I've just tried running test_io under Windows (in a virtual machine) and I get
the following failures:

======================================================================
ERROR: test_error_through_destructor (test.test_io.CTextIOWrapperTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Z:\py3k\__svn__\lib\test\test_io.py", line 1543, in test_error_through_d
estructor
    self.assertRaises(AttributeError, f)
  File "Z:\py3k\__svn__\lib\unittest.py", line 457, in failUnlessRaises
    callableObj(*args, **kwargs)
  File "Z:\py3k\__svn__\lib\test\test_io.py", line 1541, in f
    self.TextIOWrapper(rawio).xyzzy
OSError: [Errno 9] Bad file descriptor

======================================================================
ERROR: test_error_through_destructor (test.test_io.PyTextIOWrapperTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Z:\py3k\__svn__\lib\test\test_io.py", line 1543, in test_error_through_d
estructor
    self.assertRaises(AttributeError, f)
  File "Z:\py3k\__svn__\lib\unittest.py", line 457, in failUnlessRaises
    callableObj(*args, **kwargs)
  File "Z:\py3k\__svn__\lib\test\test_io.py", line 1541, in f
    self.TextIOWrapper(rawio).xyzzy
  File "Z:\py3k\__svn__\lib\_pyio.py", line 1356, in __init__
    encoding = os.device_encoding(buffer.fileno())
OSError: [Errno 9] Bad file descriptor

----------------------------------------------------------------------

Am I the only one getting those?
By the way, what happened to the Windows buildbots?

Regards

Antoine.



From cournape at gmail.com  Tue Mar 24 15:00:11 2009
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 24 Mar 2009 23:00:11 +0900
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C8C9B3.3070403@holdenweb.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BAA596.5020106@v.loewis.de> <49C79C1A.8040301@simplistix.co.uk>
	<49C7FC85.5000809@v.loewis.de> <49C80FA0.4020800@simplistix.co.uk>
	<87ab7bh5fb.fsf@xemacs.org> <49C87004.2030807@holdenweb.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com>
Message-ID: <5b8d13220903240700l72ab6e5fk479bd2fa1cdfd958@mail.gmail.com>

On Tue, Mar 24, 2009 at 8:53 PM, Steve Holden <steve at holdenweb.com> wrote:

> I'm not convinced we do need a cross-platform packaging solution, so I
> may have explained my views badly. I regard application developers as
> Python users, so I did not intend to suggest that the requirement for
> stand-alone installation came from them.
>
> My main concern is that if Linux and Unix (Lunix) application
> installation results, as is the case with setuptools, in the download
> and/or installation of arbitrary support packages then we may end up
> condemning Python app users to our own version of DLL hell (package
> purgatory?).

There already is a bit of a DLL hell in python. The whole idea to
solve the dependency problems by installing multiple version of the
same software is fundamentally flawed, it just does not work for
general deployment on multiple machines. Many systems outside python,
with more resource, have tried - and failed. By enabling a general,
system-wide installation of multiple version of the same package,
setuptools has made the situation worse. I am quite puzzled than many
people don't realize this fundamental issue, it is a simple
combinatory problem.

If the problem is to get a recent enough version of the library, then
the library would better be installed "locally", for the application.
If it is too much a problem because the application depends on
billions of libraries which are 6 months old, the problem is to allow
such a dependency in the first place. What kind of nightmare would it
be if programs developed in C would required a C library which is 6
months old ? That's exactly what multiple-versions installations
inflict on us. That's great for testing, development. But for
deployment on end-user machines, the whole thing is a failure IMO.

> I am afraid that distutils, and
> setuptools, are not really the answer to the problem, since while they
> may (as intended) guarantee that Python applications can be installed
> uniformly across different platforms they also more or less guarantee
> that Python applications are installed differently from all other
> applications on the platform.

I think they should be part of the solution, in the sense that they
should allow easier packaging for the different platforms (linux,
windows, mac os x and so on). For now, they make things much harder
than they should (difficult to follow the FHS, etc...). But otherwise,
I agree. Python applications which care about non-savy users should be
distributed as .dmg, .exe, .rpm, .deb. There will always be some work
to do that correctly: there is no way to provide a general, automatic
environment to build installers which provide a good experience on all
platforms. AFAIK, It does not even exist in the commercial landscape,
so I see little chance to see this in the python ecosystem.

cheers,

David

From ajaksu at gmail.com  Tue Mar 24 15:20:22 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 24 Mar 2009 11:20:22 -0300
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <49C7800E.5020502@gmail.com>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com> 
	<20090322231259.GA29189@idyll.org> <49C7800E.5020502@gmail.com>
Message-ID: <2d75d7660903240720m7b15c234y66a59af86d7c77f9@mail.gmail.com>

Nick Coghlan wrote:
> Everything I've seen from Daniel so far seems to be about either making
> things we already do more efficient, or else providing additional
> features in ways that don't make the tools any more confusing for others
> already used to a particular way of doing things. So he seems to be
> navigating this particular minefield pretty well so far.

Thanks a lot Nick! :)

BTW, it seems there's a procedure to follow if my navigation fails[1].

> Then again, I may be a little biased - some of the recent bug tracker
> changes are things I had wished for in the past, but never chipped in
> any code to help make them happen :)

Not a problem, sir, we accept RFEs devoid of any code bits in this store :)

Cheers,
Daniel

[1]
George: If we do happen to step on a mine, Sir, what do we do?

Capt. Blackadder: Normal procedure, Lieutenant, is to jump two hundred
feet in the air and scatter oneself over a wide area.

From exarkun at divmod.com  Tue Mar 24 15:33:46 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Tue, 24 Mar 2009 09:33:46 -0500
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <loom.20090324T134451-902@post.gmane.org>
Message-ID: <20090324143346.12853.1847249346.divmod.quotient.27372@henry.divmod.com>

On Tue, 24 Mar 2009 13:49:28 +0000 (UTC), Antoine Pitrou <solipsis at pitrou.net> wrote:
>Hello,
>
> [snip]
>By the way, what happened to the Windows buildbots?

It looks like some of them are suffering from problems which I think are
common with buildbot on Windows - primarily difficulty dealing with runaway
processes or timeouts.  Perhaps BuildBot/Windows improvements would make a
good GSoC project? :)

Jean-Paul

From ajaksu at gmail.com  Tue Mar 24 15:34:48 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 24 Mar 2009 11:34:48 -0300
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <49C7FFF2.9000903@v.loewis.de>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com> 
	<20090322231259.GA29189@idyll.org> <49C7800E.5020502@gmail.com> 
	<49C7FFF2.9000903@v.loewis.de>
Message-ID: <2d75d7660903240734t68e54123ted84499b3b819430@mail.gmail.com>

"Martin v. L?wis" wrote:
> Yes, I'm also quite grateful for the contributions I have received from
> Daniel.

Thank you Martin. I'm sure I'd still be going around in circles if it
weren't for your guidance, and I'd be MIA after the first time I broke
the tracker too. So thanks a lot for the support.

> I hope he'll stay around for some time without exhausting.
Me too. I'm trying to leave a big audit trail so other people can join
efforts or take over parts of roadmap easily, but that's also a backup
for long fieldwork trips or the eventual burn out.

Best regards,
Daniel

From a.badger at gmail.com  Tue Mar 24 15:35:03 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Tue, 24 Mar 2009 07:35:03 -0700
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <87ab7bh5fb.fsf@xemacs.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>
	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>
	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>
	<87ab7bh5fb.fsf@xemacs.org>
Message-ID: <49C8EF97.8000608@gmail.com>

Stephen J. Turnbull wrote:
> Chris Withers writes:
>
>  > - debian has an outdated and/or broken version of your package.
>
> True, but just as for the package system you are advocating, it's
> quite easy to set up your apt to use third-party repositories of
> Debian-style packages.  The question is whether those repositories
> exist.  Introducing yet another, domain-specific package manager will
> make it less likely that they do, and it will cause more work for
> downstream distributors like Debian and RH.
>
I haven't seen this mentioned so --

For many sites (including Fedora, the one I work on), the site maintains
a local yum/apt repository of packages that are necessary for getting
certain applications to run.  This way we are able to install a system
with a distribution that is maintained by other people and have local
additions that add more recent versions only where necessary.  This has
the following advantages:

1) We're able to track our changes to the base OS.
2) If the OS vendor releases an update that includes our fixes, we're
able to consume it without figuring out on which boxes we have to delete
what type of locally installed file (egg, jar, gem,
/usr/local/bin/program, etc).
3) We're using the OS vendor package management system for everything so
junior system admins can bootstrap a new machine with only familiarity
with that OS.  We don't have to teach them about rpm + eggs + gems +
where to find our custom repositories of each.
4) If we choose to, we can separate out different repositories for
different sets of machines.  Currently we have the main local repo and
one repo that only the builders pull from.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090324/7b296b8d/attachment.pgp>

From daniel at stutzbachenterprises.com  Tue Mar 24 15:42:42 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 24 Mar 2009 09:42:42 -0500
Subject: [Python-Dev] speeding up PyObject_GetItem
In-Reply-To: <eae285400903240530x2ecd50f7h53212bae422a8866@mail.gmail.com>
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com>
	<6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001>
	<49C8A846.4020507@avl.com>
	<eae285400903240530x2ecd50f7h53212bae422a8866@mail.gmail.com>
Message-ID: <eae285400903240742o1489f3c1veffae75e4493b073@mail.gmail.com>

On Tue, Mar 24, 2009 at 7:30 AM, Daniel Stutzbach <
daniel at stutzbachenterprises.com> wrote:

> On Tue, Mar 24, 2009 at 4:30 AM, Hrvoje Niksic <hrvoje.niksic at avl.com>wrote:
>
>> Agreed, and more importantly, I have yet to be convinced that those NULL
>> checks introduce a measurable slowdown.  Daniel, have you tried measuring
>> the performance difference with only the NULL checks removed?
>
>
> I'll play around with different permutations and report back on their
> impact.


After playing around with it, I see that I was barking up the wrong tree.
That's what I get for tracing through code visually instead of using a
debugger.

PyList implements tp_as_mapping, so it's mp_subscript method is called and
the PySequence line is never used.  I tried many variations of special
casing at various points in the path and removing extra checks.

It looks like function calls are the biggest expense.  Calls to functions
within the same module appear to be cheap (presumably because gcc inlines
them).

100 nanoseconds, py3k trunk:
ceval -> PyObject_GetItem (object.c) -> list_subscript (listobject.c) ->
PyNumber_AsSsize_t (object.c) -> PyLong_AsSsize_t (longobject.c)

86 nanoseconds, by special-casing PyLong in list_subscript
ceval -> PyObject_GetItem (object.c) -> list_subscript (listobject.c) ->
PyLong_AsSsize_t (longobject.c)
cost: could slow down
mylist[some_PyNumber_that_is_not_a_long_yet_still_a_valid_index] (who
cares?)

75 nanoseconds, by special-casing PyList and PyLong in PyObject and using
PyList_GET_ITEM
ceval -> PyObject_GetItem (object.c) -> PyLong_AsSsize_t (longobject.c)
cost: could slow down not_a_list[x]

75 nanoseconds, by special-casing PyList and PyLong in ceval and using
PyList_GET_ITEM
ceval -> PyLong_AsSsize_t (longobject.c)
cost: could slow down not_a_list[x]



I had been hoping to find something general to speed up all uses of
__getitem__, but that doesn't seem possible.  Oh well. :-(

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090324/81bb90de/attachment-0001.htm>

From a.badger at gmail.com  Tue Mar 24 15:59:45 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Tue, 24 Mar 2009 07:59:45 -0700
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C87004.2030807@holdenweb.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>
	<49C87004.2030807@holdenweb.com>
Message-ID: <49C8F561.1050908@gmail.com>

Steve Holden wrote:

> Seems to me that while all this is fine for developers and Python users
> it's completely unsatisfactory for people who just want to use Python
> applications. For them it's much easier if each application comes with
> all dependencies including the interpreter.
> 
> This may seem wasteful, but it removes many of the version compatibility
> issues that otherwise bog things down.
> 
The upfront cost of bundling is lower but the maintenance cost is
higher.  For instance, OS vendors have developed many ways of being
notified of and dealing with security issues.  If there's a security
issue with gtkmozdev and the python bindings to it have to be
recompiled, OS vendors will be alerted to it and have the opportunity to
release updates on zero day, the day that the security announcement goes
out.

Bundled applications suffer in two ways here:
1) the developers of the applications are unlikely to be on vendor-sec
and so the opportunity for zero day fixes is lower.

2) the developer becomes responsible for fixing problems with the
libraries, something that they often do not.  This is especially true
when developers start depending, not only on newer features of some
libraries, but older versions of others (for API changes).  It's not
clear to many developers that requiring a newer version of a library is
at least supported by upstream whereas requiring an older version leaves
them as the sole responsible party.

3) Over time, bundled libraries tend to become forked versions.  And
worse, privately forked versions.  If three python apps all use slightly
different older versions of libfoo-python and have backported fixes,
added new features, etc it is a nightmare for a system administrator or
packager to get them running with a single version from the system
library or forward port them.  And because they're private forks the
developers lose out on collaborating on security, bugfixes, etc because
they are doing their work in isolation from the other forks.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090324/0778ffa4/attachment.pgp>

From ajaksu at gmail.com  Tue Mar 24 16:04:22 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 24 Mar 2009 12:04:22 -0300
Subject: [Python-Dev] GSoC: Core Python development tools
In-Reply-To: <loom.20090323T195903-980@post.gmane.org>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com> 
	<20090322231259.GA29189@idyll.org>
	<2d75d7660903231214l2404a8d6j2f6903e948b89edc@mail.gmail.com> 
	<loom.20090323T192523-285@post.gmane.org>
	<2d75d7660903231256yccb400etfedc98a369ce6612@mail.gmail.com> 
	<loom.20090323T195903-980@post.gmane.org>
Message-ID: <2d75d7660903240804g66748e46v8754e59b2b10ebd4@mail.gmail.com>

Antoine Pitrou wrote:
> Well, it seems to me that most of these steps are separated by manual
> intervention (e.g. compile and run the test suite to check that everything works
> smoothly)

Agreed.

> so there's no real point in making a script out of them.

The idea would be to provide scripts for each step that people think
should be easier/automated. Like running the provided testcase before
and after applying the corresponding patch. Or making sure to build
before running tests for a patch that touches C code.

> The real issues with svnmerge are its occasional bugs or failures (it forgot
> some changesets when merging in the io-c branch!)

Who's handling those bugs? Is any of them avoidable/easier to
catch/easier to workaround with a script that checks for, e.g.,
svnmerge version, svnmerge correct usage and completeness of the
merge? I think this proposal could help reproducing those issues and
fixing/mitigating them until a better solution is available.

> its slowness,

Maybe something could be improved in svnmerge, as Martin suggests. Or
merging could be made a bit easier, so people would backport their
commits themselves more often and the laundry list merges could be a
little smaller.

> and its
> limitations (which are really inherent to the SVN model: e.g., if someone
> commits to the branch you have just started doing an svnmerge to, you have to
> revert everything and start over with the latest updates).

We have discussed this off-list and I have no answer regarding SVN limitations.

If a svnmerge-on-top-of-hg or svnmerge-on-top-of-bzr script can be
used to merge SVN branches with the exact same results of the real
svnmerge, only faster and prettier, maybe it should be considered for
GSoC too.

If/when the main Python repository moves to a DVCS, scripts that allow
people to perform the same steps they currently perform to achieve the
same results might help the transition.

I won't mention the bzr-on-top-of-git part :)

Cheers,
Daniel

From dickinsm at gmail.com  Tue Mar 24 16:13:53 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 24 Mar 2009 15:13:53 +0000
Subject: [Python-Dev] speeding up PyObject_GetItem
In-Reply-To: <eae285400903240742o1489f3c1veffae75e4493b073@mail.gmail.com>
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com>
	<6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001>
	<49C8A846.4020507@avl.com>
	<eae285400903240530x2ecd50f7h53212bae422a8866@mail.gmail.com>
	<eae285400903240742o1489f3c1veffae75e4493b073@mail.gmail.com>
Message-ID: <5c6f2a5d0903240813j318ea1b5m4fe6b20fedf56f99@mail.gmail.com>

2009/3/24 Daniel Stutzbach <daniel at stutzbachenterprises.com>:
> [...]
> 100 nanoseconds, py3k trunk:
> ceval -> PyObject_GetItem (object.c) -> list_subscript (listobject.c) ->
> PyNumber_AsSsize_t (object.c) -> PyLong_AsSsize_t (longobject.c)
> [more timings snipped]

Does removing the PyLong_Check call in PyLong_AsSsize_t
make any noticeable difference to these timings?

Mark

From rdmurray at bitdance.com  Tue Mar 24 16:19:56 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 24 Mar 2009 11:19:56 -0400 (EDT)
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C8F561.1050908@gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BA3154.8080408@simplistix.co.uk> <49BAA596.5020106@v.loewis.de>
	<49C79C1A.8040301@simplistix.co.uk> <49C7FC85.5000809@v.loewis.de>
	<49C80FA0.4020800@simplistix.co.uk> <87ab7bh5fb.fsf@xemacs.org>
	<49C87004.2030807@holdenweb.com> <49C8F561.1050908@gmail.com>
Message-ID: <Pine.LNX.4.64.0903241112450.31981@kimball.webabinitio.net>

On Tue, 24 Mar 2009 at 07:59, Toshio Kuratomi wrote:
> 3) Over time, bundled libraries tend to become forked versions.  And
> worse, privately forked versions.  If three python apps all use slightly
> different older versions of libfoo-python and have backported fixes,
> added new features, etc it is a nightmare for a system administrator or
> packager to get them running with a single version from the system
> library or forward port them.  And because they're private forks the
> developers lose out on collaborating on security, bugfixes, etc because
> they are doing their work in isolation from the other forks.

This is one of the things that I really disliked about Java when I
had to work with it:  the culture there is that you ship a zip bundle
that has all the libraries in it, at some unknown version or another.
I fortunately did not run into any of those security or version-drift/bug
issues in the relatively short time I worked with it, but I could feel
those issues looming in the background and it made my skin crawl :(

I'm very happy that Gentoo keeps the libraries separate when it packages
Java applications.

--
R. David Murray           http://www.bitdance.com

From cournape at gmail.com  Tue Mar 24 16:27:21 2009
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 25 Mar 2009 00:27:21 +0900
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C8F561.1050908@gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BA3154.8080408@simplistix.co.uk> <49BAA596.5020106@v.loewis.de>
	<49C79C1A.8040301@simplistix.co.uk> <49C7FC85.5000809@v.loewis.de>
	<49C80FA0.4020800@simplistix.co.uk> <87ab7bh5fb.fsf@xemacs.org>
	<49C87004.2030807@holdenweb.com> <49C8F561.1050908@gmail.com>
Message-ID: <5b8d13220903240827s4ca235e2gf3e5fae6434b3ef8@mail.gmail.com>

2009/3/24 Toshio Kuratomi <a.badger at gmail.com>:
> Steve Holden wrote:
>
>> Seems to me that while all this is fine for developers and Python users
>> it's completely unsatisfactory for people who just want to use Python
>> applications. For them it's much easier if each application comes with
>> all dependencies including the interpreter.
>>
>> This may seem wasteful, but it removes many of the version compatibility
>> issues that otherwise bog things down.
>>
> The upfront cost of bundling is lower but the maintenance cost is
> higher. ?For instance, OS vendors have developed many ways of being
> notified of and dealing with security issues. ?If there's a security
> issue with gtkmozdev and the python bindings to it have to be
> recompiled, OS vendors will be alerted to it and have the opportunity to
> release updates on zero day, the day that the security announcement goes
> out.

I don't think bundling should be compared to depending on the system
libraries, but as a lesser evil compared to requiring multiple,
system-wide installed libraries.

>
> 3) Over time, bundled libraries tend to become forked versions. ?And
> worse, privately forked versions. ?If three python apps all use slightly
> different older versions of libfoo-python and have backported fixes,
> added new features, etc it is a nightmare for a system administrator or
> packager to get them running with a single version from the system
> library or forward port them. ?And because they're private forks the
> developers lose out on collaborating on security, bugfixes, etc because
> they are doing their work in isolation from the other forks.

This is a purely technical problem, and can be handled by good source
control systems, no ?

cheers,

David

From starsareblueandfaraway at gmail.com  Tue Mar 24 16:48:14 2009
From: starsareblueandfaraway at gmail.com (Roy Hyunjin Han)
Date: Tue, 24 Mar 2009 11:48:14 -0400
Subject: [Python-Dev] About adding a new iterator method called "shuffled"
Message-ID: <6a5569ec0903240848o7c642404q3311811567d4f4d0@mail.gmail.com>

I know that Python has iterator methods called "sorted" and "reversed" and
these are handy shortcuts.


Why not add a new iterator method called "shuffled"?

>>> for x in shuffled(range(5)):
>>>    print x
>>> 3
>>> 1
>>> 2
>>> 0
>>> 4


Currently, a person has to do the following because random.shuffle() does
not return the actual shuffled list.  It is verbose.

>>> import random
>>> x = range(5)
>>> random.shuffle(x)
>>> for x in x:
>>>     print x
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090324/65b05c4e/attachment-0001.htm>

From daniel at stutzbachenterprises.com  Tue Mar 24 16:50:57 2009
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 24 Mar 2009 10:50:57 -0500
Subject: [Python-Dev] speeding up PyObject_GetItem
In-Reply-To: <5c6f2a5d0903240813j318ea1b5m4fe6b20fedf56f99@mail.gmail.com>
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com>
	<6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001>
	<49C8A846.4020507@avl.com>
	<eae285400903240530x2ecd50f7h53212bae422a8866@mail.gmail.com>
	<eae285400903240742o1489f3c1veffae75e4493b073@mail.gmail.com>
	<5c6f2a5d0903240813j318ea1b5m4fe6b20fedf56f99@mail.gmail.com>
Message-ID: <eae285400903240850n24a3ad3exf6fa5c30e00e770a@mail.gmail.com>

On Tue, Mar 24, 2009 at 10:13 AM, Mark Dickinson <dickinsm at gmail.com> wrote:

> 2009/3/24 Daniel Stutzbach <daniel at stutzbachenterprises.com>:
> > [...]
> > 100 nanoseconds, py3k trunk:
> > ceval -> PyObject_GetItem (object.c) -> list_subscript (listobject.c) ->
> > PyNumber_AsSsize_t (object.c) -> PyLong_AsSsize_t (longobject.c)
> > [more timings snipped]
>
> Does removing the PyLong_Check call in PyLong_AsSsize_t
> make any noticeable difference to these timings?
>

Making no other changes from the trunk, removing the PyLong_Check and NULL
check from PyLong_AsSsize_t shaves off 4 nanoseconds (or around 4% since the
trunk is around 100 nanoseconds).

Here's what I'm testing with, by the way:

./python.exe Lib/timeit.py -r 10 -s 'x = list(range(10))' 'x[5]'

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090324/d24871af/attachment.htm>

From aahz at pythoncraft.com  Tue Mar 24 16:58:29 2009
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 24 Mar 2009 08:58:29 -0700
Subject: [Python-Dev] About adding a new iterator method
	called	"shuffled"
In-Reply-To: <6a5569ec0903240848o7c642404q3311811567d4f4d0@mail.gmail.com>
References: <6a5569ec0903240848o7c642404q3311811567d4f4d0@mail.gmail.com>
Message-ID: <20090324155828.GA15670@panix.com>

On Tue, Mar 24, 2009, Roy Hyunjin Han wrote:
>
> I know that Python has iterator methods called "sorted" and "reversed" and
> these are handy shortcuts.
> 
> Why not add a new iterator method called "shuffled"?

Please do not post ideas like this to python-dev, please use python-ideas
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Programming language design is not a rational science. Most reasoning
about it is at best rationalization of gut feelings, and at worst plain
wrong."  --GvR, python-ideas, 2009-3-1

From a.badger at gmail.com  Tue Mar 24 17:45:21 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Tue, 24 Mar 2009 09:45:21 -0700
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <5b8d13220903240827s4ca235e2gf3e5fae6434b3ef8@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	
	<49BA3154.8080408@simplistix.co.uk> <49BAA596.5020106@v.loewis.de>	
	<49C79C1A.8040301@simplistix.co.uk> <49C7FC85.5000809@v.loewis.de>	
	<49C80FA0.4020800@simplistix.co.uk>
	<87ab7bh5fb.fsf@xemacs.org>	 <49C87004.2030807@holdenweb.com>
	<49C8F561.1050908@gmail.com>
	<5b8d13220903240827s4ca235e2gf3e5fae6434b3ef8@mail.gmail.com>
Message-ID: <49C90E21.6070407@gmail.com>

David Cournapeau wrote:
> 2009/3/24 Toshio Kuratomi <a.badger at gmail.com>:
>> Steve Holden wrote:
>>
>>> Seems to me that while all this is fine for developers and Python users
>>> it's completely unsatisfactory for people who just want to use Python
>>> applications. For them it's much easier if each application comes with
>>> all dependencies including the interpreter.
>>>
>>> This may seem wasteful, but it removes many of the version compatibility
>>> issues that otherwise bog things down.
>>>
>> The upfront cost of bundling is lower but the maintenance cost is
>> higher.  For instance, OS vendors have developed many ways of being
>> notified of and dealing with security issues.  If there's a security
>> issue with gtkmozdev and the python bindings to it have to be
>> recompiled, OS vendors will be alerted to it and have the opportunity to
>> release updates on zero day, the day that the security announcement goes
>> out.
> 
> I don't think bundling should be compared to depending on the system
> libraries, but as a lesser evil compared to requiring multiple,
> system-wide installed libraries.
> 
Well.. I'm not so sure it's even a win there.  If the libraries are
installed system-wide, at least the consumer of the application knows:

1) Where to find all the libraries to audit the versions when a security
issue is announced.
2) That the library is unforked from upstream.
3) That all the consumers of the library version have a central location
to collaborate on announcing fixes to the library.

With my distribution packager hat on, I can say I dislike both multiple
versions and bundling but I definitely dislike bundling more.

>> 3) Over time, bundled libraries tend to become forked versions.  And
>> worse, privately forked versions.  If three python apps all use slightly
>> different older versions of libfoo-python and have backported fixes,
>> added new features, etc it is a nightmare for a system administrator or
>> packager to get them running with a single version from the system
>> library or forward port them.  And because they're private forks the
>> developers lose out on collaborating on security, bugfixes, etc because
>> they are doing their work in isolation from the other forks.
> 
> This is a purely technical problem, and can be handled by good source
> control systems, no ?
> 
No.  This is a social problem.  Good source control only helps if I am
tracking upstream's trunk so I'm aware of the direction that their
changes are headed.  But there's a wide range of reasons that
application developers that bundle libraries don't do that:

1) not enough time in a day.  I'm working full-time on making my
application better.  Plus I have to update all these bundled libraries
from time to time, testing that the updates don't break anything.  I
don't have time to track trunk for all these libraries -- I barely have
time to track releases.

2) My release schedule doesn't mesh with all of the upstream libraries
I'm bundling.  When I want to release Foo-1.0, I want to have some
assurance that the libraries I'm bundling with will do the right thing.
   Since releases see more testing than trunk, tracking trunk for twenty
bundled libraries is a lot less attractive than tracking release branches.

3) This doesn't help with the fact that my bundled version of the
library and your bundled version of the library are being developed in
isolation from each other.  This needs central coordination which people
who believe bundling libraries are very unlikely to pursue.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090324/a68c4032/attachment.pgp>

From tseaver at palladion.com  Tue Mar 24 18:10:06 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 24 Mar 2009 13:10:06 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <Pine.LNX.4.64.0903240831390.31981@kimball.webabinitio.net>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>
	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>
	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>
	<87ab7bh5fb.fsf@xemacs.org>	<49C87004.2030807@holdenweb.com>
	<49C88503.2030902@v.loewis.de>	<49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com>
	<Pine.LNX.4.64.0903240831390.31981@kimball.webabinitio.net>
Message-ID: <gqb45p$o4j$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

R. David Murray wrote:
> On Tue, 24 Mar 2009 at 07:53, Steve Holden wrote:
>> I don't claim to have all the answers, but distutils and setuptools are
>> for the savvy users - people who build distros and those who maintain
>> their own systems independent of their distribution's packaging
>> standard. As time goes by, however, and the Lunix installed base
>> continues to grow, the majority of users will expect to rely on the
>> standard installation mechanisms of their distribution, and that will
>> never be setuptools or distutils.

I'll note that such users are mostly going to be installing desktop
applications, which make a good case for the "batteries included" story,
but one which ignores the "system" Python altogether, and bundles the
interpreter.  I think that building such a self-contained bundle would
be eased by a strong story for managing out-of-core distributions.

> +1.
> 
> Or any other Python-only package manager.
> 
> And that goes at least double for Windows :)

I just did an install for a customer yesterday (actually, the customer
did it, with me as "air traffic controller"), which started from the
current Plone Windows installer, and then used the zc.buildout machinery
shipped with Plone to add on the extra distributions (seven or so
"community-maintained", plus one containing custom code).  The buildout
was driven from a text-based config file I supplied to the customer,
which I developed and tested first on Linux.

This method was *way* less painful and error-prone than downloading and
running eight separate installers, *and* it ensured repeatability,
because the extra distributions were all installed from a PyPI-style
index I set up for the customer:  I can recreate the environment at will
(well, all the bits on disk under the main installation directory).

If I need to fix one of the distributions which make up the application,
ar add a new one, I will roll a new 'sdist' for it (no C extensions in
play), upload it to the index, rebuild the metadata, and tell the
customer to re-run the 'buildout.exe'.

> I am (among other things) a software developer, and I am quite capable of
> maintaining my own python installation outside the system one, and do so
> for testing and development.  But for anything else, I rely on the system
> package manager to install my Python packages.  If the extension hasn't
> been packaged for my distribution, I typically will either not use it,
> or I will install it somewhere other than the system default location.

At which point you can benefit from the distributions-with-dependencies-
in-an-index format provided, with varying levels of success, by
setuptools today.

I think the confusion-in-terms here may be that many Python developers
don't use a large number of out-of-core distributions, and thus are
mostly OK with managing the few they do use as part of their
application:  the rest they get from a "batteries include" Python.
Mostly that is a reasonable choice, although I note that the Python 2.5
releases before 2.5.2 broke applicatinos using the tarfile module, and
the 2.6 release broke applications using asyncore / asynchat:  if those
packages were shipped outside the stdlib, I could have fixed the
breakage by updating them separately.

Those of us who do need to deploy applications with larger dependency
sets need tools which make it possible to do the deployments repeatably.
 The "use .deb / .rpm / Windows installer" answer just moves the effort
around (and adds to it, in my experience), because in that world I end
up running the equivalent of a PyPI index anyway:  I can't rely on an
upstream distro to package the correct versions for me, or to avoid
breaking my application by the packaging choices they make.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJyRPu+gerLs4ltQ4RAjgPAKDB12YNxmAJX7VFN9WGvMR3H8KPHgCgt0NN
vCKUCTkVcxjP09/LQVBP8G4=
=2qbi
-----END PGP SIGNATURE-----


From songofacandy at gmail.com  Tue Mar 24 18:16:04 2009
From: songofacandy at gmail.com (INADA Naoki)
Date: Wed, 25 Mar 2009 02:16:04 +0900
Subject: [Python-Dev] GSoC: Replace MS Windows Console with Unicode UI
In-Reply-To: <49C7FF1F.6080707@g.nevcal.com>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>
	<gq70q2$hul$1@ger.gmane.org> <49C70C6A.7070106@g.nevcal.com>
	<gq8mu2$mfl$1@ger.gmane.org> <49C7FF1F.6080707@g.nevcal.com>
Message-ID: <ee9557410903241016h2f1e7b1cp4afdf96afd7f293f@mail.gmail.com>

Hi. I'm Japanese and non-ascii charactor user. (cp932)

We have to use "IME" to input non-ascii charactor in Windows.
When "> chcp 65001" in cmd.exe, we cannot use IME on cmd.exe.

So setting codepage to 65001 make output universal but make input ascii-only.
Sit!!!

I hope PyQtShell <http://code.google.com/p/pyqtshell/> become good
IDLE alternative.

From tseaver at palladion.com  Tue Mar 24 18:20:07 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 24 Mar 2009 13:20:07 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <5b8d13220903240700l72ab6e5fk479bd2fa1cdfd958@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BAA596.5020106@v.loewis.de>
	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>
	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>
	<49C87004.2030807@holdenweb.com>	<49C88503.2030902@v.loewis.de>
	<49C886EF.80203@v.loewis.de>	<49C8C9B3.3070403@holdenweb.com>
	<5b8d13220903240700l72ab6e5fk479bd2fa1cdfd958@mail.gmail.com>
Message-ID: <gqb4oi$qb9$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Cournapeau wrote:

> There already is a bit of a DLL hell in python. The whole idea to
> solve the dependency problems by installing multiple version of the
> same software is fundamentally flawed, it just does not work for
> general deployment on multiple machines. Many systems outside python,
> with more resource, have tried - and failed. By enabling a general,
> system-wide installation of multiple version of the same package,
> setuptools has made the situation worse. I am quite puzzled than many
> people don't realize this fundamental issue, it is a simple
> combinatory problem.

Many of us using setuptools extensively tend to adopt an "isolated
environment" strategy (e.g., pip, virtualenv, zc.buildout).  We don't
install the packages used by different applications into shared
directories at all.  Instead, each environment uses a restricted subset
of packages known to work together.

> If the problem is to get a recent enough version of the library, then
> the library would better be installed "locally", for the application.
> If it is too much a problem because the application depends on
> billions of libraries which are 6 months old, the problem is to allow
> such a dependency in the first place. What kind of nightmare would it
> be if programs developed in C would required a C library which is 6
> months old ? That's exactly what multiple-versions installations
> inflict on us. That's great for testing, development. But for
> deployment on end-user machines, the whole thing is a failure IMO.

It is wildly successful, even on platforms such as Windows, when you
abandon the notion that separate applications should be sharing the
libaries they need.

>> I am afraid that distutils, and
>> setuptools, are not really the answer to the problem, since while they
>> may (as intended) guarantee that Python applications can be installed
>> uniformly across different platforms they also more or less guarantee
>> that Python applications are installed differently from all other
>> applications on the platform.
> 
> I think they should be part of the solution, in the sense that they
> should allow easier packaging for the different platforms (linux,
> windows, mac os x and so on). For now, they make things much harder
> than they should (difficult to follow the FHS, etc...).

FHS is something which packagers / distributors care about:  I strongly
doubt that the "end users" will ever notice, particularly for silliness
like 'bin' vs. 'sbin', or architecture-specific vs. 'noarch' rules.

> But otherwise,
> I agree. Python applications which care about non-savy users should be
> distributed as .dmg, .exe, .rpm, .deb. There will always be some work
> to do that correctly: there is no way to provide a general, automatic
> environment to build installers which provide a good experience on all
> platforms. AFAIK, It does not even exist in the commercial landscape,
> so I see little chance to see this in the python ecosystem.

As a counter-example, I offer the current Plone installer[1], which lays
down a bunch of egg-based packages in a cross-platform way (Windows,
MacOSX, Linux, BSDs).  It uses zc.buildout, which makes
configuration-driven (repeatable) installation of add-ons easy.

[1] http://plone.org/products/plone


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJyRZH+gerLs4ltQ4RAu3qAKDXhkY73S2htl6Ytz+ZPVyRgYSHLACdGo+N
XNMrpg7ue0SX3ODgiOjHTmg=
=Rk1D
-----END PGP SIGNATURE-----


From cournape at gmail.com  Tue Mar 24 18:23:19 2009
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 25 Mar 2009 02:23:19 +0900
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C90E21.6070407@gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BAA596.5020106@v.loewis.de> <49C79C1A.8040301@simplistix.co.uk>
	<49C7FC85.5000809@v.loewis.de> <49C80FA0.4020800@simplistix.co.uk>
	<87ab7bh5fb.fsf@xemacs.org> <49C87004.2030807@holdenweb.com>
	<49C8F561.1050908@gmail.com>
	<5b8d13220903240827s4ca235e2gf3e5fae6434b3ef8@mail.gmail.com>
	<49C90E21.6070407@gmail.com>
Message-ID: <5b8d13220903241023n6d0744d5l37d38bb734bfd331@mail.gmail.com>

On Wed, Mar 25, 2009 at 1:45 AM, Toshio Kuratomi <a.badger at gmail.com> wrote:
> David Cournapeau wrote:
>> 2009/3/24 Toshio Kuratomi <a.badger at gmail.com>:
>>> Steve Holden wrote:
>>>
>>>> Seems to me that while all this is fine for developers and Python users
>>>> it's completely unsatisfactory for people who just want to use Python
>>>> applications. For them it's much easier if each application comes with
>>>> all dependencies including the interpreter.
>>>>
>>>> This may seem wasteful, but it removes many of the version compatibility
>>>> issues that otherwise bog things down.
>>>>
>>> The upfront cost of bundling is lower but the maintenance cost is
>>> higher. ?For instance, OS vendors have developed many ways of being
>>> notified of and dealing with security issues. ?If there's a security
>>> issue with gtkmozdev and the python bindings to it have to be
>>> recompiled, OS vendors will be alerted to it and have the opportunity to
>>> release updates on zero day, the day that the security announcement goes
>>> out.
>>
>> I don't think bundling should be compared to depending on the system
>> libraries, but as a lesser evil compared to requiring multiple,
>> system-wide installed libraries.
>>
> Well.. I'm not so sure it's even a win there. ?If the libraries are
> installed system-wide, at least the consumer of the application knows:
>
> 1) Where to find all the libraries to audit the versions when a security
> issue is announced.
> 2) That the library is unforked from upstream.
> 3) That all the consumers of the library version have a central location
> to collaborate on announcing fixes to the library.

Yes, those are problems, but installing multi libraries have a lot of
problems too:
 - quickly, by enabling multiple version installed, people become very
sloppy to handle versions of the dependencies, and this increases a
lot the number of libraries installed - so the advantages above for
system-wide installation  becomes intractable quite quickly
 - bundling also supports a real user-case which cannot be solved by
rpm/deb AFAIK: installation without administration privileges.
 - multi-version installation give very fragile systems. That's
actually my number one complain in python: setuptools has caused me
numerous headache, and I got many bug reports because you often do not
know why one version was loaded instead of another one.

So I am not so convinced multiple-version is better than bundling - I
can see how it sometimes can be, but I am not sure those are that
important in practice.

> No. ?This is a social problem. ?Good source control only helps if I am
> tracking upstream's trunk so I'm aware of the direction that their
> changes are headed. ?But there's a wide range of reasons that
> application developers that bundle libraries don't do that:
>
> 1) not enough time in a day. ?I'm working full-time on making my
> application better. ?Plus I have to update all these bundled libraries
> from time to time, testing that the updates don't break anything. ?I
> don't have time to track trunk for all these libraries -- I barely have
> time to track releases.

Yes, but in that case, there is nothing you can do. Putting everything
in one project is always easier than splitting into modules, coding
and deployment-wise. That's just one side of the speed of development
vs maintenance issue IMHO.

> 3) This doesn't help with the fact that my bundled version of the
> library and your bundled version of the library are being developed in
> isolation from each other. ?This needs central coordination which people
> who believe bundling libraries are very unlikely to pursue.

As above, I think that in that case, it will happen whatever tools are
available, so it is not a case worth being pursued.

cheers,

David

From cournape at gmail.com  Tue Mar 24 18:41:28 2009
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 25 Mar 2009 02:41:28 +0900
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <gqb4oi$qb9$1@ger.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C7FC85.5000809@v.loewis.de> <49C80FA0.4020800@simplistix.co.uk>
	<87ab7bh5fb.fsf@xemacs.org> <49C87004.2030807@holdenweb.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com>
	<5b8d13220903240700l72ab6e5fk479bd2fa1cdfd958@mail.gmail.com>
	<gqb4oi$qb9$1@ger.gmane.org>
Message-ID: <5b8d13220903241041o3355b8bfk61f66fc9ea1bc68c@mail.gmail.com>

On Wed, Mar 25, 2009 at 2:20 AM, Tres Seaver <tseaver at palladion.com> wrote:

>
> Many of us using setuptools extensively tend to adopt an "isolated
> environment" strategy (e.g., pip, virtualenv, zc.buildout). ?We don't
> install the packages used by different applications into shared
> directories at all. ?Instead, each environment uses a restricted subset
> of packages known to work together.

Is that a working solution when you want to enable easy installation
on a large number of "customers" ? In those discussions, I often see
different solutions depending on the kind of projects people do. I
don't know anything about plone, but I can imagine the deployment
issues are quite different from the projects I am involved in (numpy
and co).

Everytime I tried to understand what buildout was about, I was not
even sure it could help for my own problems at all. It seems very
specific to web development - I may completely miss the point ?
virtualenv, pip, yolk, those are useful tools for development/testing,
but I don't see how they can help me to make the installation of a
numpy environment easier on many different kind of platforms.

>
>> If the problem is to get a recent enough version of the library, then
>> the library would better be installed "locally", for the application.
>> If it is too much a problem because the application depends on
>> billions of libraries which are 6 months old, the problem is to allow
>> such a dependency in the first place. What kind of nightmare would it
>> be if programs developed in C would required a C library which is 6
>> months old ? That's exactly what multiple-versions installations
>> inflict on us. That's great for testing, development. But for
>> deployment on end-user machines, the whole thing is a failure IMO.
>
> It is wildly successful, even on platforms such as Windows, when you
> abandon the notion that separate applications should be sharing the
> libaries they need.

Well, I may not have been clear: I meant that in my experience,
deploying something with several dependencies was easier with bundling
than with a mechanism ala setuptools with *system-wide* installation
of multiple versions of the same library. So I think we agree here:
depending on something stable (python stdlib + a few well known
things) system-wide is OK, for anything else, not sharing is easier
and more robust in the current state of things, at least when one
needs to stay cross platform.

Almost every deployment problem I got from people using my own
softwares was related to setuptools, and in particular the multiple
version thing. For end-users who know nothing about python package
mechanism, and do not care about it, that's really a PITA to debug,
and give bad mouth taste. The fact that those problems happen when my
software was not even *using* setuptools/etc... was a real deal
breaker for me, and I am strongly biased against setuptools ever
since.

>
> FHS is something which packagers / distributors care about: ?I strongly
> doubt that the "end users" will ever notice, particularly for silliness
> like 'bin' vs. 'sbin', or architecture-specific vs. 'noarch' rules.

That's not silly, and that's a bit of a fallacy. Of course end users
do not care about the FHS in itself, but following the FHS enables
good integration in the system, which end users do care about. I like
finding my doc in /usr/share/doc whatever thing I install - as I am
sure every window user like to find his installed software in the
panel control.

>
> As a counter-example, I offer the current Plone installer[1], which lays
> down a bunch of egg-based packages in a cross-platform way (Windows,
> MacOSX, Linux, BSDs). ?It uses zc.buildout, which makes
> configuration-driven (repeatable) installation of add-ons easy.

But zc.buildout is not a solution to deploy applications, right ? In
my understanding, it is a tool to deploy plone instances on
server/test machines, but that's quite a different problem from
installing "applications" for users who may not even know what python
is.

cheers,

David

From dickinsm at gmail.com  Tue Mar 24 18:45:38 2009
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 24 Mar 2009 17:45:38 +0000
Subject: [Python-Dev] speeding up PyObject_GetItem
In-Reply-To: <eae285400903240850n24a3ad3exf6fa5c30e00e770a@mail.gmail.com>
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com>
	<6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001>
	<49C8A846.4020507@avl.com>
	<eae285400903240530x2ecd50f7h53212bae422a8866@mail.gmail.com>
	<eae285400903240742o1489f3c1veffae75e4493b073@mail.gmail.com>
	<5c6f2a5d0903240813j318ea1b5m4fe6b20fedf56f99@mail.gmail.com>
	<eae285400903240850n24a3ad3exf6fa5c30e00e770a@mail.gmail.com>
Message-ID: <5c6f2a5d0903241045w69282138v27e0224d1563bbc@mail.gmail.com>

On Tue, Mar 24, 2009 at 3:50 PM, Daniel Stutzbach
<daniel at stutzbachenterprises.com> wrote:
> On Tue, Mar 24, 2009 at 10:13 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
>> Does removing the PyLong_Check call in PyLong_AsSsize_t
>> make any noticeable difference to these timings?
>
> Making no other changes from the trunk, removing the PyLong_Check and NULL
> check from PyLong_AsSsize_t shaves off 4 nanoseconds (or around 4% since the
> trunk is around 100 nanoseconds).

Thanks.  I'd call that a noticeable difference.  I'd be +1 on changing
this particular check to an assert and so disabling it in non-debug builds.
I'd like to bet that the majority of calls to PyLong_AsSsize_t are
internal.

Mark

From db3l.net at gmail.com  Tue Mar 24 19:01:30 2009
From: db3l.net at gmail.com (David Bolen)
Date: Tue, 24 Mar 2009 14:01:30 -0400
Subject: [Python-Dev] Test failures under Windows?
References: <loom.20090324T134451-902@post.gmane.org>
Message-ID: <m2zlfa241x.fsf@valheru.db3l.homeip.net>

Antoine Pitrou <solipsis at pitrou.net> writes:

> Am I the only one getting those?
> By the way, what happened to the Windows buildbots?

Oops - sorry, mine (XP-4) apparently got stuck with C++ debug
assertion dialogs (an assertion from the internal close.c module) from
python_d during some earlier build.  I try to catch these but have been
a bit preoccupied this month.  Should be back now.

-- David


From solipsis at pitrou.net  Tue Mar 24 19:13:14 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 24 Mar 2009 18:13:14 +0000 (UTC)
Subject: [Python-Dev] =?utf-8?q?speeding_up_PyObject=5FGetItem?=
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com>
	<6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001>
	<49C8A846.4020507@avl.com>
	<eae285400903240530x2ecd50f7h53212bae422a8866@mail.gmail.com>
	<eae285400903240742o1489f3c1veffae75e4493b073@mail.gmail.com>
	<5c6f2a5d0903240813j318ea1b5m4fe6b20fedf56f99@mail.gmail.com>
	<eae285400903240850n24a3ad3exf6fa5c30e00e770a@mail.gmail.com>
	<5c6f2a5d0903241045w69282138v27e0224d1563bbc@mail.gmail.com>
Message-ID: <loom.20090324T181207-923@post.gmane.org>

Mark Dickinson <dickinsm <at> gmail.com> writes:
> 
> > Making no other changes from the trunk, removing the PyLong_Check and NULL
> > check from PyLong_AsSsize_t shaves off 4 nanoseconds (or around 4% since the
> > trunk is around 100 nanoseconds).
> 
> Thanks.  I'd call that a noticeable difference.

4% on a micro-micro-benchmark is hardly compelling...

cheers

Antoine.



From tseaver at palladion.com  Tue Mar 24 19:15:28 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 24 Mar 2009 14:15:28 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <5b8d13220903241041o3355b8bfk61f66fc9ea1bc68c@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49C7FC85.5000809@v.loewis.de>
	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>
	<49C87004.2030807@holdenweb.com>	<49C88503.2030902@v.loewis.de>
	<49C886EF.80203@v.loewis.de>	<49C8C9B3.3070403@holdenweb.com>	<5b8d13220903240700l72ab6e5fk479bd2fa1cdfd958@mail.gmail.com>	<gqb4oi$qb9$1@ger.gmane.org>
	<5b8d13220903241041o3355b8bfk61f66fc9ea1bc68c@mail.gmail.com>
Message-ID: <gqb80a$6gb$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Cournapeau wrote:
> On Wed, Mar 25, 2009 at 2:20 AM, Tres Seaver <tseaver at palladion.com> wrote:
> 
>> Many of us using setuptools extensively tend to adopt an "isolated
>> environment" strategy (e.g., pip, virtualenv, zc.buildout).  We don't
>> install the packages used by different applications into shared
>> directories at all.  Instead, each environment uses a restricted subset
>> of packages known to work together.
> 
> Is that a working solution when you want to enable easy installation
> on a large number of "customers" ? In those discussions, I often see
> different solutions depending on the kind of projects people do. I
> don't know anything about plone, but I can imagine the deployment
> issues are quite different from the projects I am involved in (numpy
> and co).

Plone is downloaded and installed on many-many systems, across all the
"mainline" platforms.  In each case (since Plone 3.2), the installer is
based on (and includes) zc.buildout, and documents[1] how to add new
bits to the installed Plone by modifying the buildout.cfg file.

> Everytime I tried to understand what buildout was about, I was not
> even sure it could help for my own problems at all. It seems very
> specific to web development - I may completely miss the point ?

I think so:  it is largely a way to get repeatable / scripted deployment
of software to disk.  It uses setuptools to install Python package
distributions, but also can use other means (e.g, configure-make-make
install to install a C library such as libxml2).  The end result is a
self-contained directory tree:

- - Scripts in the 'bin' directory are configured to have the specific
  Python pacakges (and versions) they require on the PYTHONPATH.

- - By convention, released package distributions are installed into the
  'eggs' subdirectory', which is *not* on the PYTHONPATH, nor is it a
  'site' directory for Python.

- - Other bits are typically in their own subdirectories, often under
  'parts'.

> virtualenv, pip, yolk, those are useful tools for development/testing,
> but I don't see how they can help me to make the installation of a
> numpy environment easier on many different kind of platforms.

When not doing Plone / Zope-specific work (where zc.buildout is a de
facto standard), I use 'virtualenv' to create isolated environments into
which I install the libraries for a given application.  If your
application ships as Python package distributions, then documenting the
use of 'virtualenv' as a "supported" way to install it might reduce your
support burden.  You can even ship a virtualenv-derived script which
pre-installs your own packages into such an environment, isolated from
the other pacakges installed on the machine.

>>> If the problem is to get a recent enough version of the library, then
>>> the library would better be installed "locally", for the application.
>>> If it is too much a problem because the application depends on
>>> billions of libraries which are 6 months old, the problem is to allow
>>> such a dependency in the first place. What kind of nightmare would it
>>> be if programs developed in C would required a C library which is 6
>>> months old ? That's exactly what multiple-versions installations
>>> inflict on us. That's great for testing, development. But for
>>> deployment on end-user machines, the whole thing is a failure IMO.
>> It is wildly successful, even on platforms such as Windows, when you
>> abandon the notion that separate applications should be sharing the
>> libaries they need.
> 
> Well, I may not have been clear: I meant that in my experience,
> deploying something with several dependencies was easier with bundling
> than with a mechanism ala setuptools with *system-wide* installation
> of multiple versions of the same library. So I think we agree here:
> depending on something stable (python stdlib + a few well known
> things) system-wide is OK, for anything else, not sharing is easier
> and more robust in the current state of things, at least when one
> needs to stay cross platform.

You can think of zc.buildout or the virtualenv-based script as a form of
bundling, which bootstraps from another already-installed Python, but
remains isolated from anything in its 'site-packages'.

> Almost every deployment problem I got from people using my own
> softwares was related to setuptools, and in particular the multiple
> version thing.

I never even use that switch manually.  zc.buildout does, but that is
because it wants to control the PYTHONPATH by generating the script
code:  it doesn't ask users to tweak that.

> For end-users who know nothing about python package
> mechanism, and do not care about it, that's really a PITA to debug,
> and give bad mouth taste. The fact that those problems happen when my
> software was not even *using* setuptools/etc... was a real deal
> breaker for me, and I am strongly biased against setuptools ever
> since.

I don't know why anybody who was not writing a packaging tool, or
packaging a library for something like .deb / .rpm, would even use the
multi-version option for setuptools:  I don't see any sane way to
install conflicting requirements into a shared 'site-packages'.

>> FHS is something which packagers / distributors care about:  I strongly
>> doubt that the "end users" will ever notice, particularly for silliness
>> like 'bin' vs. 'sbin', or architecture-specific vs. 'noarch' rules.
> 
> That's not silly, and that's a bit of a fallacy. Of course end users
> do not care about the FHS in itself, but following the FHS enables
> good integration in the system, which end users do care about. I like
> finding my doc in /usr/share/doc whatever thing I install - as I am
> sure every window user like to find his installed software in the
> panel control.


>> As a counter-example, I offer the current Plone installer[1], which lays
>> down a bunch of egg-based packages in a cross-platform way (Windows,
>> MacOSX, Linux, BSDs).  It uses zc.buildout, which makes
>> configuration-driven (repeatable) installation of add-ons easy.
> 
> But zc.buildout is not a solution to deploy applications, right ? In
> my understanding, it is a tool to deploy plone instances on
> server/test machines, but that's quite a different problem from
> installing "applications" for users who may not even know what python

In this case, Plone installs as an "application," in the sense that I
think you mean.  It happens to be one which can be extended after
installation via a set of add-ons, which are now mostly expected to be
installed *into* the Plone application environment using zc.buildout.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJySNA+gerLs4ltQ4RAuX8AJ4pzev40jq9aQcCFM6P3a5+lUyungCghw1p
vSPEudj3quo+mQkiv+QhxCo=
=qjze
-----END PGP SIGNATURE-----


From kristjan at ccpgames.com  Tue Mar 24 19:19:32 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Tue, 24 Mar 2009 18:19:32 +0000
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <m2zlfa241x.fsf@valheru.db3l.homeip.net>
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>

These issues should be resolved in the py3k branch, but it will need porting to 2.6.
Dialogue boxes are annoying, but do they pop up if you run your buildslave as a service without access to the console?
K

-----Original Message-----
From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of David Bolen
Sent: 24. mars 2009 18:02
To: python-dev at python.org
Subject: Re: [Python-Dev] Test failures under Windows?

Antoine Pitrou <solipsis at pitrou.net> writes:

> Am I the only one getting those?
> By the way, what happened to the Windows buildbots?

Oops - sorry, mine (XP-4) apparently got stuck with C++ debug
assertion dialogs (an assertion from the internal close.c module) from
python_d during some earlier build.  I try to catch these but have been
a bit preoccupied this month.  Should be back now.

-- David

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com


From solipsis at pitrou.net  Tue Mar 24 19:15:43 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 24 Mar 2009 18:15:43 +0000 (UTC)
Subject: [Python-Dev] Test failures under Windows?
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
Message-ID: <loom.20090324T181413-776@post.gmane.org>

David Bolen <db3l.net <at> gmail.com> writes:
> 
> Antoine Pitrou <solipsis <at> pitrou.net> writes:
> 
> > Am I the only one getting those?
> > By the way, what happened to the Windows buildbots?
> 
> Oops - sorry, mine (XP-4) apparently got stuck with C++ debug
> assertion dialogs (an assertion from the internal close.c module) from
> python_d during some earlier build.

:( This is unfortunately the kind of problem I wish the Windows buildbots had
been able to report unambiguously...

Regards

Antoine.



From a.badger at gmail.com  Tue Mar 24 19:41:09 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Tue, 24 Mar 2009 11:41:09 -0700
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <gqb4oi$qb9$1@ger.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>	<49C87004.2030807@holdenweb.com>	<49C88503.2030902@v.loewis.de>	<49C886EF.80203@v.loewis.de>	<49C8C9B3.3070403@holdenweb.com>	<5b8d13220903240700l72ab6e5fk479bd2fa1cdfd958@mail.gmail.com>
	<gqb4oi$qb9$1@ger.gmane.org>
Message-ID: <49C92945.4010502@gmail.com>

Tres Seaver wrote:
> David Cournapeau wrote:
>>> I am afraid that distutils, and
>>> setuptools, are not really the answer to the problem, since while they
>>> may (as intended) guarantee that Python applications can be installed
>>> uniformly across different platforms they also more or less guarantee
>>> that Python applications are installed differently from all other
>>> applications on the platform.
>> I think they should be part of the solution, in the sense that they
>> should allow easier packaging for the different platforms (linux,
>> windows, mac os x and so on). For now, they make things much harder
>> than they should (difficult to follow the FHS, etc...).
> 
> FHS is something which packagers / distributors care about:  I strongly
> doubt that the "end users" will ever notice, particularly for silliness
> like 'bin' vs. 'sbin', or architecture-specific vs. 'noarch' rules.
> 
That's because you're thinking of a different class of end-user than FHS
 is targeting.  Someone who wants to install a web application on a
limited number of machines (one in the home-desktop scenario) or someone
who makes their living helping people to install the software they've
written has a whole different view on things than someone who's trying
to install and maintain the software on fifteen computer labs in a
campus or the person who is trying to write software that is portable to
tens of different platforms in their spare time and every bit of
answering end user's questions, tracking other upstreams for security
bugs, etc, is time taken away from coding.

Following FHS means that the software will work for both "end-users" who
don't care about the nitty-gritty of the FHS and system administrators
of large sites.  Disregarding the FHS because it is "silliness" means
that system administrators are going to have to special-case your
application, decide not to install it at all, or pay someone else to
support it.

Note that those things do make sense sometimes.  For instance, when an
application is not intended to be distributed to a large number of
outside entities (facebook, flikr, etc) or when your revenue stream is
making money from installing and administering a piece of software for
other companies.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090324/1d8e8892/attachment-0001.pgp>

From db3l.net at gmail.com  Tue Mar 24 19:49:27 2009
From: db3l.net at gmail.com (David Bolen)
Date: Tue, 24 Mar 2009 14:49:27 -0400
Subject: [Python-Dev] Test failures under Windows?
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
Message-ID: <m2vdpy21u0.fsf@valheru.db3l.homeip.net>

Kristj?n Valur J?nsson <kristjan at ccpgames.com> writes:

> These issues should be resolved in the py3k branch, but it will need
> porting to 2.6.  Dialogue boxes are annoying, but do they pop up if
> you run your buildslave as a service without access to the console?

Not sure what the MSVC++ runtime does with assertion dialogs if run
without interactive access (they aren't console messages, but UI dialog
popups).

The buildslave already disables all normal MS runtime error dialogs,
but can't stop the internal C runtime from putting up its own, at
least I don't think so.  It can be disabled from within Python itself
at startup, but not external to the process.

There was a discussion about this on the py3k mailing list back in
mid-2007 ("buildbots" thread) and perhaps later as well, at which
point I believe Martin added an "-n" option to regrtest and the
buildbot test.bat file to disable the assertions.  Is that the py3k
branch piece you are referring to?

Although in looking now I can't find that option in use in any of the
current buildbot branches as part of tools/buildbot/test.bat.  Looks
like it got lost in one of the merges (unless it was intentional)?  I
wasn't paying close attention, so maybe there's some other mechanism
in place at this point?

-- David


From cournape at gmail.com  Tue Mar 24 19:53:58 2009
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 25 Mar 2009 03:53:58 +0900
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <gqb80a$6gb$1@ger.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<87ab7bh5fb.fsf@xemacs.org> <49C87004.2030807@holdenweb.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com>
	<5b8d13220903240700l72ab6e5fk479bd2fa1cdfd958@mail.gmail.com>
	<gqb4oi$qb9$1@ger.gmane.org>
	<5b8d13220903241041o3355b8bfk61f66fc9ea1bc68c@mail.gmail.com>
	<gqb80a$6gb$1@ger.gmane.org>
Message-ID: <5b8d13220903241153i6f2276aq24710e629139becc@mail.gmail.com>

On Wed, Mar 25, 2009 at 3:15 AM, Tres Seaver <tseaver at palladion.com> wrote:

>
>> Everytime I tried to understand what buildout was about, I was not
>> even sure it could help for my own problems at all. It seems very
>> specific to web development - I may completely miss the point ?
>
> I think so: ?it is largely a way to get repeatable / scripted deployment
> of software to disk. ?It uses setuptools to install Python package
> distributions, but also can use other means (e.g, configure-make-make
> install to install a C library such as libxml2). ?The end result is a
> self-contained directory tree:
>
> - - Scripts in the 'bin' directory are configured to have the specific
> ?Python pacakges (and versions) they require on the PYTHONPATH.
>
> - - By convention, released package distributions are installed into the
> ?'eggs' subdirectory', which is *not* on the PYTHONPATH, nor is it a
> ?'site' directory for Python.
>
> - - Other bits are typically in their own subdirectories, often under
> ?'parts'.

Ok - but I don't think it helps much, see below.


> When not doing Plone / Zope-specific work (where zc.buildout is a de
> facto standard), I use 'virtualenv' to create isolated environments into
> which I install the libraries for a given application. ?If your
> application ships as Python package distributions, then documenting the
> use of 'virtualenv' as a "supported" way to install it might reduce your
> support burden.

I now realize why we don't understand each other - in my case, the one
doing the installation is the user, who cannot be assumed to know much
about python.q11 That's what I mean by "application deployment vs
webapp deployment". Ideally, the user has to enter one command/click
one button, and he can use the application - he may not even be a
programmer (I deploy things based on numpy/scipy for scientific
people, who often have little patience for things which take more than
1 minute to set up software-wise).

That's the user case I am mostly interested in - and I think it is
quite general, and quite different from plone deployment kind of
things.

>
> You can think of zc.buildout or the virtualenv-based script as a form of
> bundling, which bootstraps from another already-installed Python, but
> remains isolated from anything in its 'site-packages'.

Yes, I know what virtualenv is, I use it sometimes - but it is
definitely too complicated for the people I want to distribute my
software to.

>
> I never even use that switch manually. ?zc.buildout does, but that is
> because it wants to control the PYTHONPATH by generating the script
> code: ?it doesn't ask users to tweak that.

Well, that's the thing: neither do I :) but if my software is a
dependency of another software, then I am bothered for problems with
software which are not used at all by my own package (because
setuptools makes an egg of my software automatically, screw things up,
and I am the one blamed for it).

>
> I don't know why anybody who was not writing a packaging tool, or
> packaging a library for something like .deb / .rpm, would even use the
> multi-version option for setuptools: ?I don't see any sane way to
> install conflicting requirements into a shared 'site-packages'.

But that's the problem: it often happens *even if you don't use
setuptools yourself*. I would not be surprised if that's one reason
why so many people have a seemingly unfair bias against setuptools.
That's certainly the reason in my case.

cheers,

David

From collinw at gmail.com  Tue Mar 24 20:05:23 2009
From: collinw at gmail.com (Collin Winter)
Date: Tue, 24 Mar 2009 14:05:23 -0500
Subject: [Python-Dev] speeding up PyObject_GetItem
In-Reply-To: <eae285400903240850n24a3ad3exf6fa5c30e00e770a@mail.gmail.com>
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com>
	<6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001>
	<49C8A846.4020507@avl.com>
	<eae285400903240530x2ecd50f7h53212bae422a8866@mail.gmail.com>
	<eae285400903240742o1489f3c1veffae75e4493b073@mail.gmail.com>
	<5c6f2a5d0903240813j318ea1b5m4fe6b20fedf56f99@mail.gmail.com>
	<eae285400903240850n24a3ad3exf6fa5c30e00e770a@mail.gmail.com>
Message-ID: <43aa6ff70903241205q507cf7c1i28a67aa634b05a41@mail.gmail.com>

2009/3/24 Daniel Stutzbach <daniel at stutzbachenterprises.com>:
> On Tue, Mar 24, 2009 at 10:13 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
>>
>> 2009/3/24 Daniel Stutzbach <daniel at stutzbachenterprises.com>:
>> > [...]
>> > 100 nanoseconds, py3k trunk:
>> > ceval -> PyObject_GetItem (object.c) -> list_subscript (listobject.c) ->
>> > PyNumber_AsSsize_t (object.c) -> PyLong_AsSsize_t (longobject.c)
>> > [more timings snipped]
>>
>> Does removing the PyLong_Check call in PyLong_AsSsize_t
>> make any noticeable difference to these timings?
>
> Making no other changes from the trunk, removing the PyLong_Check and NULL
> check from PyLong_AsSsize_t shaves off 4 nanoseconds (or around 4% since the
> trunk is around 100 nanoseconds).
>
> Here's what I'm testing with, by the way:
>
> ./python.exe Lib/timeit.py -r 10 -s 'x = list(range(10))' 'x[5]'

What difference does it make on real applications? Are you running any
macro-benchmarks against this?

Collin

From curt at hagenlocher.org  Tue Mar 24 20:13:58 2009
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Tue, 24 Mar 2009 12:13:58 -0700
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <m2vdpy21u0.fsf@valheru.db3l.homeip.net>
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
Message-ID: <d2155e360903241213v49e3b2ccr3bcdc7facbe36a@mail.gmail.com>

On Tue, Mar 24, 2009 at 11:49 AM, David Bolen <db3l.net at gmail.com> wrote:
>
> Kristj?n Valur J?nsson <kristjan at ccpgames.com> writes:
>
>> These issues should be resolved in the py3k branch, but it will need
>> porting to 2.6. ?Dialogue boxes are annoying, but do they pop up if
>> you run your buildslave as a service without access to the console?
>
> Not sure what the MSVC++ runtime does with assertion dialogs if run
> without interactive access (they aren't console messages, but UI dialog
> popups).

At least at some point in the past, it didn't check to see if there
was an attached window station.  But the dialog can be disabled in
debug mode by using the _set_error_mode function.
(http://msdn.microsoft.com/en-us/library/sas1dkb2(VS.71).aspx)

--
Curt Hagenlocher
curt at hagenlocher.org

From a.badger at gmail.com  Tue Mar 24 20:11:29 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Tue, 24 Mar 2009 12:11:29 -0700
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <5b8d13220903241153i6f2276aq24710e629139becc@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<87ab7bh5fb.fsf@xemacs.org>
	<49C87004.2030807@holdenweb.com>	<49C88503.2030902@v.loewis.de>
	<49C886EF.80203@v.loewis.de>	<49C8C9B3.3070403@holdenweb.com>	<5b8d13220903240700l72ab6e5fk479bd2fa1cdfd958@mail.gmail.com>	<gqb4oi$qb9$1@ger.gmane.org>	<5b8d13220903241041o3355b8bfk61f66fc9ea1bc68c@mail.gmail.com>	<gqb80a$6gb$1@ger.gmane.org>
	<5b8d13220903241153i6f2276aq24710e629139becc@mail.gmail.com>
Message-ID: <49C93061.200@gmail.com>

David Cournapeau wrote:
> On Wed, Mar 25, 2009 at 1:45 AM, Toshio Kuratomi <a.badger at gmail.com>
wrote:
>> David Cournapeau wrote:
>>> 2009/3/24 Toshio Kuratomi <a.badger at gmail.com>:
>>>> Steve Holden wrote:
>>>>
>>>>> Seems to me that while all this is fine for developers and Python
users
>>>>> it's completely unsatisfactory for people who just want to use Python
>>>>> applications. For them it's much easier if each application comes with
>>>>> all dependencies including the interpreter.
>>>>>
>>>>> This may seem wasteful, but it removes many of the version
compatibility
>>>>> issues that otherwise bog things down.
>>>>>
>>>> The upfront cost of bundling is lower but the maintenance cost is
>>>> higher.  For instance, OS vendors have developed many ways of being
>>>> notified of and dealing with security issues.  If there's a security
>>>> issue with gtkmozdev and the python bindings to it have to be
>>>> recompiled, OS vendors will be alerted to it and have the
opportunity to
>>>> release updates on zero day, the day that the security announcement
goes
>>>> out.
>>> I don't think bundling should be compared to depending on the system
>>> libraries, but as a lesser evil compared to requiring multiple,
>>> system-wide installed libraries.
>>>
>> Well.. I'm not so sure it's even a win there.  If the libraries are
>> installed system-wide, at least the consumer of the application knows:
>>
>> 1) Where to find all the libraries to audit the versions when a security
>> issue is announced.
>> 2) That the library is unforked from upstream.
>> 3) That all the consumers of the library version have a central location
>> to collaborate on announcing fixes to the library.
>
> Yes, those are problems, but installing multi libraries have a lot of
> problems too:
>  - quickly, by enabling multiple version installed, people become very
> sloppy to handle versions of the dependencies, and this increases a
> lot the number of libraries installed - so the advantages above for
> system-wide installation  becomes intractable quite quickly

This is somewhat true.  Sloppiness and increased libraries are bad.  But
there are checks on this sloppiness.  Distributions, for instance, are
quite active about porting software to use only a subset of versions.
So in the open source world, there's a large number of players
interested in keeping the number of versions down.  Using multiple
libraries will point people at where work needs to be done whereas
bundling hides it behind the monolithic bundle.

>  - bundling also supports a real user-case which cannot be solved by
> rpm/deb AFAIK: installation without administration privileges.

This is only sortof true.  You can install rpms into a local directory
without root privileges with a commandline switch.  But rpm/deb are
optimized for system administrators so the documentation on doing this
is not well done.  There can also be code issues with doing things this
way but those issues can affect bundled apps as well. And finally, since
rpm's primary use is installing systems, the toolset around it builds
systems.  So it's a lot easier to build a private root filesystem than
it is to cherrypick a single package.  It should be possible to create a
tool that merges a system rpmdb and a user's local rpmdb using the
existing API but I'm not aware of any applications built to do that yet.

>  - multi-version installation give very fragile systems. That's
> actually my number one complain in python: setuptools has caused me
> numerous headache, and I got many bug reports because you often do not
> know why one version was loaded instead of another one.
>
I won't argue for setuptools' implementation of multi-version.  It
sucks.  But multi-version can be done well.  Sonames in C libraries are
a simple system that does this better.

> So I am not so convinced multiple-version is better than bundling - I
> can see how it sometimes can be, but I am not sure those are that
> important in practice.
>
Bundling is always harmful.  Whether multiple versioning is any better
is certainly debatable :-)

>> No.  This is a social problem.  Good source control only helps if I am
>> tracking upstream's trunk so I'm aware of the direction that their
>> changes are headed.  But there's a wide range of reasons that
>> application developers that bundle libraries don't do that:
>>
>> 1) not enough time in a day.  I'm working full-time on making my
>> application better.  Plus I have to update all these bundled libraries
>> from time to time, testing that the updates don't break anything.  I
>> don't have time to track trunk for all these libraries -- I barely have
>> time to track releases.
>
> Yes, but in that case, there is nothing you can do. Putting everything
> in one project is always easier than splitting into modules, coding
> and deployment-wise. That's just one side of the speed of development
> vs maintenance issue IMHO.
>
>> 3) This doesn't help with the fact that my bundled version of the
>> library and your bundled version of the library are being developed in
>> isolation from each other.  This needs central coordination which people
>> who believe bundling libraries are very unlikely to pursue.
>
> As above, I think that in that case, it will happen whatever tools are
> available, so it is not a case worth being pursued.
>
I'm confused -- if it will happen whatever tools are available, how does
"good source control" solve the issue?  I'm saying that this is not an
issue that can be solved by having good source control... it's a social
issue that has to be solved by people learning to avoid bad practices.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090324/275e64fa/attachment-0001.pgp>

From martin at v.loewis.de  Tue Mar 24 20:51:22 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Mar 2009 20:51:22 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C8C9B3.3070403@holdenweb.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>	<49C87004.2030807@holdenweb.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com>
Message-ID: <49C939BA.8040206@v.loewis.de>

> But I don't think that a "normal" Lunix user should ever have to involve
> themselves directly with distutils or setuptools (it's OK to invoke them
> from the installation routine, but not to require the user to unpack the
> application then run setup.py).

I completely agree. It would be best if the Linux distribution provided
packages for most of the stuff (in RPM or deb format as appropriate),
and volunteers provided a repository of packages in a format that the
native package management tool actually understands (rather than
PyPI - although it might be useful to have PyPI provide a Debian
package repository for all projects that upload .deb files)

> I am afraid that distutils, and
> setuptools, are not really the answer to the problem, since while they
> may (as intended) guarantee that Python applications can be installed
> uniformly across different platforms they also more or less guarantee
> that Python applications are installed differently from all other
> applications on the platform.

Ah, so we are clearly in agreement here.

> Mike Driscoll did some work providing Windows installers for various
> Python packages and extension modules, and people were amused that he
> provided executable installers for pure Python libraries. But I saw that
> as a sensible decision, since it meant that Windows users (and even
> Windows Python application developers) used the same mechanism to
> install everything.

The Windows story is indeed sad, as none of the Windows packaging
formats provides support for dependencies (MSI has some support,
but as far as I understand it, it's pretty useless). But yes, for
Windows, you want .exe or .msi installers, not something proprietary.

Regards,
Martin

From martin at v.loewis.de  Tue Mar 24 21:01:58 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 24 Mar 2009 21:01:58 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <5b8d13220903240700l72ab6e5fk479bd2fa1cdfd958@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	
	<49BAA596.5020106@v.loewis.de> <49C79C1A.8040301@simplistix.co.uk>	
	<49C7FC85.5000809@v.loewis.de> <49C80FA0.4020800@simplistix.co.uk>	
	<87ab7bh5fb.fsf@xemacs.org> <49C87004.2030807@holdenweb.com>	
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>	
	<49C8C9B3.3070403@holdenweb.com>
	<5b8d13220903240700l72ab6e5fk479bd2fa1cdfd958@mail.gmail.com>
Message-ID: <49C93C36.5050300@v.loewis.de>

> If the problem is to get a recent enough version of the library, then
> the library would better be installed "locally", for the application.
> If it is too much a problem because the application depends on
> billions of libraries which are 6 months old, the problem is to allow
> such a dependency in the first place. What kind of nightmare would it
> be if programs developed in C would required a C library which is 6
> months old ? That's exactly what multiple-versions installations
> inflict on us. That's great for testing, development. But for
> deployment on end-user machines, the whole thing is a failure IMO.

I see this as a consequence of Web 2.0, and open source: Publish
early, publish often, make the pieces you publish as tiny as possible
to maximize reuse :-(

If people invent a new marshaling format every year (XML-RPC, SOAP,
JSON, YAML, ...), you *have* to use a just-published library to follow
the hype curve. If you have been raised in this tradition, you
are shocked by the number of modules in the Python library, and wish
for this dinosaur to die.

Apparently, deployment to end user machines is not so much of an issue
usually, since the mess will live on the web servers, and not on
everybody's end user machine.

Regards,
Martin

From cournape at gmail.com  Tue Mar 24 21:08:57 2009
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 25 Mar 2009 05:08:57 +0900
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C93061.200@gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com>
	<5b8d13220903240700l72ab6e5fk479bd2fa1cdfd958@mail.gmail.com>
	<gqb4oi$qb9$1@ger.gmane.org>
	<5b8d13220903241041o3355b8bfk61f66fc9ea1bc68c@mail.gmail.com>
	<gqb80a$6gb$1@ger.gmane.org>
	<5b8d13220903241153i6f2276aq24710e629139becc@mail.gmail.com>
	<49C93061.200@gmail.com>
Message-ID: <5b8d13220903241308p6df4fa6bjf71ebcbc8a2ff589@mail.gmail.com>

>
> This is only sortof true. ?You can install rpms into a local directory
> without root privileges with a commandline switch. ?But rpm/deb are
> optimized for system administrators so the documentation on doing this
> is not well done. ?There can also be code issues with doing things this
> way but those issues can affect bundled apps as well. And finally, since
> rpm's primary use is installing systems, the toolset around it builds
> systems. ?So it's a lot easier to build a private root filesystem than
> it is to cherrypick a single package. ?It should be possible to create a
> tool that merges a system rpmdb and a user's local rpmdb using the
> existing API but I'm not aware of any applications built to do that yet.

Building private root kind of defeat the purpose :) Deploying linux
packages in a reliable way without requiring admin privileges is an
"interesting" experience. The tools just don't support this well - or
there exists some magical tools that I've never seen mention of.

> I won't argue for setuptools' implementation of multi-version. ?It
> sucks. ?But multi-version can be done well. ?Sonames in C libraries are
> a simple system that does this better.

I would say simplistic instead of simple :) what works for C won't
necessarily work for python - and even in C, library versioning is not
used that often except for a few core libraries. Library versioning
works in C because C model is very simple. It already breaks for C++.
More high-level languages like C# already have a more complicated
scheme (GAC) - and my impression is that it did not work that well.
The SxS for dll on recent windows to handle multiple version is a
nightmare too in my (limited) experience.

>>
> I'm confused -- if it will happen whatever tools are available, how does
> "good source control" solve the issue? ?I'm saying that this is not an
> issue that can be solved by having good source control... it's a social
> issue that has to be solved by people learning to avoid bad practices.

I meant that whatever technology is available, bundling everything
will always be easier. And sometimes, given the time/resource
constraints, that's even the only realistic option. I could tell you
many stories about wasted hours related to some fortran libraries
which were hopelessly broken (missing symbols) on most distributions,
or ABI conflicts - for which bundling sometimes is the only path to
keep its sanity (short of giving up support for the platform). When
you need to solve the problem now because you want to demo things
tomorrow, not in 6 months, that's the only solution. It is not always
bad practice.

IMHO, one should focus on making it easier to avoid bundling
everything - robust and simple dependency scheme, adapting distutils
installation scheme to the various OS conventions, be it FHS, windows,
etc... But we can't and shouldn't prevent it totally, and tools are
already there to help minimizing the problems of bundling. For
multiple system-wide libraries, I have yet to encounter anything which
makes it somewhat reliable - it has only caused problems for me, and
not solved any single problem.

cheers,

David

From guido at python.org  Tue Mar 24 21:14:03 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Mar 2009 13:14:03 -0700
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
	currently conform
In-Reply-To: <b8ad139e0903231400h570d771bt588ccbc71a295db4@mail.gmail.com>
References: <b8ad139e0903231352i66bc38f3v4f88b0914c760df7@mail.gmail.com>
	<b8ad139e0903231400h570d771bt588ccbc71a295db4@mail.gmail.com>
Message-ID: <ca471dc20903241314t318f2fb2vf864dae094592f53@mail.gmail.com>

Please don't do this. We need stable APIs. Trying to switch the entire
community to use CapWord APIs for something as commonly used as
datetime sounds like wasting a lot of cycles with no reason except the
mythical "PEP 8 conformance". As I said, it's a pity we didn't change
this at the 3.0 point, but I think going forward we should try to be
more committed to slow change. Additions of new functionality are of
course fine. But renamings (even if the old names remain available)
are just noise.

--Guido

On Mon, Mar 23, 2009 at 2:00 PM, Jess Austin <jaustin at post.harvard.edu> wrote:
> On Tue, Mar 3 at 19:25:21 Guido van Rossum <guido at python.org> wrote:
>> On Tue, Mar 3, 2009 at 5:15 PM, Brett Cannon <brett at python.org> wrote:
>>>
>>>
>>> On Tue, Mar 3, 2009 at 05:13, <rdmurray at bitdance.com> wrote:
>>>>
>>>> On Tue, 3 Mar 2009 at 06:01, Ivan Krsti?~G wrote:
>>>>>
>>>>> On Mar 2, 2009, at 7:08 PM, Steve Holden wrote:
>>>>>>
>>>>>> > > ?PS.: so is datetime.datetime a builtin then? :)
>>>>>> > > ?Another historic accident. Like socket.socket. :-(
>>>>>> >
>>>>>> ?A pity this stuff wasn't addressed for 3.0. Way too late now, though.
>>
>> A pity indeed.
>>
>>>>> It may be too late to rename the existing accidents, but why not add
>>>>> consistently-named aliases (socket.Socket, datetime.DateTime, etc) and
>>>>> strongly encourage their use in new code?
>>>
>>> Or make the old names aliases for the new names and start a
>>> PendingDeprecationWarning on the old names so they can be switched in the
>>> distant future?
>>
>> +1, if it's not done in a rush and only for high-visibility modules --
>> let's start with socket and datetime.
>>
>> We need a really long lead time before we can remove these. I
>> recommend starting with a *silent* deprecation in 3.1 combined with a
>> PR offensive for the new names.
>
> I've uploaded a patch for the datetime module with respect to this
> issue at http://bugs.python.org/issue5530 . I would appreciate it if
> experienced developers could take a look at it and provide some
> feedback. ?Since I've only been hacking on CPython for about a month,
> please be kind! ?I'm happy to make changes to this.
>
> As it stands now, the patch changes the current objects to have
> CapWords names, and subclasses these objects to provide objects with
> the old names. Use of methods (including __new__) of the derived
> objects causes PendingDeprecations (if one has warning filters
> appropriately set).
>
> A warning: this patch requires the patch to the test refactoring at
> Issue 5520 to completely apply. ?It will fail one test without the
> patch at Issue 5516. ?Both of these are (inexpertly) linked from the
> roundup page for this issue.
>
> I hope this will be helpful.
>
> cheers,
> Jess Austin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jaustin at post.harvard.edu  Tue Mar 24 21:23:19 2009
From: jaustin at post.harvard.edu (Jess Austin)
Date: Tue, 24 Mar 2009 15:23:19 -0500
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
	currently conform
In-Reply-To: <ca471dc20903241314t318f2fb2vf864dae094592f53@mail.gmail.com>
References: <b8ad139e0903231352i66bc38f3v4f88b0914c760df7@mail.gmail.com>
	<b8ad139e0903231400h570d771bt588ccbc71a295db4@mail.gmail.com>
	<ca471dc20903241314t318f2fb2vf864dae094592f53@mail.gmail.com>
Message-ID: <b8ad139e0903241323m3496f0fci9ec020495f1e680c@mail.gmail.com>

On Tue, Mar 24, 2009 at 3:14 PM, Guido van Rossum <guido at python.org> wrote:
> Please don't do this. We need stable APIs. Trying to switch the entire
> community to use CapWord APIs for something as commonly used as
> datetime sounds like wasting a lot of cycles with no reason except the
> mythical "PEP 8 conformance". As I said, it's a pity we didn't change
> this at the 3.0 point, but I think going forward we should try to be
> more committed to slow change. Additions of new functionality are of
> course fine. But renamings (even if the old names remain available)
> are just noise.

OK, I had misunderstood your earlier message.  Sorry for the confusion.

thanks,
Jess

From tjreedy at udel.edu  Tue Mar 24 21:25:09 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 24 Mar 2009 16:25:09 -0400
Subject: [Python-Dev] tracker status options
In-Reply-To: <Pine.LNX.4.64.0903240514490.31981@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0903240514490.31981@kimball.webabinitio.net>
Message-ID: <gqbfj5$2js$1@ger.gmane.org>

R. David Murray wrote:

> So, having triaged a few issues, here are my thoughts.
> 
> The current workflow is roughly:
> 
>     o test needed
>     o patch needed
>     o patch review
>     o commit review
> 
> One can look at these and see what needs to be done "next".  I think
> that in practice the above list actually expands something like this:
> 
>     o consensus needed
>     o test needed
>     o patch needed
>     o patch needs work
>     o patch review
>     o commit review

As a point of comparison, here are the GNUnet mantis status codes.
The following status codes are used in Mantis:

* New
     A new bug, developers did not look into these yet.
* Feedback
     Developers require feedback from users reporting the bug to resolve 
it. Also used if a general discussion between the researches is needed 
on how to address a problem.
* Acknowledged
     Developers have seen the bug.
* Confirmed
     Developers are convinced that the bug is a problem that needs to be 
fixed.
* Assigned
     Some developer has started working on the problem. Note that 
developers may give up on problems, putting the bug back to confirmed, 
or feedback.
* Resolved
     The bug has been fixed in some version in Subversion or in a patch 
attached to the bug report.
* Closed
     Resolved bugs are closed after the bugfix has made it into a full 
release of GNUnet.

tjr


From guido at python.org  Tue Mar 24 21:25:36 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Mar 2009 13:25:36 -0700
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
	currently conform
In-Reply-To: <b8ad139e0903241323m3496f0fci9ec020495f1e680c@mail.gmail.com>
References: <b8ad139e0903231352i66bc38f3v4f88b0914c760df7@mail.gmail.com>
	<b8ad139e0903231400h570d771bt588ccbc71a295db4@mail.gmail.com>
	<ca471dc20903241314t318f2fb2vf864dae094592f53@mail.gmail.com>
	<b8ad139e0903241323m3496f0fci9ec020495f1e680c@mail.gmail.com>
Message-ID: <ca471dc20903241325i72b45328y2dcebc08c274c1e8@mail.gmail.com>

On Tue, Mar 24, 2009 at 1:23 PM, Jess Austin <jaustin at post.harvard.edu> wrote:
> On Tue, Mar 24, 2009 at 3:14 PM, Guido van Rossum <guido at python.org> wrote:
>> Please don't do this. We need stable APIs. Trying to switch the entire
>> community to use CapWord APIs for something as commonly used as
>> datetime sounds like wasting a lot of cycles with no reason except the
>> mythical "PEP 8 conformance". As I said, it's a pity we didn't change
>> this at the 3.0 point, but I think going forward we should try to be
>> more committed to slow change. Additions of new functionality are of
>> course fine. But renamings (even if the old names remain available)
>> are just noise.
>
> OK, I had misunderstood your earlier message. ?Sorry for the confusion.

No problem! I was probably using shorthand that only experienced devs
understood.

I hope you'll find other things to contribute!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Tue Mar 24 21:27:59 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Mar 2009 21:27:59 +0100
Subject: [Python-Dev] tracker status options
In-Reply-To: <Pine.LNX.4.64.0903240514490.31981@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0903240514490.31981@kimball.webabinitio.net>
Message-ID: <49C9424F.2030407@v.loewis.de>

>     o consensus needed
>     o test needed
>     o patch needed
>     o patch needs work
>     o patch review
>     o commit review
> 
> The first of these additional items is equivalent to your bullet item
> above.  I would propose that the issue, regardless of whether or not
> it is a bug fix or a feature request, goes into 'consensus needed'

Well, there is always the "unset" state, which means "not triaged".
So if you propose that this should be the default initial state,
I'm opposed.

Instead, would it help to add a "confirmed" state? For a bug, that
would mean that it was successfully reproduced; for a patch, it
was confirmed as desirable.

If the person performing triage cannot confirm the bug, they should
reject the issue (or recommend rejection, in case they are unsure).
Somebody performing triage should never conclude with "I don't
know"; this would be time-wasting.

If, for a bug, the reviewer disagrees that it would be a bug if the
behavior actually exists, then the issue should be closed (resolution
not-a-bug/invalid). If the reviewer agrees that it would be a bug,
but fails to reproduce it, a test is needed.

Regards,
Martin

From martin at v.loewis.de  Tue Mar 24 21:34:51 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 24 Mar 2009 21:34:51 +0100
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <m2vdpy21u0.fsf@valheru.db3l.homeip.net>
References: <loom.20090324T134451-902@post.gmane.org>	<m2zlfa241x.fsf@valheru.db3l.homeip.net>	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
Message-ID: <49C943EB.90506@v.loewis.de>

> There was a discussion about this on the py3k mailing list back in
> mid-2007 ("buildbots" thread) and perhaps later as well, at which
> point I believe Martin added an "-n" option to regrtest and the
> buildbot test.bat file to disable the assertions.  Is that the py3k
> branch piece you are referring to?

I don't quite remember the -n flag, but I believe that Kristjan
just removed all that stuff, ie. there is now no way to block
CRT assertions anymore.

> I
> wasn't paying close attention, so maybe there's some other mechanism
> in place at this point?

Unfortunately, not. There was a long debate, I lost, and Kristjan
removed all the changes to support this setup, claiming that it is
good to get assertions :-(

Regards,
Martin

From martin at v.loewis.de  Tue Mar 24 21:40:42 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Mar 2009 21:40:42 +0100
Subject: [Python-Dev] GSoC: Replace MS Windows Console with Unicode UI
In-Reply-To: <ee9557410903241016h2f1e7b1cp4afdf96afd7f293f@mail.gmail.com>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>	<gq70q2$hul$1@ger.gmane.org>
	<49C70C6A.7070106@g.nevcal.com>	<gq8mu2$mfl$1@ger.gmane.org>
	<49C7FF1F.6080707@g.nevcal.com>
	<ee9557410903241016h2f1e7b1cp4afdf96afd7f293f@mail.gmail.com>
Message-ID: <49C9454A.7090600@v.loewis.de>

> Hi. I'm Japanese and non-ascii charactor user. (cp932)
> 
> We have to use "IME" to input non-ascii charactor in Windows.
> When "> chcp 65001" in cmd.exe, we cannot use IME on cmd.exe.
> 
> So setting codepage to 65001 make output universal but make input ascii-only.
> Sit!!!

Is there a code page that still allows IME input, but supports all
of Unicode? I know there is GB18030 - is it any good?

In any case, for *that* problem, I think you need to look for a
different terminal program.

> I hope PyQtShell <http://code.google.com/p/pyqtshell/> become good
> IDLE alternative.

Could well be. To see it integrated into Python (and perhaps replace
IDLE), a lot of steps would need to happen, though. But perhaps you
weren't asking for it to be included in Python - it can also be a good
alternative if it's not included (like several other IDEs)

Regards,
Martin

From guido at python.org  Tue Mar 24 21:44:48 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Mar 2009 13:44:48 -0700
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49C5620A.4080908@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
Message-ID: <ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>

On Sat, Mar 21, 2009 at 2:54 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Antoine Pitrou wrote:
>
>> Do we really want to add a syntactic feature which has such a complicated
>> expansion? I fear it will make code using "yield from" much more difficult
>> to
>> understand and audit.
>
> As I've said before, I don't think the feature itself is
> difficult to understand. You're not meant to learn about
> it by reading the expansion -- that's only there to pin
> down all the details for language lawyers.
>
> For humans, almost all the important information is
> contained in one paragraph near the top:
>
> "When the iterator is another generator, the effect is the same as if
> the body of the subgenerator were inlined at the point of the ``yield
> from`` expression. Furthermore, the subgenerator is allowed to execute
> a ``return`` statement with a value, and that value becomes the value of
> the ``yield from`` expression."
>
> Armed with this perspective, do you still think there will
> be difficulty in understanding or auditing code?

Well, hmm... I've been out of the loop due to other commitments
(sorry), but I really don't like to have things whose semantics is
defined in terms of code inlining -- even if you don't mean that as
the formal semantics but just as a mnemonic hint. It causes all sorts
of confusion about scopes.

What happened to the first-order approximation

  "yield from X" means roughly the same as "for _x in X: yield x"

?

The more specialized semantics in some cases can probably be put off
until later in the document.

FWIW I am okay with the notion that if the immediate subiterator
returns a value, that value becomes the value of the
yield-from-expression. Suitable semantics that make this effect pass
through multiple layers of sub-iterators are fine too. But the exact
semantics in the light of try/except or try/finally blocks on the
stack are incredibly (perhaps impossibly) tricky to get right -- and
it probably doesn't matter all that much what exactly happens as long
as it's specified in sufficient detail that different implementations
behave the same way (apart from obvious GC differences, alas).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tseaver at palladion.com  Tue Mar 24 21:58:50 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 24 Mar 2009 16:58:50 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <5b8d13220903241153i6f2276aq24710e629139becc@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<87ab7bh5fb.fsf@xemacs.org>
	<49C87004.2030807@holdenweb.com>	<49C88503.2030902@v.loewis.de>
	<49C886EF.80203@v.loewis.de>	<49C8C9B3.3070403@holdenweb.com>	<5b8d13220903240700l72ab6e5fk479bd2fa1cdfd958@mail.gmail.com>	<gqb4oi$qb9$1@ger.gmane.org>	<5b8d13220903241041o3355b8bfk61f66fc9ea1bc68c@mail.gmail.com>	<gqb80a$6gb$1@ger.gmane.org>
	<5b8d13220903241153i6f2276aq24710e629139becc@mail.gmail.com>
Message-ID: <gqbhic$a0m$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Cournapeau wrote:
> On Wed, Mar 25, 2009 at 3:15 AM, Tres Seaver <tseaver at palladion.com> wrote:
> 
>>> Everytime I tried to understand what buildout was about, I was not
>>> even sure it could help for my own problems at all. It seems very
>>> specific to web development - I may completely miss the point ?
>> I think so:  it is largely a way to get repeatable / scripted deployment
>> of software to disk.  It uses setuptools to install Python package
>> distributions, but also can use other means (e.g, configure-make-make
>> install to install a C library such as libxml2).  The end result is a
>> self-contained directory tree:
>>
>> - - Scripts in the 'bin' directory are configured to have the specific
>>  Python pacakges (and versions) they require on the PYTHONPATH.
>>
>> - - By convention, released package distributions are installed into the
>>  'eggs' subdirectory', which is *not* on the PYTHONPATH, nor is it a
>>  'site' directory for Python.
>>
>> - - Other bits are typically in their own subdirectories, often under
>>  'parts'.
> 
> Ok - but I don't think it helps much, see below.
> 
> 
>> When not doing Plone / Zope-specific work (where zc.buildout is a de
>> facto standard), I use 'virtualenv' to create isolated environments into
>> which I install the libraries for a given application.  If your
>> application ships as Python package distributions, then documenting the
>> use of 'virtualenv' as a "supported" way to install it might reduce your
>> support burden.
> 
> I now realize why we don't understand each other - in my case, the one
> doing the installation is the user, who cannot be assumed to know much
> about python.q11 That's what I mean by "application deployment vs
> webapp deployment". Ideally, the user has to enter one command/click
> one button, and he can use the application - he may not even be a
> programmer (I deploy things based on numpy/scipy for scientific
> people, who often have little patience for things which take more than
> 1 minute to set up software-wise).
> 
> That's the user case I am mostly interested in - and I think it is
> quite general, and quite different from plone deployment kind of
> things.

OK, but I'm still not convinced that the Plone installers don't fit this
case.  It's just that they also allow add-on installation afterward:
the "main" Plone application is installed via a GUI wizard (on Windows /
OSX).

>> You can think of zc.buildout or the virtualenv-based script as a form of
>> bundling, which bootstraps from another already-installed Python, but
>> remains isolated from anything in its 'site-packages'.
> 
> Yes, I know what virtualenv is, I use it sometimes - but it is
> definitely too complicated for the people I want to distribute my
> software to.

Have you tried using the script you can generate from it?  E.g.,:

 >>> MY_ADDON_CODE = """\
 ...
 def after_install(options, home_dir):
     subprocess.call([join(home_dir, 'bin', 'easy_install'),
                                 'MyPackage'])
 """
 >>> from virtualenv import create_bootstrap_script
 >>> f = open('/tmp/myinstaller.py', 'w')
 >>> f.write(create_bootstrap_script(MY_ADDON_CODE)
 >>> f.close()

Users would then be able to run that script and get a virtualenv, with
your 'MyPackage' distribution already installed in it.

>> I never even use that switch manually.  zc.buildout does, but that is
>> because it wants to control the PYTHONPATH by generating the script
>> code:  it doesn't ask users to tweak that.
> 
> Well, that's the thing: neither do I :) but if my software is a
> dependency of another software, then I am bothered for problems with
> software which are not used at all by my own package (because
> setuptools makes an egg of my software automatically, screw things up,
> and I am the one blamed for it).

Hmm, if your package uses "pure" distutils facilities, then it should be
installable by setuptools as an egg without problems.  Why would your
users be trying to to '--multi-version' installs of your package?

>> I don't know why anybody who was not writing a packaging tool, or
>> packaging a library for something like .deb / .rpm, would even use the
>> multi-version option for setuptools:  I don't see any sane way to
>> install conflicting requirements into a shared 'site-packages'.
> 
> But that's the problem: it often happens *even if you don't use
> setuptools yourself*. I would not be surprised if that's one reason
> why so many people have a seemingly unfair bias against setuptools.
> That's certainly the reason in my case.

Again:  I've *never* seen packages installed in multi-version fashion,
except in the very different mode (not in sys.path, nor in a directory
significant to site.py) used by zc.buildout.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJyUmK+gerLs4ltQ4RAmL6AJ4pzQiVLeoHWPkJ4yi1nNX2d9svBACfXrEZ
wKDmINnLwkDzTmzhYoJQnP0=
=pTeA
-----END PGP SIGNATURE-----


From rdmurray at bitdance.com  Tue Mar 24 22:20:27 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 24 Mar 2009 17:20:27 -0400 (EDT)
Subject: [Python-Dev] tracker status options
In-Reply-To: <49C9424F.2030407@v.loewis.de>
References: <Pine.LNX.4.64.0903240514490.31981@kimball.webabinitio.net>
	<49C9424F.2030407@v.loewis.de>
Message-ID: <Pine.LNX.4.64.0903241646550.31981@kimball.webabinitio.net>

On Tue, 24 Mar 2009 at 21:27, "Martin v. L?wis" wrote:

>>     o consensus needed
>>     o test needed
>>     o patch needed
>>     o patch needs work
>>     o patch review
>>     o commit review
>>
>> The first of these additional items is equivalent to your bullet item
>> above.  I would propose that the issue, regardless of whether or not
>> it is a bug fix or a feature request, goes into 'consensus needed'
>
> Well, there is always the "unset" state, which means "not triaged".
> So if you propose that this should be the default initial state,
> I'm opposed.

No, I was not suggesting it be the default state.

> Instead, would it help to add a "confirmed" state? For a bug, that
> would mean that it was successfully reproduced; for a patch, it
> was confirmed as desirable.

So, 'confirmed' instead of 'consensus needed' (or confirmed/approved,
to cover feature requests), and 'patch is appropriate' that comes...I'm
not quite sure where?

> If the person performing triage cannot confirm the bug, they should
> reject the issue (or recommend rejection, in case they are unsure).
> Somebody performing triage should never conclude with "I don't
> know"; this would be time-wasting.

The cases I was considering are cases where in the comments on the ticket
there is disagreement either on whether or not the bug is a bug or (more
often) whether or not the feature is desirable, and at the patch stage
whether or not the patch is the appropriate fix.  I think currently
things sit in 'patch needed' until consensus is reached on the patch,
but I haven't watched enough tickets progress yet to be sure :)
Adding another stage here may be more confusing than it is helpful,
seeing as I can't really figure out where it would go.

Did I guess correctly that the process for "recommending rejection"
is to set the stage to 'commit/reject', the resolution to 'invalid'
(or whatever is appropriate) and the status to 'pending'?  That
seemed to work for the issue I did it to, in that someone came
along and closed it shortly after that.

> If, for a bug, the reviewer disagrees that it would be a bug if the
> behavior actually exists, then the issue should be closed (resolution
> not-a-bug/invalid). If the reviewer agrees that it would be a bug,
> but fails to reproduce it, a test is needed.

OK, so I guess I now understand how the current workflow is supposed to
work: if its stage is 'unassigned', then it hasn't been accepted as a
bug/enhancement yet, and triage should make that accept/reject judgement.

The tricky bit here for me is that I, as a new person doing triage,
don't always feel comfortable in passing judgement on a bug or feature
request.  So what would be the mechanism for a triage person to request a
"passing of judgement" from someone with more experience/authority?  Post
to python-dev?  Given such a mechanism, I think I would be comfortable
with the current workflow, with the expectation that I would need to
call for assistance less and less frequently over time, and ultimately
only for those things where discussion among the devs really is needed.

Hmm.  Maybe I should write a short "guide to performing triage" and
post it for feedback.

--
R. David Murray           http://www.bitdance.com

From kristjan at ccpgames.com  Tue Mar 24 22:29:07 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Tue, 24 Mar 2009 21:29:07 +0000
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <d2155e360903241213v49e3b2ccr3bcdc7facbe36a@mail.gmail.com>
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<d2155e360903241213v49e3b2ccr3bcdc7facbe36a@mail.gmail.com>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F0519BB4A9F@exchis.ccp.ad.local>

There was a recent test added to test_fileio.py which triggered runtime checks on windows.
I fixed this and other related issues in a series of revisions around 70580.
Now, I know that this msvc behaviour can be disabled, but it was decided that it was not appropriate to meddle with runtime flags of the whole process for python.
It is unfortunate that a dialogue box is put up, even when the process is running without a UI.  I wonder if there is a way to fix that without changing the way python is compiled...

K

-----Original Message-----
From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Curt Hagenlocher
Sent: 24. mars 2009 19:14
To: David Bolen
Cc: python-dev at python.org
Subject: Re: [Python-Dev] Test failures under Windows?

On Tue, Mar 24, 2009 at 11:49 AM, David Bolen <db3l.net at gmail.com> wrote:
>
> Kristj?n Valur J?nsson <kristjan at ccpgames.com> writes:
>
>> These issues should be resolved in the py3k branch, but it will need
>> porting to 2.6. ?Dialogue boxes are annoying, but do they pop up if
>> you run your buildslave as a service without access to the console?
>
> Not sure what the MSVC++ runtime does with assertion dialogs if run
> without interactive access (they aren't console messages, but UI dialog
> popups).

At least at some point in the past, it didn't check to see if there
was an attached window station.  But the dialog can be disabled in
debug mode by using the _set_error_mode function.
(http://msdn.microsoft.com/en-us/library/sas1dkb2(VS.71).aspx)

--
Curt Hagenlocher
curt at hagenlocher.org
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com


From martin at v.loewis.de  Tue Mar 24 22:41:34 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Mar 2009 22:41:34 +0100
Subject: [Python-Dev] tracker status options
In-Reply-To: <Pine.LNX.4.64.0903241646550.31981@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0903240514490.31981@kimball.webabinitio.net>
	<49C9424F.2030407@v.loewis.de>
	<Pine.LNX.4.64.0903241646550.31981@kimball.webabinitio.net>
Message-ID: <49C9538E.7060606@v.loewis.de>

> So, 'confirmed' instead of 'consensus needed' (or confirmed/approved,
> to cover feature requests), and 'patch is appropriate' that comes...I'm
> not quite sure where?

If the person who did the triage actually thinks the patch is ready to
be committed, it would be "commit review". Not sure in what other way
a patch might be appropriate.

> The cases I was considering are cases where in the comments on the ticket
> there is disagreement either on whether or not the bug is a bug or (more
> often) whether or not the feature is desirable, and at the patch stage
> whether or not the patch is the appropriate fix.

If the person doing the triage has made a final call, the issue can
enter the next stage. There should never be debate on the tracker, IMO
(although there often is). It might be that people disagree with a
triage, then they should appeal to python-dev, not on the issue itself.
It might be that they think the reviewer misunderstood the issue,
then they should clarify, and the reviewer should revert the status.

IMO, the reviewer should *always* take action, either by asking for
more information, or by advancing the issue to the next stage.

> Did I guess correctly that the process for "recommending rejection"
> is to set the stage to 'commit/reject', the resolution to 'invalid'
> (or whatever is appropriate) and the status to 'pending'?  That
> seemed to work for the issue I did it to, in that someone came
> along and closed it shortly after that.

If you have permission to do so, you should just close the issue
(in that manner). If you don't have permission, you can leave a message
saying "I recommend to close the issue".

If you are unsure, you can set it to Pending, and ask for help on
python-dev. In that case, you haven't actually done triage, but merely
considered it, and determined that it looks too difficult.

> The tricky bit here for me is that I, as a new person doing triage,
> don't always feel comfortable in passing judgement on a bug or feature
> request.  So what would be the mechanism for a triage person to request a
> "passing of judgement" from someone with more experience/authority?  Post
> to python-dev?

Exactly. Never assign to somebody, unless that somebody has explicitly
requested such an assignment, or unless he gave blank permission to
be assigned certain kinds of issues.

> Hmm.  Maybe I should write a short "guide to performing triage" and
> post it for feedback.

That can't hurt. Different people make different experiences.

Regards,
Martin

From rdmurray at bitdance.com  Tue Mar 24 23:14:18 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 24 Mar 2009 18:14:18 -0400 (EDT)
Subject: [Python-Dev] tracker status options
In-Reply-To: <49C9538E.7060606@v.loewis.de>
References: <Pine.LNX.4.64.0903240514490.31981@kimball.webabinitio.net>
	<49C9424F.2030407@v.loewis.de>
	<Pine.LNX.4.64.0903241646550.31981@kimball.webabinitio.net>
	<49C9538E.7060606@v.loewis.de>
Message-ID: <Pine.LNX.4.64.0903241759010.31981@kimball.webabinitio.net>

On Tue, 24 Mar 2009 at 22:41, "Martin v. L?wis" wrote:
> If the person doing the triage has made a final call, the issue can
> enter the next stage. There should never be debate on the tracker, IMO
> (although there often is). It might be that people disagree with a
> triage, then they should appeal to python-dev, not on the issue itself.
> It might be that they think the reviewer misunderstood the issue,
> then they should clarify, and the reviewer should revert the status.

OK, so given this then I revise the way I understand what is happening
in the ticket I'm looking at: a reviewer has said "this patch needs
work" and the submitter has not responded.  Since the behavior has been
accepted as a valid bug this means...I can either work on the patch, or
post a message asking if the submitter wants to either revise the patch
or discuss it on python-dev.  In the latter case if the submitter does
not respond, then the issue continues to languish.  IMO it shouldn't be
closed, because it really is a bug.

Does this match what you would expect a reviewer to do?  (A person
really doing triage would of course not work on the patch themselves :)

> IMO, the reviewer should *always* take action, either by asking for
> more information, or by advancing the issue to the next stage.
>
>> Did I guess correctly that the process for "recommending rejection"
>> is to set the stage to 'commit/reject', the resolution to 'invalid'
>> (or whatever is appropriate) and the status to 'pending'?  That
>> seemed to work for the issue I did it to, in that someone came
>> along and closed it shortly after that.
>
> If you have permission to do so, you should just close the issue
> (in that manner). If you don't have permission, you can leave a message
> saying "I recommend to close the issue".
>
> If you are unsure, you can set it to Pending, and ask for help on
> python-dev. In that case, you haven't actually done triage, but merely
> considered it, and determined that it looks too difficult.

OK, so I guess I've been given more power than I was expecting, and I'll
just have to step up the bar and learn to use it appropriately :)

--
R. David Murray           http://www.bitdance.com

From python at rcn.com  Tue Mar 24 23:15:29 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 24 Mar 2009 15:15:29 -0700
Subject: [Python-Dev] speeding up PyObject_GetItem
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com><6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001><49C8A846.4020507@avl.com><eae285400903240530x2ecd50f7h53212bae422a8866@mail.gmail.com><eae285400903240742o1489f3c1veffae75e4493b073@mail.gmail.com><5c6f2a5d0903240813j318ea1b5m4fe6b20fedf56f99@mail.gmail.com><eae285400903240850n24a3ad3exf6fa5c30e00e770a@mail.gmail.com><5c6f2a5d0903241045w69282138v27e0224d1563bbc@mail.gmail.com>
	<loom.20090324T181207-923@post.gmane.org>
Message-ID: <A5D07132EF7E4444B4CC911DE883BE92@RaymondLaptop1>


> 4% on a micro-micro-benchmark is hardly compelling...

I concur!  This is utterly insignificant and certainly does
not warrant removing the checks.

-1 on these sort of fake optimizations.  We should focus
on algorithmic improvements and eliminating redundant
work and whatnot.  Removing checks that were put 
there for a reason doesn't seem useful at all.


Raymond

From bignose+hates-spam at benfinney.id.au  Tue Mar 24 23:19:25 2009
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Wed, 25 Mar 2009 09:19:25 +1100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49BAA596.5020106@v.loewis.de> <49C79C1A.8040301@simplistix.co.uk>
	<49C7FC85.5000809@v.loewis.de>
	<49C80FA0.4020800@simplistix.co.uk> <87ab7bh5fb.fsf@xemacs.org>
	<49C87004.2030807@holdenweb.com> <49C88503.2030902@v.loewis.de>
	<49C886EF.80203@v.loewis.de> <49C8C9B3.3070403@holdenweb.com>
	<5b8d13220903240700l72ab6e5fk479bd2fa1cdfd958@mail.gmail.com>
	<49C93C36.5050300@v.loewis.de>
Message-ID: <8763hya7iq.fsf@benfinney.id.au>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> I see this as a consequence of Web 2.0, and open source: Publish
> early, publish often, make the pieces you publish as tiny as
> possible to maximize reuse :-(

I'm all in favour of re-use. But re-use should mean having the library
in a *single* predictable location (per declared version) on the
system, so that any code that needs it can declare the dependency and
find it there, and so that bugs in the library only need to be found
and fixed in *one* place.

> If people invent a new marshaling format every year (XML-RPC, SOAP,
> JSON, YAML, ...), you *have* to use a just-published library to
> follow the hype curve.

That's still no reason why that new version of the library can't be
installed to a central, predictable location.

> If you have been raised in this tradition, you are shocked by the
> number of modules in the Python library, and wish for this dinosaur
> to die.

It's more shocking to realise that there are even *more* libraries
installed privately by applications, where they can't be easily found
and fixed by a single update of the library.

-- 
 \      ?The process by which banks create money is so simple that the |
  `\     mind is repelled.? ?John Kenneth Galbraith, _Money: Whence It |
_o__)                                       Came, Where It Went_, 1975 |
Ben Finney


From skippy.hammond at gmail.com  Tue Mar 24 23:28:45 2009
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Wed, 25 Mar 2009 09:28:45 +1100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C939BA.8040206@v.loewis.de>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>	<49C87004.2030807@holdenweb.com>	<49C88503.2030902@v.loewis.de>
	<49C886EF.80203@v.loewis.de>	<49C8C9B3.3070403@holdenweb.com>
	<49C939BA.8040206@v.loewis.de>
Message-ID: <49C95E9D.8070604@gmail.com>

On 25/03/2009 6:51 AM, "Martin v. L?wis" wrote:
>> Mike Driscoll did some work providing Windows installers for various
>> Python packages and extension modules, and people were amused that he
>> provided executable installers for pure Python libraries. But I saw that
>> as a sensible decision, since it meant that Windows users (and even
>> Windows Python application developers) used the same mechanism to
>> install everything.
>
> The Windows story is indeed sad, as none of the Windows packaging
> formats provides support for dependencies (MSI has some support,
> but as far as I understand it, it's pretty useless). But yes, for
> Windows, you want .exe or .msi installers, not something proprietary.

Isn't this discussion slightly at cross-purposes?

* py2exe doesn't create *installers*, just the target application.  It 
(poorly) manages dependencies on the build machine.  There doesn't seem 
to be a need for runtime dependency management in this tool.

* The creation of an *installer* is something quite different.  An 
installer for a py2exe based tool also doesn't need dependency 
management.  An installer for a pure-python package that made no attempt 
to bundle dependencies might be nice, but I don't quite see how that 
falls outside the scope of distutils/setuptools/etc.  In other words, I 
don't see why the installer can't bootstrap the 'normal' dependency 
management which would be used if the package was installed any other 
way or on other platforms.

* distutils already has the ability to create Windows installer 
executables for pure-python apps/libs.  I agree it would be nice if it 
was an MSI but that is an implementation detail rather than 
implementation requirement.  How were Mike's packages fundamentally 
different than that?

Cheers,

Mark

From martin at v.loewis.de  Tue Mar 24 23:34:30 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Mar 2009 23:34:30 +0100
Subject: [Python-Dev] tracker status options
In-Reply-To: <Pine.LNX.4.64.0903241759010.31981@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0903240514490.31981@kimball.webabinitio.net>
	<49C9424F.2030407@v.loewis.de>
	<Pine.LNX.4.64.0903241646550.31981@kimball.webabinitio.net>
	<49C9538E.7060606@v.loewis.de>
	<Pine.LNX.4.64.0903241759010.31981@kimball.webabinitio.net>
Message-ID: <49C95FF6.5060403@v.loewis.de>

> OK, so given this then I revise the way I understand what is happening
> in the ticket I'm looking at: a reviewer has said "this patch needs
> work" and the submitter has not responded.  Since the behavior has been
> accepted as a valid bug this means...I can either work on the patch, or
> post a message asking if the submitter wants to either revise the patch
> or discuss it on python-dev.  In the latter case if the submitter does
> not respond, then the issue continues to languish.  IMO it shouldn't be
> closed, because it really is a bug.
> 
> Does this match what you would expect a reviewer to do?  (A person
> really doing triage would of course not work on the patch themselves :)

Right. The issue would be in the "patch needs work" stage until somebody
contributes. No further review needed, and yes, the issue will languish
until somebody improves the patch.

> OK, so I guess I've been given more power than I was expecting, and I'll
> just have to step up the bar and learn to use it appropriately :)

:-) With power comes responsibility, of course.

However, review/triage doesn't really help unless it gets issues to be
resolved, eventually. In the specific case: if the bug has already been
confirmed, there is nothing else to be done for the review.

Regards,
Martin

From skippy.hammond at gmail.com  Tue Mar 24 23:35:46 2009
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Wed, 25 Mar 2009 09:35:46 +1100
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <49C943EB.90506@v.loewis.de>
References: <loom.20090324T134451-902@post.gmane.org>	<m2zlfa241x.fsf@valheru.db3l.homeip.net>	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<49C943EB.90506@v.loewis.de>
Message-ID: <49C96042.2020207@gmail.com>

On 25/03/2009 7:34 AM, "Martin v. L?wis" wrote:
> I don't quite remember the -n flag, but I believe that Kristjan
> just removed all that stuff, ie. there is now no way to block
> CRT assertions anymore.
>
>> I
>> wasn't paying close attention, so maybe there's some other mechanism
>> in place at this point?
>
> Unfortunately, not. There was a long debate, I lost, and Kristjan
> removed all the changes to support this setup, claiming that it is
> good to get assertions :-(

Note that I previously opened http://bugs.python.org/issue5116 with a 
patch to enable this to be controlled from Python via the msvcrt module. 
  This would enable the test suite to disable assertions for its entire run.

Mark

From solipsis at pitrou.net  Tue Mar 24 23:45:47 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 24 Mar 2009 22:45:47 +0000 (UTC)
Subject: [Python-Dev] Test failures under Windows?
References: <loom.20090324T134451-902@post.gmane.org>	<m2zlfa241x.fsf@valheru.db3l.homeip.net>	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<49C943EB.90506@v.loewis.de> <49C96042.2020207@gmail.com>
Message-ID: <loom.20090324T224457-354@post.gmane.org>

Mark Hammond <skippy.hammond <at> gmail.com> writes:
> 
> Note that I previously opened http://bugs.python.org/issue5116 with a 
> patch to enable this to be controlled from Python via the msvcrt module. 
>   This would enable the test suite to disable assertions for its entire run.

We certainly don't want to disable assertions during the test suite if it
silences bugs which can crash the user's interpreter.

Regards

Antoine.



From martin at v.loewis.de  Tue Mar 24 23:47:32 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Mar 2009 23:47:32 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C95E9D.8070604@gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>	<49C87004.2030807@holdenweb.com>	<49C88503.2030902@v.loewis.de>	<49C886EF.80203@v.loewis.de>	<49C8C9B3.3070403@holdenweb.com>	<49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com>
Message-ID: <49C96304.7050207@v.loewis.de>

> * The creation of an *installer* is something quite different.  An
> installer for a py2exe based tool also doesn't need dependency
> management.

Right. I wasn't really talking about py2exe (anymore), but about
installers for libraries.

> An installer for a pure-python package that made no attempt
> to bundle dependencies might be nice, but I don't quite see how that
> falls outside the scope of distutils/setuptools/etc.  In other words, I
> don't see why the installer can't bootstrap the 'normal' dependency
> management which would be used if the package was installed any other
> way or on other platforms.

Perhaps that could be a solution. However, in package management
systems that solve this properly, you also have proper uninstallation,
which includes:
- uninstallation is rejected if packages still depend on the
  to-be-removed package (or else offers to remove the relying packages
  as well)
- uninstallation reference-counts, causing an automatically-installed
  package to be uninstalled if it is no longer needed, or else offers
  to compute-then-uninstall all packages which are no longer needed.
The .exe/.msi installers do support uninstallation, but, alas, no
dependency management.

> * distutils already has the ability to create Windows installer
> executables for pure-python apps/libs.  I agree it would be nice if it
> was an MSI but that is an implementation detail rather than
> implementation requirement.

Right - plus, distutils also supports creating MSIs.

>  How were Mike's packages fundamentally
> different than that?

Dunno.

Martin

From curt at hagenlocher.org  Tue Mar 24 23:50:31 2009
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Tue, 24 Mar 2009 15:50:31 -0700
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <loom.20090324T224457-354@post.gmane.org>
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net> <49C943EB.90506@v.loewis.de>
	<49C96042.2020207@gmail.com> <loom.20090324T224457-354@post.gmane.org>
Message-ID: <d2155e360903241550q287d00c7ga37efdfef8baa22c@mail.gmail.com>

On Tue, Mar 24, 2009 at 3:45 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Mark Hammond <skippy.hammond <at> gmail.com> writes:
>>
>> Note that I previously opened http://bugs.python.org/issue5116 with a
>> patch to enable this to be controlled from Python via the msvcrt module.
>> ? This would enable the test suite to disable assertions for its entire run.
>
> We certainly don't want to disable assertions during the test suite if it
> silences bugs which can crash the user's interpreter.

I assume the goal would be to write them to stdout or stderr instead
of hanging the process by throwing up a dialog box.  For that matter,
you could use CrtSetReportFile to redirect them to any given file and
then assert at the end of the test run that the file is empty.

--
Curt Hagenlocher
curt at hagenlocher.org

From martin at v.loewis.de  Tue Mar 24 23:51:45 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Mar 2009 23:51:45 +0100
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <49C96042.2020207@gmail.com>
References: <loom.20090324T134451-902@post.gmane.org>	<m2zlfa241x.fsf@valheru.db3l.homeip.net>	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<49C943EB.90506@v.loewis.de> <49C96042.2020207@gmail.com>
Message-ID: <49C96401.9020904@v.loewis.de>

> Note that I previously opened http://bugs.python.org/issue5116 with a
> patch to enable this to be controlled from Python via the msvcrt module.
>  This would enable the test suite to disable assertions for its entire run.

This patch is fine with me. It might need some documentation, though.

However, I would still be unhappy if Python on Windows would pop up
might pop up windows in cases where you get a proper exception on Unix.
So I'd rather see these bugs fixed instead of silencing them.

Regards,
Martin

From greg.ewing at canterbury.ac.nz  Wed Mar 25 00:02:56 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 25 Mar 2009 11:02:56 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
Message-ID: <49C966A0.6090401@canterbury.ac.nz>

Guido van Rossum wrote:

> I really don't like to have things whose semantics is
> defined in terms of code inlining -- even if you don't mean that as
> the formal semantics but just as a mnemonic hint.

Think about it the other way around, then. Take any chunk
of code containing a yield, factor it out into a separate
function (using whatever techniques you would normally
use when performing such a refactoring to deal with
references to variables in the surrounding scope) and
call it using yield-from. The result should be the same
as the original unfactored code.

That's the fundamental reason behind all of this -- to
make such refactorings possible in a straightforward way.

> What happened to the first-order approximation
> 
>   "yield from X" means roughly the same as "for _x in X: yield x"

Everybody's reaction to that when it's been suggested
before has been "that's trivial, why bother?" So I've
been trying to present it in a way that doesn't make it
appear so trivial.

Also, my understanding is that a PEP is not meant to be
a tutorial for naive users, but a document for communicating
ideas between core Python developers, who are presumably
savvy enough not to need such watered-down material.

But I'll be happy to add a paragraph about this at the
beginning if you think it would help.

> But the exact
> semantics in the light of try/except or try/finally blocks on the
> stack are incredibly (perhaps impossibly) tricky to get right -- and
> it probably doesn't matter all that much what exactly happens as long
> as it's specified in sufficient detail that different implementations
> behave the same way (apart from obvious GC differences, alas).

This is part of the reason I've been emphasising the
inlining principle. When pondering what should happen
in such cases, I've been able to think to myself
"What would happen if the subgenerator were inlined?"
Most of the time that makes the answer fairly obvious,
at least in the case where the subiterator is another
generator. Then it's a matter of generalising it to
other iterators.

-- 
Greg

From db3l.net at gmail.com  Wed Mar 25 00:05:22 2009
From: db3l.net at gmail.com (David Bolen)
Date: Tue, 24 Mar 2009 19:05:22 -0400
Subject: [Python-Dev] Test failures under Windows?
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<d2155e360903241213v49e3b2ccr3bcdc7facbe36a@mail.gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F0519BB4A9F@exchis.ccp.ad.local>
Message-ID: <m2ljqu1pzh.fsf@valheru.db3l.homeip.net>

Kristj?n Valur J?nsson <kristjan at ccpgames.com> writes:

> Now, I know that this msvc behaviour can be disabled, but it was
> decided that it was not appropriate to meddle with runtime flags of
> the whole process for python.

I must have missed that discussion, but I can't see what the problem
is if such an override only occurs during the buildbot non-interactive
test runs?  Isn't that what using -n in the buildbot test.bat did?

It's not like it hides the error, as I believe the process will still
terminate if it would have otherwise, thus causing a test failure, but
the error itself just goes to stderr (or wherever) without popping up
a blocking UI dialog.

In that way it's no different than disabling the Windows (non-CRT)
dialog boxes, which I do in my buildslave for all builds (via code in
buildbot).  The process still fails, but it doesn't hang waiting for
UI interaction.

I can't see the downside.

-- David


From mal at egenix.com  Wed Mar 25 00:14:55 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 25 Mar 2009 00:14:55 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C96304.7050207@v.loewis.de>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>	<49C87004.2030807@holdenweb.com>	<49C88503.2030902@v.loewis.de>	<49C886EF.80203@v.loewis.de>	<49C8C9B3.3070403@holdenweb.com>	<49C939BA.8040206@v.loewis.de>	<49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de>
Message-ID: <49C9696F.2040408@egenix.com>

On 2009-03-24 23:47, Martin v. L?wis wrote:
>> An installer for a pure-python package that made no attempt
>> to bundle dependencies might be nice, but I don't quite see how that
>> falls outside the scope of distutils/setuptools/etc.  In other words, I
>> don't see why the installer can't bootstrap the 'normal' dependency
>> management which would be used if the package was installed any other
>> way or on other platforms.
> 
> Perhaps that could be a solution. However, in package management
> systems that solve this properly, you also have proper uninstallation,
> which includes:
> - uninstallation is rejected if packages still depend on the
>   to-be-removed package (or else offers to remove the relying packages
>   as well)
> - uninstallation reference-counts, causing an automatically-installed
>   package to be uninstalled if it is no longer needed, or else offers
>   to compute-then-uninstall all packages which are no longer needed.
> The .exe/.msi installers do support uninstallation, but, alas, no
> dependency management.

Question is: who really needs such dependency management ?

* It may be helpful to developers who wrap up 3rd party code for
an application (e.g. Miro).

* It may also help users that want to install a few plugins for an
already installed application (e.g. Zope).

* It will help users who use OSes that rely on software management
tools to keep the initial distribution size small and prefer
sharing over application isolation (e.g. Ubuntu).

* It won't simplify things if such a system gets in the way of
how the users or developer usually works or wants to work in
a project.

* It introduces dependencies on network resources that may
potentially not be trusted.

* If the package dependencies are not managed with lots of quality
assurance, it can easily ruin your complete installation or
simply prevent you from installing two sets of packages at
the same time.

There are both lots of reasons for wanting dependency checking
and against such dependency checking.

As a result, there is no definite answer as to whether it's
good or bad and there is no single system that would satisfy
all users/developers.

Instead, there needs to be freedom of choice and
distutils provides this freedom of choice by allowing you
to ship .exes, .msi installers, binary drop-in ZIP archives,
RPM packages, Debian packages, etc. etc.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 24 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2009-03-19: Released mxODBC.Connect 1.0.1      http://python.egenix.com/

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ajaksu at gmail.com  Wed Mar 25 00:22:48 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 24 Mar 2009 20:22:48 -0300
Subject: [Python-Dev] tracker status options
In-Reply-To: <Pine.LNX.4.64.0903232156330.31981@kimball.webabinitio.net>
References: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com> 
	<49C1EA9E.3040603@v.loewis.de>
	<bbaeab100903191414o1710bd87x3d4c9a713c63c832@mail.gmail.com> 
	<49C2D040.4080609@v.loewis.de>
	<2d75d7660903191653v65640c90pc4df8ddcf60343f1@mail.gmail.com> 
	<43c8685c0903222220h6ecc841ar27ce66adf3c3d064@mail.gmail.com> 
	<Pine.LNX.4.64.0903232156330.31981@kimball.webabinitio.net>
Message-ID: <2d75d7660903241622o13ac9f34j251fb887684448c9@mail.gmail.com>

R. David Murray wrote:
> I understood from posts I saw go by earlier from Daniel that 'pending'
> meant 'close pending unless there is feedback to the contrary' (and I
> just used it that way). ?It sounds like that is indeed correct but not
> universally known, and thus I would suggest that at a minimum this status
> be changed to 'close pending' to make it clearer.

Aha, at least one post I feel I can reply to. Sort of. I do have a set
of triaging principles and beliefs about each issue property, but if
you explained to me that 'closed' actually requires unsetting
versions, priority and keywords, I'd buy it and would add that to my
set of triaging beliefs. OK, maybe I'd ask for an explanation :)

ISTM that each tracker user understands different things from each
option, each assignee reacts differently to changes in their issues. I
kinda try to understand what people mean with the options, not what
the options should mean.

If I set issues to pending and simultaneously declare my
interpretation of pending, in part that's because I expect different
people to see different meanings in the status change. I've seen
issues in the top 10 most active that lived in a pending status for
most of their existence.

I'm having trouble posting in these threads because I actually can't
tell whether adding a stage, removing a component or renaming a status
is better or worse than the status quo.

I feel we should make the tracker more useful for core developers,
volunteers and end-users. I also think having a clear workflow helps a
lot. Yet, I'd rather have a tracker that allowed users with different
interpretations to work as they feel most comfortable than one that
requires a change of assumptions.

On my first triaging sprint Brett asked whether a new stage would make
things better and Martin said he could add any new components that
could make triaging easier. I told them I was unsure about it, that I
might have requests for new options after some thinking, but that I
had a couple of RFEs. It's been over a month, but new options still
make me uneasy.

Let me hit send before I conclude (again) this adds nothing to the
discussion and discard it :)

Regards,
Daniel

From guido at python.org  Wed Mar 25 00:23:34 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Mar 2009 16:23:34 -0700
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49C966A0.6090401@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
Message-ID: <ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>

On Tue, Mar 24, 2009 at 4:02 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>> I really don't like to have things whose semantics is
>> defined in terms of code inlining -- even if you don't mean that as
>> the formal semantics but just as a mnemonic hint.
>
> Think about it the other way around, then. Take any chunk
> of code containing a yield, factor it out into a separate
> function (using whatever techniques you would normally
> use when performing such a refactoring to deal with
> references to variables in the surrounding scope) and
> call it using yield-from. The result should be the same
> as the original unfactored code.

The way I think of it, that refactoring has nothing to do with
yield-from. It's not just variable references -- I used "scope" as a
shorthand for everything that can be done within a function body,
including control flow: try/except/finally,
continue/break/raise/return.

> That's the fundamental reason behind all of this -- to
> make such refactorings possible in a straightforward way.

Well, it solves one particular detail.

>> What happened to the first-order approximation
>>
>> ?"yield from X" means roughly the same as "for _x in X: yield x"
>
> Everybody's reaction to that when it's been suggested
> before has been "that's trivial, why bother?" So I've
> been trying to present it in a way that doesn't make it
> appear so trivial.

Maybe you're confusing motivation with explanation? That feedback
seems to tell me that the *motivation* needs more work; but IMO the
*explanation* should start with this simple model and then expand upon
the edge cases.

> Also, my understanding is that a PEP is not meant to be
> a tutorial for naive users, but a document for communicating
> ideas between core Python developers, who are presumably
> savvy enough not to need such watered-down material.

Not quite. PEPs aren't *just* for core developers -- they are also for
communicating to (savvy) developers outside the core group. A good PEP
needs to summarize both the motivation and specification concisely so
prospective readers can quickly determine what it is about, and
whether they care.

> But I'll be happy to add a paragraph about this at the
> beginning if you think it would help.
>
>> But the exact
>> semantics in the light of try/except or try/finally blocks on the
>> stack are incredibly (perhaps impossibly) tricky to get right -- and
>> it probably doesn't matter all that much what exactly happens as long
>> as it's specified in sufficient detail that different implementations
>> behave the same way (apart from obvious GC differences, alas).
>
> This is part of the reason I've been emphasising the
> inlining principle. When pondering what should happen
> in such cases, I've been able to think to myself
> "What would happen if the subgenerator were inlined?"
> Most of the time that makes the answer fairly obvious,
> at least in the case where the subiterator is another
> generator. Then it's a matter of generalising it to
> other iterators.

This is a good way of thinking about use cases, because it helps
deciding how the edge cases should be specified that the simplest
model (my one-liner above) doesn't answer in a useful way. But it
should not be confused with an explanation of the semantics.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ajaksu at gmail.com  Wed Mar 25 00:27:22 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 24 Mar 2009 20:27:22 -0300
Subject: [Python-Dev] Roundup / Python Tracker enhancements
Message-ID: <2d75d7660903241627x6f3d22fek81314a02245c2843@mail.gmail.com>

  This proposal has two main goals: making the Python bug tracker more
efficient for core developers and improving Roundup in areas that
don't directly concern the PSF trackers. Most of the code would land
in Roundup's repositories, but many instance-level changes would be
specific to our tracker.

Justification
 Python Tracker
  Some feedback is necessary to find out unvoiced problems,
shortcomings and RFEs core developers have in mind regarding the
tracker. From my work on old issues and with a couple of meta-tracker
tickets, I can highlight a few pain points.
  Quoting myself:
    [A] relevant time-sink consists of inadequacies of the current [tracker]
    interface. Many search features are hard to use or notice, among them
    date spans and entering multiple inputs as lists. Other search features
    are lacking, mostly simple boolean relations, e.g. those including more
    than one keyword, full search terms, type, component, etc. Besides
    searching, the lack of interfaces (and backend support) for selecting
    and working with multiple issues tends to waste considerable amounts
    of time.
  To that, I can add some RFEs and bugs with the email interface,
support for third-party authentication and  many possible UI
improvements. See a list of these below.
 Roundup
  The project has suffered from lack of developer time for a while and
has grown a list of important improvements that would help it's
adoption and revitalize the community. Correct handling of different
charsets, DB performance and alternative indexers and templating
engines are examples of often requested features. Making Roundup a
better tool for developers and end-users, with a productive community
can only benefit the PSF.

Pre-Requisites and Status
  We have been working in this area recently, so many ideas are well
developed, with a few fleshed out in great detail.
  Regarding the Python tracker, Martin is guiding my work on simple
tracker issues, Brett gave us documents and many discussions about the
issue handling workflow, Tennessee Leeuwenburg and R. David Murray are
willing to triage issues and contribute to a better workflow.
  On the Roundup side, Stefan Seefeld and Richard Jones are leading
new wave of features and fixes, with contributions from Tobias Herp,
Bernhard Reiter and others.

Student
  With the caveat that I'm probably unable to assess this objectively,
a student should be comfortable with Python and have a grasp of
general bug/issue handling procedures. Familiarity with Roundup is a
huge plus, but the code is newbie-friendly.

Mentors
  Under the condition that enough attention is given to Roundup
itself, Stefan Seefeld is willing to mentor this proposal. I'll be
available to help mentors and the student in many tasks. Martin von
L?wis would be able to discuss many important parts of the work.

Deliverables:
  Patches for upstream Roundup and for the Python instance.


For a given proposal, a major goal should be declared, e.g. 'improve
and clean up the infrastructure' or 'work on integration with third
party services and tools'. Under that goal,  task groups like
'refactor the different frontends' or 'implement VCS integration'
should organize the developement effort in somewhat isolated steps.

Some suggested topics, with examples:
* means the idea is well developed and suitable for implementation
# means a simple task or group of tasks, easy to implement

  Clean up and improve the base infrastructure
    Audit and fix any potential security issues *
    Fix support of diverse character encodings

  Performance improvements
    Reduce the number of RDBMS calls
    Improve caching across requests

  Improve end-user UI and features
    Enhance search abilities, better indexing
        Boolean operators
    Make forms easier to use and understand
        Support upload of multiple files in one request *#
    Support for third-party authentication, e.g. OpenID *

  Add developer-focused features to Roundup
    Support for milestones, tasks, workgroups
    Integration with code review tools
        Integration with Rietveld *
    Update/implement [D]VCS integration
        SVN integration
        Mercurial integration
    Interface to continuous integration tools
        Buildbot

  Administration enhancements
    Mass-update/batch-editing support *#
    UI for Class administration

Discussion of more detailed tasks and open issues on roundup-devel[5]
is highly encouraged.

[1] https://lists.sourceforge.net/lists/listinfo/roundup-devel

From python at rcn.com  Wed Mar 25 00:38:54 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 24 Mar 2009 16:38:54 -0700
Subject: [Python-Dev] MutableSet and Set named methods
Message-ID: <9467B8320BD04137973283CD2A8A9F83@RaymondLaptop1>

Does anyone remember the reason that most of the named methods were omitted from the ABC for mutablesets and sets?  The update() 
method in particular would be nice to have.


RAymond 


From guido at python.org  Wed Mar 25 00:44:31 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Mar 2009 16:44:31 -0700
Subject: [Python-Dev] MutableSet and Set named methods
In-Reply-To: <9467B8320BD04137973283CD2A8A9F83@RaymondLaptop1>
References: <9467B8320BD04137973283CD2A8A9F83@RaymondLaptop1>
Message-ID: <ca471dc20903241644v7d78a20dr3dda58b5235f6a64@mail.gmail.com>

On Tue, Mar 24, 2009 at 4:38 PM, Raymond Hettinger <python at rcn.com> wrote:
> Does anyone remember the reason that most of the named methods were omitted
> from the ABC for mutablesets and sets? ?The update() method in particular
> would be nice to have.

Because it's the same as +=, and if you want it as a callable, you can
use operator.__ior__? :-)

(And frozen sets don't have it.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From v+python at g.nevcal.com  Wed Mar 25 01:02:30 2009
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Tue, 24 Mar 2009 17:02:30 -0700
Subject: [Python-Dev] GSoC: Replace MS Windows Console with Unicode UI
In-Reply-To: <ee9557410903241016h2f1e7b1cp4afdf96afd7f293f@mail.gmail.com>
References: <2d75d7660903221530g2ad1590u32fb2843051cc930@mail.gmail.com>	<gq70q2$hul$1@ger.gmane.org>
	<49C70C6A.7070106@g.nevcal.com>	<gq8mu2$mfl$1@ger.gmane.org>
	<49C7FF1F.6080707@g.nevcal.com>
	<ee9557410903241016h2f1e7b1cp4afdf96afd7f293f@mail.gmail.com>
Message-ID: <49C97496.4070609@g.nevcal.com>

On approximately 3/24/2009 10:16 AM, came the following characters from 
the keyboard of INADA Naoki:
> Hi. I'm Japanese and non-ascii charactor user. (cp932)
> 
> We have to use "IME" to input non-ascii charactor in Windows.
> When "> chcp 65001" in cmd.exe, we cannot use IME on cmd.exe.
> 
> So setting codepage to 65001 make output universal but make input ascii-only.
> Sit!!!
> 
> I hope PyQtShell <http://code.google.com/p/pyqtshell/> become good
> IDLE alternative.


Thanks for the feedback.

So at least one version of the code I posted shows that 
programmatically, the code page can be set differently for input and 
output, although the last version brought both to 65001.  It seems that 
the chcp 65001 always does both.  If the IME only works for cp932, then 
leave input at cp932, and set output to 65001?

I have no idea if that could be a solution for you, but I would be 
interested in your results if you find that it is, or isn't, as that 
would add to the collective knowledge base about the subject.  This is 
idea 2, below, where I tried to cover the solution space more broadly.

Looking briefly at the definition of cp932, it seems that it covers most 
of the Unicode characters... so perhaps any or several of the following 
could happen:

1) the IME could be converted to produce UTF-8 instead of cp932, 
allowing use of 65001 for input and output
2) the split code page could be used to avoid the conversion of Unicode 
to cp932 for output.
3) Unicode could be converted to cp932 for output, allowing use of cp932 
for both input and output.

These are listed in the order of increased overhead for character handling.

Perhaps you could enlighten us all as to the issues with each of these 
ideas.

I realize the IME exists today, and is likely coded to use cp932, and 
that it would take some work to convert it to produce Unicode.  However, 
there seems to be a straightforward conversion chart between cp932 and 
Unicode at Wikipedia, so perhaps that isn't a huge effort.

It seems that the long term goal of having all software speak Unicode 
would increase the efficiency of all software when dealing with 
multi-lingual issues, as a common solution can be applied universally, 
rather than re-inventing solutions that only work for particular code pages.

But I'm not fully aware of whether or not the design or implementation 
of Unicode precludes universal solutions: I have heard rumors that 
certain characters must be interpreted differently in different locale 
contexts, which seems to be counter to the "one solution fits all" 
possibility.

-- 
Glenn -- http://nevcal.com/
===========================
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

From aahz at pythoncraft.com  Wed Mar 25 01:08:23 2009
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 24 Mar 2009 17:08:23 -0700
Subject: [Python-Dev] Roundup / Python Tracker enhancements
In-Reply-To: <2d75d7660903241627x6f3d22fek81314a02245c2843@mail.gmail.com>
References: <2d75d7660903241627x6f3d22fek81314a02245c2843@mail.gmail.com>
Message-ID: <20090325000823.GA18770@panix.com>

On Tue, Mar 24, 2009, Daniel (ajax) Diniz wrote:
>
>   This proposal has two main goals: making the Python bug tracker more
> efficient for core developers and improving Roundup in areas that
> don't directly concern the PSF trackers. Most of the code would land
> in Roundup's repositories, but many instance-level changes would be
> specific to our tracker.

Is this for GSoC?  If yes, please make sure to include that tag in the
Subject: line to make it easier to track.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"At Resolver we've found it useful to short-circuit any doubt and just
refer to comments in code as 'lies'. :-)"
--Michael Foord paraphrases Christian Muirhead on python-dev, 2009-3-22

From ajaksu at gmail.com  Wed Mar 25 01:19:04 2009
From: ajaksu at gmail.com (Daniel (ajax) Diniz)
Date: Tue, 24 Mar 2009 21:19:04 -0300
Subject: [Python-Dev] GSoC 2009: Roundup / Python Tracker enhancements
Message-ID: <2d75d7660903241719t7a67c648ob93ac4bc7621ca1c@mail.gmail.com>

Aahz wrote:
> Is this for GSoC? ?If yes, please make sure to include that tag in the
> Subject: line to make it easier to track.

Oops, makes a lot of sense :)

Daniel

From guido at python.org  Wed Mar 25 02:26:56 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Mar 2009 18:26:56 -0700
Subject: [Python-Dev] In case I seem to disappear...
Message-ID: <ca471dc20903241826r1c022313y6c3acd9030d55c6a@mail.gmail.com>

I've received several emails recently indicating that someone is
trying to get me unsubscribed from python-dev. I'm sure they're just
pranksters, but just in case they succeed, if I seem to be absent for
a really long time, let me know. After all mailman passwords are not
secure. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ggpolo at gmail.com  Wed Mar 25 03:01:41 2009
From: ggpolo at gmail.com (Guilherme Polo)
Date: Tue, 24 Mar 2009 23:01:41 -0300
Subject: [Python-Dev] Non-Core project: IDLE
In-Reply-To: <gq93lh$368$1@ger.gmane.org>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com> 
	<Pine.LNX.4.64.0903181706310.26433@kimball.webabinitio.net> 
	<gprp3t$icr$2@ger.gmane.org> <gps1in$e1i$1@ger.gmane.org> 
	<ac2200130903231207x16d30091g2347abb95f72efce@mail.gmail.com> 
	<gq93lh$368$1@ger.gmane.org>
Message-ID: <ac2200130903241901x2f61ffc2oadb64fea7f09a8df@mail.gmail.com>

On Mon, Mar 23, 2009 at 7:52 PM, Scott David Daniels
<Scott.Daniels at acm.org> wrote:
> Guilherme Polo wrote:
>>
>> On Wed, Mar 18, 2009 at 8:53 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>>>
>>> IDLE needs lots of attention -- more than any one experienced person is
>>> likely to have
>>
>> I'm willing to step up as a student for this but I still have to write
>> a good proposal for it.
>> My actual concern is about mentor availability, is someone around
>> interested on being an IDLE mentor ?
>
> You might want to talk to Bruce Sherwood, as VPython suffered through
> a bunch of Idle problems. ?I got the impression he had a person or two
> who were his Idle experts (so I am thinking they might be the mentors
> you are looking for). ?In any case, even a charter of "unit tests to 50%
> coverage" of Idle would be a huge improvement.
>

Thanks for the direction Scott. I have talked with Bruce yesterday and
now again, he included a list of issues that is considered important
to be fixed but unfortunately several of them are related to mac only
and I won't be able to properly address them.

He also pointed me to the person that worked on some issues regarding
IDLE and VPython, waiting now to hear if he is interested on gsoc.

> I've run after specific bugs in Idle, but don't really know the lay of
> the land.
>
>
> --Scott David Daniels
> Scott.Daniels at Acm.Org

Thanks,


-- 
-- Guilherme H. Polo Goncalves

From rrr at ronadam.com  Wed Mar 25 03:11:42 2009
From: rrr at ronadam.com (Ron Adam)
Date: Tue, 24 Mar 2009 21:11:42 -0500
Subject: [Python-Dev] tracker status options
In-Reply-To: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>
References: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>
Message-ID: <49C992DE.7080209@ronadam.com>



Tennessee Leeuwenburg wrote:
> Hi all,
> 
> I'm continuing to (slowly) work through issues. I have been looking 
> particularly at a lot of the open issues regarding strftime.
> 
> It would be great to put in some of those extra status options that were 
> discussed recently...
> 
>    "Open/New"
>    "Needs help / Chatting"
>    "Under development"
>    "Pending feedback"
>    "Closed"
> 
> For everyone's reference, after some debate, the above list of status 
> options was where the conversation finished. So the two options "Needs 
> help / chatting" and "Under development" would need to be added. Issues 
> that are "Under development" should revert to "Needs help / chatting" 
> after a month of inactivity.
> 
> Would it be possible to get these put in? Maybe I'm one of a small 
> number of people who are nibbling at the bottom end of the bugs, but I 
> find it somewhat frustrating not to be able to classify the issues that 
> I find into "needs help / chatting" vs "under development" to help make 
> more sense of the search results.
> 
> Cheers,
> -T



Maybe a flow chart can help unambiguize things a bit.  What follows is 
maybe one starting place.  I tried to separate steps that might be done by 
users from developers so that there is a clear order to the process.  This 
seems to follow the general way that python issues are resolved from what 
I've observed.

While this chart separates the descriptive/brain storming and patch making 
parts of an issue, in actuality, a patch (tests, fix, and docs) can be 
developed partially or fully in the discussion faze for the purpose of 
helping the discussion, then later reused and fine tuned in the patch 
making faze.

Hope this is helpful and doesn't get too mangled in sending.

     (new issue)
         |
         v

     [discuss issue] <-------------------------+
         |                                     |
         v                                     |
                                               |
     {request issue review}                    |
         |                                     |
         v                                     |
                                               |
     <invalid issue ?> (yes)--> [close issue]  |
        (no)                        |          |
         |                          v          |
         |                       (closed)      |
         v                                     |
                                               |
     <valid issue ?> (no)----------------------+
       (yes)
         |
         v

     [develop patch]  <------+
         |                   |
         v                   |
                             |
     {request patch review}  |
         |                   |
         v                   |
                             |
     <patch ready ?> (no) ---+
       (yes)
         |
         v

     [apply patch]
         |
         v

     [close issue]
         |
         v

     (closed)




From rrr at ronadam.com  Wed Mar 25 03:11:42 2009
From: rrr at ronadam.com (Ron Adam)
Date: Tue, 24 Mar 2009 21:11:42 -0500
Subject: [Python-Dev] tracker status options
In-Reply-To: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>
References: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>
Message-ID: <49C992DE.7080209@ronadam.com>



Tennessee Leeuwenburg wrote:
> Hi all,
> 
> I'm continuing to (slowly) work through issues. I have been looking 
> particularly at a lot of the open issues regarding strftime.
> 
> It would be great to put in some of those extra status options that were 
> discussed recently...
> 
>    "Open/New"
>    "Needs help / Chatting"
>    "Under development"
>    "Pending feedback"
>    "Closed"
> 
> For everyone's reference, after some debate, the above list of status 
> options was where the conversation finished. So the two options "Needs 
> help / chatting" and "Under development" would need to be added. Issues 
> that are "Under development" should revert to "Needs help / chatting" 
> after a month of inactivity.
> 
> Would it be possible to get these put in? Maybe I'm one of a small 
> number of people who are nibbling at the bottom end of the bugs, but I 
> find it somewhat frustrating not to be able to classify the issues that 
> I find into "needs help / chatting" vs "under development" to help make 
> more sense of the search results.
> 
> Cheers,
> -T



Maybe a flow chart can help unambiguize things a bit.  What follows is 
maybe one starting place.  I tried to separate steps that might be done by 
users from developers so that there is a clear order to the process.  This 
seems to follow the general way that python issues are resolved from what 
I've observed.

While this chart separates the descriptive/brain storming and patch making 
parts of an issue, in actuality, a patch (tests, fix, and docs) can be 
developed partially or fully in the discussion faze for the purpose of 
helping the discussion, then later reused and fine tuned in the patch 
making faze.

Hope this is helpful and doesn't get too mangled in sending.

     (new issue)
         |
         v

     [discuss issue] <-------------------------+
         |                                     |
         v                                     |
                                               |
     {request issue review}                    |
         |                                     |
         v                                     |
                                               |
     <invalid issue ?> (yes)--> [close issue]  |
        (no)                        |          |
         |                          v          |
         |                       (closed)      |
         v                                     |
                                               |
     <valid issue ?> (no)----------------------+
       (yes)
         |
         v

     [develop patch]  <------+
         |                   |
         v                   |
                             |
     {request patch review}  |
         |                   |
         v                   |
                             |
     <patch ready ?> (no) ---+
       (yes)
         |
         v

     [apply patch]
         |
         v

     [close issue]
         |
         v

     (closed)




From ggpolo at gmail.com  Wed Mar 25 03:11:54 2009
From: ggpolo at gmail.com (Guilherme Polo)
Date: Tue, 24 Mar 2009 23:11:54 -0300
Subject: [Python-Dev] Non-Core project: IDLE
In-Reply-To: <200903240726.34577.list@qtrac.plus.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com> 
	<gq8s28$95b$1@ger.gmane.org>
	<ac2200130903231517k4115b130ga6e04f675274e56d@mail.gmail.com> 
	<200903240726.34577.list@qtrac.plus.com>
Message-ID: <ac2200130903241911l30c096bbjefef87c7c30a9edf@mail.gmail.com>

On Tue, Mar 24, 2009 at 4:26 AM, Mark Summerfield <list at qtrac.plus.com> wrote:
> On 2009-03-23, Guilherme Polo wrote:
>> On Mon, Mar 23, 2009 at 5:39 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> > Guilherme Polo wrote:
>> >> On Wed, Mar 18, 2009 at 8:53 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> >>> IDLE needs lots of attention -- more than any one experienced person is
>> >>> likely to have
>> >>
>> >> I'm willing to step up as a student for this but I still have to write
>> >> a good proposal for it.
>> >> My actual concern is about mentor availability, is someone around
>> >> interested on being an IDLE mentor ?
>> >
>> > If I could, I would, and would have said so. ?But I have only read about
>> > tk and have not actually used it. ?If I did decide to dive into it, you'd
>> > be mentoring me ;-). ?What I can and would do is give ideas for changes,
>> > read and comment on a proposal, and user test patched versions.
>>
>> That is very nice Terry. Do you have some specific ideas that you want
>> to share publicly (or in private) about IDLE ? Your expectations about
>> what should be addressed first, or areas that should be improved.. you
>> know, anything.
>
> I have one suggestion that I think might be widely appreciated:
>
> Add somewhere in the configuration dialog when users can enter a block
> of Python code to be executed at startup and whenever Restart Shell is
> executed.
>
> Use case: for people who use IDLE for calculations/experiments they
> might like to always have certain module imported. For me personally, it
> would be:
>
> ? ?import os
> ? ?import re
> ? ?import sys
> ? ?from math import *
>
> but of course the whole point is that people can write any code they
> like. (Some people might want to do various from __future__ imports in
> Python 2.6 to get various Python 3 features for example.)
>
> I know that you can use the -c option, but that only works at startup,
> not every time you Restart Shell.
>

Looks like a good suggestion to me, Mark.
I would recommend adding it as a feature request on the typical place
(bugs.python.org) because although I could just go and do it, I
believe you are aware that new features in IDLE are subject to
approval or disapproval by other members involved with IDLE. Hope you
understand my position.

> [snip]
>
> --
> Mark Summerfield, Qtrac Ltd, www.qtrac.eu
> ? ?C++, Python, Qt, PyQt - training and consultancy
> ? ? ? ?"Rapid GUI Programming with Python and Qt" - ISBN 0132354187
>
>

Regards,


-- 
-- Guilherme H. Polo Goncalves

From steve at holdenweb.com  Wed Mar 25 03:22:59 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 24 Mar 2009 22:22:59 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49C966A0.6090401@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>	<loom.20090321T122452-862@post.gmane.org>	<49C5620A.4080908@canterbury.ac.nz>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
Message-ID: <gqc4id$ud8$1@ger.gmane.org>

Greg Ewing wrote:
> Guido van Rossum wrote:
> 
>> I really don't like to have things whose semantics is
>> defined in terms of code inlining -- even if you don't mean that as
>> the formal semantics but just as a mnemonic hint.
> 
> Think about it the other way around, then. Take any chunk
> of code containing a yield, factor it out into a separate
> function (using whatever techniques you would normally
> use when performing such a refactoring to deal with
> references to variables in the surrounding scope) and
> call it using yield-from. The result should be the same
> as the original unfactored code.
> 
> That's the fundamental reason behind all of this -- to
> make such refactorings possible in a straightforward way.
> 
>> What happened to the first-order approximation
>>
>>   "yield from X" means roughly the same as "for _x in X: yield x"
> 
> Everybody's reaction to that when it's been suggested
> before has been "that's trivial, why bother?" So I've
> been trying to present it in a way that doesn't make it
> appear so trivial.
> 
There is one non-trivial extension that I've been chewing over for a
while. What if you want to yield not the values from the generator but
some function of those values? The present proposal appears to have no
way to specify that. What about extending the syntax somewhat to

  yield expr for x from X

The idea is that x should be a a bound variable in expr, but the "expr
for x" could be optional to yield the existing proposal as a degenerate
case.

> Also [...]

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From tleeuwenburg at gmail.com  Wed Mar 25 03:34:05 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Wed, 25 Mar 2009 13:34:05 +1100
Subject: [Python-Dev] tracker status options
In-Reply-To: <49C992DE.7080209@ronadam.com>
References: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>
	<49C992DE.7080209@ronadam.com>
Message-ID: <43c8685c0903241934y569b91aas92612e702fdf256b@mail.gmail.com>

Hi Ron,

Good flowchart.

Cheers,
-T

On Wed, Mar 25, 2009 at 1:11 PM, Ron Adam <rrr at ronadam.com> wrote:

>
>
> Tennessee Leeuwenburg wrote:
>
>> Hi all,
>>
>> I'm continuing to (slowly) work through issues. I have been looking
>> particularly at a lot of the open issues regarding strftime.
>>
>> It would be great to put in some of those extra status options that were
>> discussed recently...
>>
>>   "Open/New"
>>   "Needs help / Chatting"
>>   "Under development"
>>   "Pending feedback"
>>   "Closed"
>>
>> For everyone's reference, after some debate, the above list of status
>> options was where the conversation finished. So the two options "Needs help
>> / chatting" and "Under development" would need to be added. Issues that are
>> "Under development" should revert to "Needs help / chatting" after a month
>> of inactivity.
>>
>> Would it be possible to get these put in? Maybe I'm one of a small number
>> of people who are nibbling at the bottom end of the bugs, but I find it
>> somewhat frustrating not to be able to classify the issues that I find into
>> "needs help / chatting" vs "under development" to help make more sense of
>> the search results.
>>
>> Cheers,
>> -T
>>
>
>
>
> Maybe a flow chart can help unambiguize things a bit.  What follows is
> maybe one starting place.  I tried to separate steps that might be done by
> users from developers so that there is a clear order to the process.  This
> seems to follow the general way that python issues are resolved from what
> I've observed.
>
> While this chart separates the descriptive/brain storming and patch making
> parts of an issue, in actuality, a patch (tests, fix, and docs) can be
> developed partially or fully in the discussion faze for the purpose of
> helping the discussion, then later reused and fine tuned in the patch making
> faze.
>
> Hope this is helpful and doesn't get too mangled in sending.
>
>    (new issue)
>        |
>        v
>
>    [discuss issue] <-------------------------+
>        |                                     |
>        v                                     |
>                                              |
>    {request issue review}                    |
>        |                                     |
>        v                                     |
>                                              |
>    <invalid issue ?> (yes)--> [close issue]  |
>       (no)                        |          |
>        |                          v          |
>        |                       (closed)      |
>        v                                     |
>                                              |
>    <valid issue ?> (no)----------------------+
>      (yes)
>        |
>        v
>
>    [develop patch]  <------+
>        |                   |
>        v                   |
>                            |
>    {request patch review}  |
>        |                   |
>        v                   |
>                            |
>    <patch ready ?> (no) ---+
>      (yes)
>        |
>        v
>
>    [apply patch]
>        |
>        v
>
>    [close issue]
>        |
>        v
>
>    (closed)
>
>
>
>


-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090325/65e4afbe/attachment-0001.htm>

From steve at holdenweb.com  Wed Mar 25 03:52:43 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 24 Mar 2009 22:52:43 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C96304.7050207@v.loewis.de>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>	<49C87004.2030807@holdenweb.com>	<49C88503.2030902@v.loewis.de>	<49C886EF.80203@v.loewis.de>	<49C8C9B3.3070403@holdenweb.com>	<49C939BA.8040206@v.loewis.de>	<49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de>
Message-ID: <49C99C7B.3080805@holdenweb.com>

Martin v. L?wis wrote:
>> * The creation of an *installer* is something quite different.  An
>> installer for a py2exe based tool also doesn't need dependency
>> management.
> 
> Right. I wasn't really talking about py2exe (anymore), but about
> installers for libraries.
> 
>> An installer for a pure-python package that made no attempt
>> to bundle dependencies might be nice, but I don't quite see how that
>> falls outside the scope of distutils/setuptools/etc.  In other words, I
>> don't see why the installer can't bootstrap the 'normal' dependency
>> management which would be used if the package was installed any other
>> way or on other platforms.
> 
> Perhaps that could be a solution. However, in package management
> systems that solve this properly, you also have proper uninstallation,
> which includes:
> - uninstallation is rejected if packages still depend on the
>   to-be-removed package (or else offers to remove the relying packages
>   as well)
> - uninstallation reference-counts, causing an automatically-installed
>   package to be uninstalled if it is no longer needed, or else offers
>   to compute-then-uninstall all packages which are no longer needed.
> The .exe/.msi installers do support uninstallation, but, alas, no
> dependency management.
> 
>> * distutils already has the ability to create Windows installer
>> executables for pure-python apps/libs.  I agree it would be nice if it
>> was an MSI but that is an implementation detail rather than
>> implementation requirement.
> 
> Right - plus, distutils also supports creating MSIs.
> 
>>  How were Mike's packages fundamentally
>> different than that?
> 
> Dunno.

As far as memory serves, Mike built the installers precisely by using
distutils to build Windows installers. He then had to suffer criticism
from people who suggested this was inappropriately complex for pure
Python modules.

In so far as "end users" won't wish to install individual Python modules
this argument may have had some merit, but I personally thought the
criticism unjustified since Mike's technique gave a uniform install
procedure for everything.

I don't think anyone was suggesting that py2exe would be helpful for
writing installers--simply to reduce complexity from the user's point of
view and bundle dependencies in isolation from other applications. It
seems to do that quite well.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Wed Mar 25 03:58:53 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 24 Mar 2009 22:58:53 -0400
Subject: [Python-Dev] tracker status options
In-Reply-To: <Pine.LNX.4.64.0903241646550.31981@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0903240514490.31981@kimball.webabinitio.net>	<49C9424F.2030407@v.loewis.de>
	<Pine.LNX.4.64.0903241646550.31981@kimball.webabinitio.net>
Message-ID: <49C99DED.2070306@holdenweb.com>

R. David Murray wrote:
> On Tue, 24 Mar 2009 at 21:27, "Martin v. L?wis" wrote:
> 
>>>     o consensus needed
>>>     o test needed
>>>     o patch needed
>>>     o patch needs work
>>>     o patch review
>>>     o commit review
>>>
>>> The first of these additional items is equivalent to your bullet item
>>> above.  I would propose that the issue, regardless of whether or not
>>> it is a bug fix or a feature request, goes into 'consensus needed'
>>
>> Well, there is always the "unset" state, which means "not triaged".
>> So if you propose that this should be the default initial state,
>> I'm opposed.
> 
> No, I was not suggesting it be the default state.
> 
>> Instead, would it help to add a "confirmed" state? For a bug, that
>> would mean that it was successfully reproduced; for a patch, it
>> was confirmed as desirable.
> 
> So, 'confirmed' instead of 'consensus needed' (or confirmed/approved,
> to cover feature requests), and 'patch is appropriate' that comes...I'm
> not quite sure where?
> 
>> If the person performing triage cannot confirm the bug, they should
>> reject the issue (or recommend rejection, in case they are unsure).
>> Somebody performing triage should never conclude with "I don't
>> know"; this would be time-wasting.
> 
> The cases I was considering are cases where in the comments on the ticket
> there is disagreement either on whether or not the bug is a bug or (more
> often) whether or not the feature is desirable, and at the patch stage
> whether or not the patch is the appropriate fix.  I think currently
> things sit in 'patch needed' until consensus is reached on the patch,
> but I haven't watched enough tickets progress yet to be sure :)
> Adding another stage here may be more confusing than it is helpful,
> seeing as I can't really figure out where it would go.
> 
> Did I guess correctly that the process for "recommending rejection"
> is to set the stage to 'commit/reject', the resolution to 'invalid'
> (or whatever is appropriate) and the status to 'pending'?  That
> seemed to work for the issue I did it to, in that someone came
> along and closed it shortly after that.
> 
>> If, for a bug, the reviewer disagrees that it would be a bug if the
>> behavior actually exists, then the issue should be closed (resolution
>> not-a-bug/invalid). If the reviewer agrees that it would be a bug,
>> but fails to reproduce it, a test is needed.
> 
> OK, so I guess I now understand how the current workflow is supposed to
> work: if its stage is 'unassigned', then it hasn't been accepted as a
> bug/enhancement yet, and triage should make that accept/reject judgement.
> 
> The tricky bit here for me is that I, as a new person doing triage,
> don't always feel comfortable in passing judgement on a bug or feature
> request.  So what would be the mechanism for a triage person to request a
> "passing of judgement" from someone with more experience/authority?  Post
> to python-dev?  Given such a mechanism, I think I would be comfortable
> with the current workflow, with the expectation that I would need to
> call for assistance less and less frequently over time, and ultimately
> only for those things where discussion among the devs really is needed.
> 
> Hmm.  Maybe I should write a short "guide to performing triage" and
> post it for feedback.
> 
Might I suggest studying the status values used by the Django team in
their Trac tracker instance. They seem to have a very methodical workflow.

http://docs.djangoproject.com/en/dev/internals/contributing/#ticket-triage

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Wed Mar 25 03:52:43 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 24 Mar 2009 22:52:43 -0400
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C96304.7050207@v.loewis.de>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>	<49C87004.2030807@holdenweb.com>	<49C88503.2030902@v.loewis.de>	<49C886EF.80203@v.loewis.de>	<49C8C9B3.3070403@holdenweb.com>	<49C939BA.8040206@v.loewis.de>	<49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de>
Message-ID: <49C99C7B.3080805@holdenweb.com>

Martin v. L?wis wrote:
>> * The creation of an *installer* is something quite different.  An
>> installer for a py2exe based tool also doesn't need dependency
>> management.
> 
> Right. I wasn't really talking about py2exe (anymore), but about
> installers for libraries.
> 
>> An installer for a pure-python package that made no attempt
>> to bundle dependencies might be nice, but I don't quite see how that
>> falls outside the scope of distutils/setuptools/etc.  In other words, I
>> don't see why the installer can't bootstrap the 'normal' dependency
>> management which would be used if the package was installed any other
>> way or on other platforms.
> 
> Perhaps that could be a solution. However, in package management
> systems that solve this properly, you also have proper uninstallation,
> which includes:
> - uninstallation is rejected if packages still depend on the
>   to-be-removed package (or else offers to remove the relying packages
>   as well)
> - uninstallation reference-counts, causing an automatically-installed
>   package to be uninstalled if it is no longer needed, or else offers
>   to compute-then-uninstall all packages which are no longer needed.
> The .exe/.msi installers do support uninstallation, but, alas, no
> dependency management.
> 
>> * distutils already has the ability to create Windows installer
>> executables for pure-python apps/libs.  I agree it would be nice if it
>> was an MSI but that is an implementation detail rather than
>> implementation requirement.
> 
> Right - plus, distutils also supports creating MSIs.
> 
>>  How were Mike's packages fundamentally
>> different than that?
> 
> Dunno.

As far as memory serves, Mike built the installers precisely by using
distutils to build Windows installers. He then had to suffer criticism
from people who suggested this was inappropriately complex for pure
Python modules.

In so far as "end users" won't wish to install individual Python modules
this argument may have had some merit, but I personally thought the
criticism unjustified since Mike's technique gave a uniform install
procedure for everything.

I don't think anyone was suggesting that py2exe would be helpful for
writing installers--simply to reduce complexity from the user's point of
view and bundle dependencies in isolation from other applications. It
seems to do that quite well.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From pje at telecommunity.com  Wed Mar 25 04:35:10 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Tue, 24 Mar 2009 23:35:10 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <gqc4id$ud8$1@ger.gmane.org>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz> <gqc4id$ud8$1@ger.gmane.org>
Message-ID: <20090325033249.C8A3F3A4097@sparrow.telecommunity.com>

At 10:22 PM 3/24/2009 -0400, Steve Holden wrote:
>There is one non-trivial extension that I've been chewing over for a
>while. What if you want to yield not the values from the generator but
>some function of those values? The present proposal appears to have no
>way to specify that. What about extending the syntax somewhat to
>
>   yield expr for x from X
>
>The idea is that x should be a a bound variable in expr, but the "expr
>for x" could be optional to yield the existing proposal as a degenerate
>case.

That would be spelled:

    yield from (expr for x in X)

And the compiler could optionally optimize away the 
genexpr.  Assuming, of course, that this is considered valuable 
enough to implement in the first place, which I don't think it 
is...  especially not with the return bit factored in.

Now, if somebody came up with a different way to spell the extra 
value return, I wouldn't object as much to that part.  I can just see 
people inadvertently writing 'return x' as a shortcut for 'yield x; 
return', and then having what seem like mysterious off-by-one errors, 
or being confused by receiving a generator object instead of their 
desired non-generator return value.

It also seems weird that the only syntactically-supported way to get 
the generator's "return value" is to access it inside *another* 
generator...  which *also* can't return the return value to anyone!

But if it were spelled 'raise Return(value)' or 'raise 
StopIteration(value)' or something similar (or even had its own 
syntax!), I wouldn't object, as it would then be obvious how to get 
the value, and there could be no possible confusion with a regular 
return value.

The unusual spelling would also signal that something unusual (i.e., 
multitasking) is taking place, similar to the way some frameworks use 
things like 'yield Return(value)' to signal the end of a task and its 
return value, in place of a value in the stream.


From guido at python.org  Wed Mar 25 04:52:17 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Mar 2009 20:52:17 -0700
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090325033249.C8A3F3A4097@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz> <gqc4id$ud8$1@ger.gmane.org>
	<20090325033249.C8A3F3A4097@sparrow.telecommunity.com>
Message-ID: <ca471dc20903242052i6fd100c8wb1b2e72e3a0aa77f@mail.gmail.com>

On Tue, Mar 24, 2009 at 8:35 PM, P.J. Eby <pje at telecommunity.com> wrote:
> Now, if somebody came up with a different way to spell the extra value
> return, I wouldn't object as much to that part. ?I can just see people
> inadvertently writing 'return x' as a shortcut for 'yield x; return', and
> then having what seem like mysterious off-by-one errors, or being confused
> by receiving a generator object instead of their desired non-generator
> return value.
>
> It also seems weird that the only syntactically-supported way to get the
> generator's "return value" is to access it inside *another* generator...
> ?which *also* can't return the return value to anyone!
>
> But if it were spelled 'raise Return(value)' or 'raise StopIteration(value)'
> or something similar (or even had its own syntax!), I wouldn't object, as it
> would then be obvious how to get the value, and there could be no possible
> confusion with a regular return value.
>
> The unusual spelling would also signal that something unusual (i.e.,
> multitasking) is taking place, similar to the way some frameworks use things
> like 'yield Return(value)' to signal the end of a task and its return value,
> in place of a value in the stream.

I'm sympathetic to this point of view.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Wed Mar 25 06:09:36 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 25 Mar 2009 17:09:36 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <gqc4id$ud8$1@ger.gmane.org>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz> <gqc4id$ud8$1@ger.gmane.org>
Message-ID: <49C9BC90.3050508@canterbury.ac.nz>

Steve Holden wrote:
> What about extending the syntax somewhat to
> 
>   yield expr for x from X

I can't see much advantage that would give you
over writing

   for x in X:
     yield expr

There would be little or no speed advantage,
since you would no longer be able to shortcut
the intermediate generator during next().

-- 
Greg

From mike.klaas at gmail.com  Wed Mar 25 06:13:41 2009
From: mike.klaas at gmail.com (Mike Klaas)
Date: Tue, 24 Mar 2009 22:13:41 -0700
Subject: [Python-Dev] speeding up PyObject_GetItem
In-Reply-To: <A5D07132EF7E4444B4CC911DE883BE92@RaymondLaptop1>
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com><6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001><49C8A846.4020507@avl.com><eae285400903240530x2ecd50f7h53212bae422a8866@mail.gmail.com><eae285400903240742o1489f3c1veffae75e4493b073@mail.gmail.com><5c6f2a5d0903240813j318ea1b5m4fe6b20fedf56f99@mail.gmail.com><eae285400903240850n24a3ad3exf6fa5c30e00e770a@mail.gmail.com><5c6f2a5d0903241045w69282138v27e0224d1563bbc@mail.gmail.com>
	<loom.20090324T181207-923@post.gmane.org>
	<A5D07132EF7E4444B4CC911DE883BE92@RaymondLaptop1>
Message-ID: <C837E6D6-B3FB-4C30-B908-FC380A0C1138@gmail.com>


On 24-Mar-09, at 3:15 PM, Raymond Hettinger wrote:

>
>> 4% on a micro-micro-benchmark is hardly compelling...
>
> I concur!  This is utterly insignificant and certainly does
> not warrant removing the checks.
>
> -1 on these sort of fake optimizations.  We should focus
> on algorithmic improvements and eliminating redundant
> work and whatnot.  Removing checks that were put there for a reason  
> doesn't seem useful at all.

To be fair, the main proposed optimization(s) would speed up the  
microbenchmark by 15-25% (Daniel already stated that the NULL checks  
didn't have a significant impact).  This seems significant,  
considering that tight loops whose cost is heavily due to array access  
are common.

-Mike

From stephen at xemacs.org  Wed Mar 25 06:24:45 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 25 Mar 2009 14:24:45 +0900
Subject: [Python-Dev] tracker status options
In-Reply-To: <49C9424F.2030407@v.loewis.de>
References: <Pine.LNX.4.64.0903240514490.31981@kimball.webabinitio.net>
	<49C9424F.2030407@v.loewis.de>
Message-ID: <87k56edvj6.fsf@xemacs.org>

"Martin v. L?wis" writes:

 > >     o consensus needed
 > >     o test needed
 > >     o patch needed
 > >     o patch needs work
 > >     o patch review
 > >     o commit review
 > > 
 > > The first of these additional items is equivalent to your bullet item
 > > above.  I would propose that the issue, regardless of whether or not
 > > it is a bug fix or a feature request, goes into 'consensus needed'

All of these passives are making me aggressive....

Sure, these are real statuses internally, but they seem mostly of
interest to people who are actually working on the issue.  They'll be
getting nosy mail and following the issue in their browsers.  Is this
really needed for them?

On the other hand, for a non-developer who wants to know what's
happening to their issue, these are all rather fuzzy.  The submitter
of a bug wants to know

(1) Does python-dev admit it's an issue?
    No -> resolved because invalid
    Yes -> confirmed
    Don't know -> waiting for more information from submitter (I would
                  call this "pending response from OP")
    Not triaged -> "-- unset --" or new
(2) Has anybody accepted responsibility for working on this?
    Yes -> assigned
(3) Has anybody done any work recently?
    No -> sleeping
(4) Is there a solution?
    Yes -> resolved because implemented
(5) Can I get the solution by normal upgrading?
    Yes -> closed

So submitters already have *at least* seven statuses that they would
like to be able to recognize at a glance.  I think these correlate
well with what developers who accept broad responsibility (release
engineers, for example) need to know, too.

Does workflow coordination require more than that?  Almost certainly,
yes.  But is it a good idea to pack all that into status?

Also, note that code, doc, and test can be done in any order.  A
developer may write a test so he can automatically elicit the problem
behavior, then document the desired behavior, then come up with code
to implement it.  Or the OP could submit a patch, then the maintainer
decides what parts of the patch are specified behavior and what parts
are implementation details and documents it, and finally they come up
with a test.  That means you have at least *eight* different statuses,
or maybe 27 (ie, each component is tri-state: no improvement needed,
improvement needed, implemented).

I'm not suggesting that this kind of thing should be implemented;
rather, until the use case and who needs it is clear, the status field
should be kept as simple as possible.

 > Well, there is always the "unset" state, which means "not triaged".

I've had complaints from my users about the "-- not specified --"
status; they prefer "new".  They seem to think that "unspecified"
means the tracker is broken.

 > Instead, would it help to add a "confirmed" state? For a bug, that
 > would mean that it was successfully reproduced; for a patch, it
 > was confirmed as desirable.

Yes.  Lack of such a state is a PR disaster.  There needs to be a
signal that triage has happened, but the person doing triage typically
will not accept responsibility for fixing the bug, only for improving
the description of the reproduction recipe (including adding a test)
etc.

 > If the person performing triage cannot confirm the bug, they should
 > reject the issue (or recommend rejection, in case they are unsure).

If they're unsure, they should ping the submitter and request more
information, and set the status to "waiting for response".  As you say:

 > Somebody performing triage should never conclude with "I don't
 > know"; this would be time-wasting.
 > 
 > If, for a bug, the reviewer disagrees that it would be a bug if the
 > behavior actually exists, then the issue should be closed (resolution
 > not-a-bug/invalid).

Maybe set it "resolved" (if there's such a status separate from
"closed").  The idea is that if the issue gets "me toos", then it
should be added to the FAQ.  Also, such issues should be easy to
search for, but most standard searches restrict to "open" issues.

 > If the reviewer agrees that it would be a bug, but fails to
 > reproduce it, a test is needed.

By "reviewer" you mean the person doing triage, right?  By "test is
needed", you really mean "cooperation from the submitter in defining a
test", right?  (Tests are always needed to prevent regressions.)

From python at rcn.com  Wed Mar 25 06:18:04 2009
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 24 Mar 2009 22:18:04 -0700
Subject: [Python-Dev] speeding up PyObject_GetItem
References: <eae285400903230923w6695796w71aa25c56e765aae@mail.gmail.com><6420902.3717512.1237843319202.JavaMail.xicrypt@atgrzls001><49C8A846.4020507@avl.com><eae285400903240530x2ecd50f7h53212bae422a8866@mail.gmail.com><eae285400903240742o1489f3c1veffae75e4493b073@mail.gmail.com><5c6f2a5d0903240813j318ea1b5m4fe6b20fedf56f99@mail.gmail.com><eae285400903240850n24a3ad3exf6fa5c30e00e770a@mail.gmail.com><5c6f2a5d0903241045w69282138v27e0224d1563bbc@mail.gmail.com>
	<loom.20090324T181207-923@post.gmane.org>
	<A5D07132EF7E4444B4CC911DE883BE92@RaymondLaptop1>
	<C837E6D6-B3FB-4C30-B908-FC380A0C1138@gmail.com>
Message-ID: <0487F63F8C664878B4FFB069F8BFCE60@RaymondLaptop1>


>>> 4% on a micro-micro-benchmark is hardly compelling...
>>
>> I concur!  This is utterly insignificant and certainly does
>> not warrant removing the checks.
>>
>> -1 on these sort of fake optimizations.  We should focus
>> on algorithmic improvements and eliminating redundant
>> work and whatnot.  Removing checks that were put there for a reason  
>> doesn't seem useful at all.
> 
> To be fair, the main proposed optimization(s) would speed up the  
> microbenchmark by 15-25% (Daniel already stated that the NULL checks  
> didn't have a significant impact).  This seems significant,  
> considering that tight loops whose cost is heavily due to array access  
> are common.

I thought people used PyList_GET_ITEM or something similar
in those use situations.


Raymond

From greg.ewing at canterbury.ac.nz  Wed Mar 25 07:03:51 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 25 Mar 2009 18:03:51 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
Message-ID: <49C9C947.3080608@canterbury.ac.nz>

Guido van Rossum wrote:

> The way I think of it, that refactoring has nothing to do with
> yield-from.

I'm not sure what you mean by that. Currently it's
*impossible* to factor out code containing a yield.
Providing a way to do that is what led me to invent
this particular version of yield-from in the first
place.

I wanted a way of writing suspendable functions that
can call each other easily. (You may remember I
originally wanted to call it "call".) Then I noticed
that it would also happen to provide the functionality
of earlier "yield from" suggestions, so I adopted that
name.

But for me, factorability has always been the fundamental
idea, and the equivalence, in one particular restricted
situation, to a for loop containing a yield is just a
nice bonus.

That's what I've tried to get across in the PEP, and
it's the reason I've presented things in the way I have.

> It's not just variable references -- I used "scope" as a
> shorthand for everything that can be done within a function body,
> including control flow: try/except/finally,
> continue/break/raise/return.

Same answer applies -- use the usual techniques.

When I talk about inlining, I mean inlining the
*functionality* of the code, not its literal text. I'm
leaving the reader to imagine performing the necessary
transformations to preserve the semantics.

> Maybe you're confusing motivation with explanation? That feedback
> seems to tell me that the *motivation* needs more work; but IMO the
> *explanation* should start with this simple model and then expand upon
> the edge cases.

Perhaps what I should do is add a Motivation section before
the Proposal and move some of the material from the beginning
of the Rationale sectiomn there.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Wed Mar 25 07:15:27 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 25 Mar 2009 18:15:27 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090325033249.C8A3F3A4097@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz> <gqc4id$ud8$1@ger.gmane.org>
	<20090325033249.C8A3F3A4097@sparrow.telecommunity.com>
Message-ID: <49C9CBFF.3080705@canterbury.ac.nz>

P.J. Eby wrote:

> Now, if somebody came up with a different way to spell the extra value 
> return, I wouldn't object as much to that part.  I can just see people 
> inadvertently writing 'return x' as a shortcut for 'yield x; return', 

Well, they need to be educated not to do that. I'm
not sure they'll need much education about this
anyway. They've already been taught not to say
'return' when they mean 'yield', so I don't see why
they should suddenly start doing so now.

I'd be disappointed to lose that part of the proposal.
Part of my philosophy is that suspendable functions
should have the same rights and privileges as ordinary
ones, and that includes the ability to return values
using 'return'.

> It also seems weird that the only syntactically-supported way to get the 
> generator's "return value" is to access it inside *another* 
> generator...  which *also* can't return the return value to anyone!

Would you be happier if some syntactic way to do that
were provided?

It could perhaps be done by enhancing the part of the
'for' loop that gets executed upon normal termination
of the iterator.

   for x in my_iter:
     do_something_with(x)
   else v:
     handle_return_value(v)

> The unusual spelling would also signal that something unusual (i.e., 
> multitasking) is taking place, similar to the way some frameworks use 
> things like 'yield Return(value)' to signal the end of a task and its 
> return value, in place of a value in the stream.

Difference in philosophy again. To me, the need for such
an unusual construct when using these frameworks is a
wart, not a feature.

-- 
Greg

From skippy.hammond at gmail.com  Wed Mar 25 09:43:33 2009
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Wed, 25 Mar 2009 19:43:33 +1100
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <m2ljqu1pzh.fsf@valheru.db3l.homeip.net>
References: <loom.20090324T134451-902@post.gmane.org>	<m2zlfa241x.fsf@valheru.db3l.homeip.net>	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>	<d2155e360903241213v49e3b2ccr3bcdc7facbe36a@mail.gmail.com>	<930F189C8A437347B80DF2C156F7EC7F0519BB4A9F@exchis.ccp.ad.local>
	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net>
Message-ID: <49C9EEB5.2090804@gmail.com>

On 25/03/2009 10:05 AM, David Bolen wrote:
> Kristj?n Valur J?nsson<kristjan at ccpgames.com>  writes:
>
>> Now, I know that this msvc behaviour can be disabled, but it was
>> decided that it was not appropriate to meddle with runtime flags of
>> the whole process for python.
>
> I must have missed that discussion, but I can't see what the problem
> is if such an override only occurs during the buildbot non-interactive
> test runs?  Isn't that what using -n in the buildbot test.bat did?

The issue was that Python unconditionally changed the behaviour of the 
CRT, not only during the test suite.  I'm inclined to agree that Python 
itself (ie, pythonxx.dll on windows) has no business changing these 
flags, but it should be up to the application - ie, either python.exe 
itself, or exposed to the python app - the latter of which is what my 
patch offers.  Note however that my patch doesn't cause any .py code to 
actually change the flags, just offers the ability for them to do so.

Curt's suggestion of redirecting to a file is better still - I'll look 
at tweaking my patch to also offer that capability...

Cheers,

Mark

From db3l.net at gmail.com  Wed Mar 25 09:58:47 2009
From: db3l.net at gmail.com (David Bolen)
Date: Wed, 25 Mar 2009 04:58:47 -0400
Subject: [Python-Dev] Test failures under Windows?
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<d2155e360903241213v49e3b2ccr3bcdc7facbe36a@mail.gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F0519BB4A9F@exchis.ccp.ad.local>
	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net>
	<49C9EEB5.2090804@gmail.com>
Message-ID: <m2eiwm0yig.fsf@valheru.db3l.homeip.net>

Mark Hammond <skippy.hammond at gmail.com> writes:

> The issue was that Python unconditionally changed the behaviour of the
> CRT, not only during the test suite.

Hmm... I was more or less referring to the state of the py3k tree as
of, say, r57823 back in 2007.  It appeared to just add access to the
necessary functions in the msvcrt module, and a -n/--nowindow option
to regrtest.py which then used those functions to disable stuff before
running the test, and finally added -n to the tools/buildbot/test.bat
file, so things got disabled in buildbot test runs.  (By disabled, I
believe that all of the CRT warning/error/assert went to stderr)

So nothing permanently built into Python, nor enabled outside of a
buildbot regression test.  I agree permanently disabling them wouldn't
make sense, but I thought we also agreed to that back in 2007?

-- David

PS: The dialogs are showing up again on my build slave.  I tried
clearing it but then like 6+ of them showed up and kept showing up
even though I kept aborting them (I guess the tests kept running new
python_d instances or something).  I finally managed to catch up and then
whack the python_d processes manually through task manager.  Really seems
like disabling them can only make things more robust, otherwise the
Windows build slaves are just going to keep ending up unresponsive.


From kristjan at ccpgames.com  Wed Mar 25 10:25:53 2009
From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=)
Date: Wed, 25 Mar 2009 09:25:53 +0000
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <49C9EEB5.2090804@gmail.com>
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<d2155e360903241213v49e3b2ccr3bcdc7facbe36a@mail.gmail
	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net> <49C9EEB5.2090804@gmail.com>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>

Right.
In fact, having embedded python25.dll into an app once, I'm inclined to think that there is a lot of stuff that should be moved from that dll into python.exe, like argument parsing, path magic, and so on.  Py_Initialize() really is designed in terms of python.exe
Anyway,
What I was trying to say is that there are other things that can cause dialogue boxes to pop up beside this.  Assertions in other libraries, unhandled exceptions, and whatnot.  Surely, all of this poses a problem on windows?

Are the windows buildbots currently run as services, or are they started manually?

I'm going to poke my contacts at Microsoft and ask them if there is a way to disable popups like this for a process that runs unattended and/or is running as a windows service.

K

-----Original Message-----
From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Mark Hammond
Sent: 25. mars 2009 08:44
To: David Bolen
Cc: python-dev at python.org
Subject: Re: [Python-Dev] Test failures under Windows?

On 25/03/2009 10:05 AM, David Bolen wrote:
> Kristj?n Valur J?nsson<kristjan at ccpgames.com>  writes:
>
>> Now, I know that this msvc behaviour can be disabled, but it was
>> decided that it was not appropriate to meddle with runtime flags of
>> the whole process for python.
>
> I must have missed that discussion, but I can't see what the problem
> is if such an override only occurs during the buildbot non-interactive
> test runs?  Isn't that what using -n in the buildbot test.bat did?

The issue was that Python unconditionally changed the behaviour of the 
CRT, not only during the test suite.  I'm inclined to agree that Python 
itself (ie, pythonxx.dll on windows) has no business changing these 
flags, but it should be up to the application - ie, either python.exe 
itself, or exposed to the python app - the latter of which is what my 
patch offers.  Note however that my patch doesn't cause any .py code to 
actually change the flags, just offers the ability for them to do so.

Curt's suggestion of redirecting to a file is better still - I'll look 
at tweaking my patch to also offer that capability...

Cheers,

Mark
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com

From skippy.hammond at gmail.com  Wed Mar 25 12:40:32 2009
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Wed, 25 Mar 2009 22:40:32 +1100
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <m2eiwm0yig.fsf@valheru.db3l.homeip.net>
References: <loom.20090324T134451-902@post.gmane.org>	<m2zlfa241x.fsf@valheru.db3l.homeip.net>	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>	<d2155e360903241213v49e3b2ccr3bcdc7facbe36a@mail.gmail.com>	<930F189C8A437347B80DF2C156F7EC7F0519BB4A9F@exchis.ccp.ad.local>	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net>	<49C9EEB5.2090804@gmail.com>
	<m2eiwm0yig.fsf@valheru.db3l.homeip.net>
Message-ID: <49CA1830.8040005@gmail.com>

On 25/03/2009 7:58 PM, David Bolen wrote:
> Mark Hammond<skippy.hammond at gmail.com>  writes:
>
>> The issue was that Python unconditionally changed the behaviour of the
>> CRT, not only during the test suite.
>
> Hmm... I was more or less referring to the state of the py3k tree as
> of, say, r57823 back in 2007.

I was referring to the issue described in 
http://bugs.python.org/issue4804, which notes that exceptions.c 
unconditionally called:

   _CrtSetReportMode(_CRT_ASSERT, 0);

which has the effect of disabling crt assertions that would otherwise 
result in a modal dialog.  That bug remains open though, so maybe I'm 
confused about the problem which started this discussion...

Cheers,

Mark

From skippy.hammond at gmail.com  Wed Mar 25 12:42:01 2009
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Wed, 25 Mar 2009 22:42:01 +1100
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>
References: <loom.20090324T134451-902@post.gmane.org>	<m2zlfa241x.fsf@valheru.db3l.homeip.net>	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>	<d2155e360903241213v49e3b2ccr3bcdc7facbe36a@mail.gmail	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net>
	<49C9EEB5.2090804@gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>
Message-ID: <49CA1889.5000001@gmail.com>

> I'm going to poke my contacts at Microsoft and ask them if there is a way to disable popups like this for a process that runs unattended and/or is running as a windows service.

There is, and Curt pointed out one strategy for achieving this without 
losing the checks it provides...

> > Curt's suggestion of redirecting to a file is better still - I'll look
> > at tweaking my patch to also offer that capability...

From p.f.moore at gmail.com  Wed Mar 25 13:08:39 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 25 Mar 2009 12:08:39 +0000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49C99C7B.3080805@holdenweb.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<87ab7bh5fb.fsf@xemacs.org> <49C87004.2030807@holdenweb.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
Message-ID: <79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>

2009/3/25 Steve Holden <steve at holdenweb.com>:

> As far as memory serves, Mike built the installers precisely by using
> distutils to build Windows installers. He then had to suffer criticism
> from people who suggested this was inappropriately complex for pure
> Python modules.
>
> In so far as "end users" won't wish to install individual Python modules
> this argument may have had some merit, but I personally thought the
> criticism unjustified since Mike's technique gave a uniform install
> procedure for everything.

I've lost the context for this discussion completely, but can I offer
a couple of points from a Windows user's point of view:

1. Using distutils to build Windows installers for pure-python
packages is, in my view, *entirely* appropriate. The uniform install
procedure (and more importantly, uniform *un*install procedure) is a
key benefit for me. I can (and do) build my own installers for
pure-python modules which do not come with an installer (easy enough,
precisely because it's pure Python) but I feel that it's considerate
and helpful of distributors to offer bdist_wininst (or bdist_msi)
installers even when all it saves is a bit of my time.

2. Setuptools messes this clean picture up, for reasons I cannot
understand, but which annoy me intensely. When setuptools is involved,
it changes the default bdist_wininst behaviour somehow, so that the
installers become version-specific. This increases the burden on
distributors, which as a result means that you are less likely to find
bdist_wininst installers for setuptools-using pure python packages
(yes, I know, they give you eggs, and easy_install, yadda yadda...) So
where setuptools is involved, things do start to become
"inappropriately complex" - but that is a setuptools problem, NOT an
issue with providing installers.

3. Setuptools, unfortunately, has divided the Python distribution
community quite badly. I'd be curious to know whether the "people who
suggested this was inappropriately complex" you mention above were
setuptools supporters. My (unfounded) suspicion is that the argument
was "having to use a separate installer is more complex than just
using easy_install" - which is a highly debatable (and frequently
debated!) point of view.

> I don't think anyone was suggesting that py2exe would be helpful for
> writing installers--simply to reduce complexity from the user's point of
> view and bundle dependencies in isolation from other applications. It
> seems to do that quite well.

py2exe is good at creating bundled applications with no external
dependencies. It doesn't make installers for these apps, just a
directory containing the exe and all supporting files. That's a
completely different scenario, as you say.

Paul.

From solipsis at pitrou.net  Wed Mar 25 13:25:28 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 25 Mar 2009 12:25:28 +0000 (UTC)
Subject: [Python-Dev] "setuptools has divided the Python community"
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<87ab7bh5fb.fsf@xemacs.org> <49C87004.2030807@holdenweb.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
Message-ID: <loom.20090325T121432-793@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:
> 
> 3. Setuptools, unfortunately, has divided the Python distribution
> community quite badly.

Wait a little bit, and it's gonna be even worse, now that buildout and pip seem
to become popular. For example, the TurboGears people are considering switching
from setuptools to pip...

Tarek is now doing a lot of very useful work on distutils (thanks Tarek!), but
I'm not sure it'll be enough to stop people from converting to whatever of the
many build/packaging systems which have been popping up recently. Combined with
the current trend that everything must be exploded into lots of interdependent
but separately packaged libraries (the paste/pylons syndrome... try pulling
something like TurboGears2 and see how many third-party packages it installs), I
fear this is going to generate a very painful user/developer experience :-(

> My (unfounded) suspicion is that the argument
> was "having to use a separate installer is more complex than just
> using easy_install" - which is a highly debatable (and frequently
> debated!) point of view.

I'm not a Windows user, but I suppose it boils down to whether people are
comfortable with the command-line or not (which even many Windows /developers/
aren't). Since having GUIs for everything is part of the Windows philosophy,
it's a fair expectation that Python libraries come with graphical Windows
installers.

Regards

Antoine.



From ncoghlan at gmail.com  Wed Mar 25 13:46:53 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 25 Mar 2009 22:46:53 +1000
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
 currently conform
In-Reply-To: <ca471dc20903241314t318f2fb2vf864dae094592f53@mail.gmail.com>
References: <b8ad139e0903231352i66bc38f3v4f88b0914c760df7@mail.gmail.com>	<b8ad139e0903231400h570d771bt588ccbc71a295db4@mail.gmail.com>
	<ca471dc20903241314t318f2fb2vf864dae094592f53@mail.gmail.com>
Message-ID: <49CA27BD.7080904@gmail.com>

Guido van Rossum wrote:
> Please don't do this. We need stable APIs. Trying to switch the entire
> community to use CapWord APIs for something as commonly used as
> datetime sounds like wasting a lot of cycles with no reason except the
> mythical "PEP 8 conformance". As I said, it's a pity we didn't change
> this at the 3.0 point, but I think going forward we should try to be
> more committed to slow change. Additions of new functionality are of
> course fine. But renamings (even if the old names remain available)
> are just noise.

Even for 3.0, the only API I can recall doing this for was the threading
module, and there we had the additional motivation of being able to add
multiprocessing with only a PEP 8 compliant API while still having it be
close to a drop-in replacement for the corresponding threading API.

Having helped with that kind of rename once (and for a relatively small
API at that), I'd want a *really* compelling reason before ever going
through it again - it's messy, tedious and a really good way to burn
volunteer time without a great deal to show for it at the end.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From curt at hagenlocher.org  Wed Mar 25 13:53:45 2009
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Wed, 25 Mar 2009 05:53:45 -0700
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net> <49C9EEB5.2090804@gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>
Message-ID: <d2155e360903250553u2e134d61rc005e8817ad82a60@mail.gmail.com>

2009/3/25 Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
>
> I'm going to poke my contacts at Microsoft and ask them if there is
> a way to disable popups like this for a process that runs unattended
> and/or is running as a windows service.

MSVC has shipped with runtime library source since the 16-bit days, so
the easier thing is just to read the source code.  :)

It looks like there are two possible code paths for the assertion.
The first code path uses functions in dbgrpt[t].c and its behavior is
controlled by CrtSetReportMode and CrtSetReportFile.  The second uses
functions in assert.c and its behavior is controlled by
_set_error_mode.  Both code paths use __crtMessageBox in crtmbox.c to
actually display the message box.  This function -- when running as a
service -- will helpfully set MB_SERVICE_NOTIFICATION before calling
the USER32 MessageBox, which ensures that you'll see exactly why your
service has hung the next time you actually log in with sufficient
rights. :/

The variation that goes through assert.c should write to stderr for a
console-mode application, so it's reasonable to assume that we're
hitting the other code path -- and that Mark's suggestion to use
CrtSetReportMode would address the issue.

--
Curt Hagenlocher
curt at hagenlocher.org

From ncoghlan at gmail.com  Wed Mar 25 14:22:41 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 25 Mar 2009 23:22:41 +1000
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49C9CBFF.3080705@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>	<loom.20090321T122452-862@post.gmane.org>	<49C5620A.4080908@canterbury.ac.nz>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	<49C966A0.6090401@canterbury.ac.nz>
	<gqc4id$ud8$1@ger.gmane.org>	<20090325033249.C8A3F3A4097@sparrow.telecommunity.com>
	<49C9CBFF.3080705@canterbury.ac.nz>
Message-ID: <49CA3021.5050905@gmail.com>

Greg Ewing wrote:
> Would you be happier if some syntactic way to do that
> were provided?
> 
> It could perhaps be done by enhancing the part of the
> 'for' loop that gets executed upon normal termination
> of the iterator.
> 
>   for x in my_iter:
>     do_something_with(x)
>   else v:
>     handle_return_value(v)

I think something like that would actually make the PEP much stronger on
this front - it would promote the idea of a "final value" for iterators
as a more fundamental concept that can be worked with in a non-generator
context.

I'm also reminded of an idea that I believe existed in the early drafts
of PEP 342: using "continue value" to invoke an iterator's send() method
instead of next() as part of a normal for loop.

With those two ideas combined, the PEP's "yield from" expansion could
then look like:

  for x in EXPR:
    _v = yield x
    if _v is not None:
      continue _v
  else _r:
    RESULT = _r

(If "continue None" was defined as invoking .next() instead of
.send(None), then that loop body could be simplified to just "continue
yield x". However, I think it is preferable to keep the bare 'continue'
and dropping off the end of the loop as invoking next(), while "continue
arg" invokes send(None), since the latter form clearly *expects* the
iterator to have a send() method and it is best to emit the
AttributeError immediately if the method isn't there)

Strangely enough, I think proposing a more general change to the
iteration model to include sending values into iterators and having an
accessible "final value" may actually be easier to sell than trying to
sell "yield from" as a pure generator construct with no more general
statement level equivalent. Trying to sell the multi-stage function
iteration model and the concise expression form for invoking them from
another generator all at once is a lot to take in one gulp.

I suspect that angle of attack would also make *testing* this kind of
code far simpler as well. For example:

  for value, send_arg, expected in zip(gf_under_test(),
                                     send_args, expected_values):
    assertEqual(value, expected)
    continue send_arg
  else result:
    assertEqual(result, expected_result)

I'm not actually sure how you would go about writing a test driver for
that example multi-stage function *without* either making some kind of
change to for loops or developing some rather ugly test code.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From olemis at gmail.com  Wed Mar 25 14:32:27 2009
From: olemis at gmail.com (Olemis Lang)
Date: Wed, 25 Mar 2009 08:32:27 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <loom.20090325T121432-793@post.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
Message-ID: <24ea26600903250632r69d34c3bifede13fdbeb83736@mail.gmail.com>

On Wed, Mar 25, 2009 at 7:25 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Paul Moore <p.f.moore <at> gmail.com> writes:
>>
>> 3. Setuptools, unfortunately, has divided the Python distribution
>> community quite badly.
>
> Wait a little bit, and it's gonna be even worse, now that buildout and pip seem
> to become popular. For example, the TurboGears people are considering switching
> from setuptools to pip...
>

Yes ... this is certainly true ...

> Tarek is now doing a lot of very useful work on distutils (thanks Tarek!), but
> I'm not sure it'll be enough to stop people from converting to whatever of the
> many build/packaging systems which have been popping up recently.

Sometimes it also happens that, once one such build/packaging systems
is adopted, it is difficult to switch to using another one, since apps
(... and plugins systems ...) are frequently hard-coupled to the pkg
sys ?exotic features? and support (... what comes to my mind right now
is Trac plugins which are a little bit dependant on setuptools entry
points and so on ...).

>
>> My (unfounded) suspicion is that the argument
>> was "having to use a separate installer is more complex than just
>> using easy_install" - which is a highly debatable (and frequently
>> debated!) point of view.
>
> I'm not a Windows user, but I suppose it boils down to whether people are
> comfortable with the command-line or not (which even many Windows /developers/
> aren't).

Perhaps there is a little bit more contextual info that I miss ...
however I wanted to say something about this ... I'm not a Windows
user either (... although I do use Windows hosts pretty often for
testing or development purposes ;). In my case I use Ubuntu Hardy and
?Debian-children-or-brothers? ...

At least I see a point in using easy_install since ...

- it is quite easy to install the latest pkgs directly from PyPI ...
- Fresh dependencies are also retrieved from there ...

... but I dont actually use it (... I do use setuptools since apps are
tightly coupled with it ;), what I do is to use distutils exts to
build deb pkgs. Those are the ones that get actually installed in my
PC ... I prefer this way since:

- I avoid the kind of chaos created by easy_install due to the fact
that I cant control and manage my file/pkg associations and inter-pkg
dependencies ... so there are some kind of ?dangling? references to
files ... and I dont like that ... I like philosophies like the one
behind dpkg + apt  ...

- I can safely remove the packages formerly installed, including
dependencies and so on ...

... however this means that ...

- Quite often distros only include stable pkgs and sometimes official
debs are not up to date, ... I dont like that ...

- Packaging involves an extra effort ...

I definitely would like to see the day when :

- easy_install allow me to control in detail my installs, pks, deps &
uninsts ... and `--record` is not enought for that ... IMO

- distutils extensions for debs be distributed with core (similar to
rpms | msi ...), however it is possible that some dependency issues be
around ... I know this is a potentially long debate possibly for
distutils SIG ;)

- debs can be uploaded to & distributed in PyPI ... this way it could
be even possible to make easy_install use these debs and try to
install them using the underlying Debian support libs ... in case they
be installed ... and this is nothing but some kind of crazy ?random
idea? ...

... and since that day on I think I'll be seriously considering
easy_install as a candidate for my installs, even if there are some
other side-effects while using setuptools ... ;)

> Since having GUIs for everything is part of the Windows philosophy,
> it's a fair expectation that Python libraries come with graphical Windows
> installers.
>

Possibly yes ... in my case, I dont pretty like GUI installers (but I
understand that other people might like them ...) and still I build
them for Py pkgs in Win32 systems, since doing so I can control what's
happenning with them (not as well as with apt + dpkg ... but at least
I can cleanly remove them ;), however I need to handle dependencies by
hand ... :$

Well ... all this is IMO ... and I am not a proficient easy_install
user either, so CMIIW ... ;)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From ziade.tarek at gmail.com  Wed Mar 25 14:36:05 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 25 Mar 2009 14:36:05 +0100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <loom.20090325T121432-793@post.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
Message-ID: <94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>

On Wed, Mar 25, 2009 at 1:25 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Paul Moore <p.f.moore <at> gmail.com> writes:
>>
>> 3. Setuptools, unfortunately, has divided the Python distribution
>> community quite badly.
>
> Wait a little bit, and it's gonna be even worse, now that buildout and pip seem
> to become popular. For example, the TurboGears people are considering switching
> from setuptools to pip...
>
> Tarek is now doing a lot of very useful work on distutils (thanks Tarek!), but
> I'm not sure it'll be enough to stop people from converting to whatever of the
> many build/packaging systems which have been popping up recently. Combined with
> the current trend that everything must be exploded into lots of interdependent
> but separately packaged libraries (the paste/pylons syndrome... try pulling
> something like TurboGears2 and see how many third-party packages it installs), I
> fear this is going to generate a very painful user/developer experience :-(
>

I think we are in a point in Python development where we need to clearly define
how dependencies work. And this has to be defined in Python (in Distutils)
for the sake of standardization.

I think the Language Summit tomorrow is a good place to discuss about
these problems,
and to make sure pip, setuptools and zc.buildout rely on the same
standards and pieces.

PEP 376 is my first attempt to make it happen, and my goal is to see another
pre-PEP coming out of thea language summit, adressing the dependencies problem.

I can't hear that setuptools has divided the Python community. It has provided
solutions to real problems we had in web development. It's unperfect,
and it has to be
fixed and integrated into Python. But it should not be done outside Python imho.

If you had worked with Zope 5 years ago, you would understand what
setuptools and
zc.buildout brought to these communities. And today's experience is a
whole less painful trust me.

But I agree that the sizes of the packages are too small now, and it has gone
to far. Installing a web app like Plone is scary (+100 packages)

But this is the responsability of Zope, TG, etc to distribute their packages in
bigger pieces I guess.

(I can't wait to be at the summit ! :))

Cheers
Tarek

From ncoghlan at gmail.com  Wed Mar 25 14:37:49 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 25 Mar 2009 23:37:49 +1000
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CA3021.5050905@gmail.com>
References: <49C48D03.7030709@gmail.com>	<loom.20090321T122452-862@post.gmane.org>	<49C5620A.4080908@canterbury.ac.nz>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	<49C966A0.6090401@canterbury.ac.nz>
	<gqc4id$ud8$1@ger.gmane.org>	<20090325033249.C8A3F3A4097@sparrow.telecommunity.com>
	<49C9CBFF.3080705@canterbury.ac.nz> <49CA3021.5050905@gmail.com>
Message-ID: <49CA33AD.20802@gmail.com>

Nick Coghlan wrote:
> With those two ideas combined, the PEP's "yield from" expansion could
> then look like:
> 
>   for x in EXPR:
>     _v = yield x
>     if _v is not None:
>       continue _v
>   else _r:
>     RESULT = _r

Oops, got a little carried away there. Obviously, that doesn't handle
thrown in exceptions the way "yield from" is intended to.

So even with an adjusted for loop the full semantic expansion of 'yield
from' would still need to be defined directly in terms of try/except and
method calls on the underlying iterator to get the desired exception
handling characteristics.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From solipsis at pitrou.net  Wed Mar 25 14:45:32 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 25 Mar 2009 13:45:32 +0000 (UTC)
Subject: [Python-Dev] "setuptools has divided the Python community"
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
Message-ID: <loom.20090325T133856-563@post.gmane.org>

Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
> 
> But I agree that the sizes of the packages are too small now, and it has gone
> to far. Installing a web app like Plone is scary (+100 packages)

I am working on a TurboGears2-based app and I just did a count of the .egg
packages in the virtualenv. There are 45 of them....

People should really stop splitting their work into micro-libraries (with such
ludicrous names as "AddOns" or "Extremes", I might add (*)), and myriads of
separately-packaged plugins (the repoze stuff). The Twisted approach is much
saner, where you have a cohesive whole in a single package.

(*) The package descriptions are worth their salt too:
http://pypi.python.org/pypi/AddOns
http://pypi.python.org/pypi/Extremes

Regards

Antoine.



From ncoghlan at gmail.com  Wed Mar 25 15:01:55 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Mar 2009 00:01:55 +1000
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <873ad3gxdd.fsf@xemacs.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>
	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>
	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>
	<87ab7bh5fb.fsf@xemacs.org>	<49C87004.2030807@holdenweb.com>
	<873ad3gxdd.fsf@xemacs.org>
Message-ID: <49CA3953.1080506@gmail.com>

Stephen J. Turnbull wrote:
> "Batteries included"
> is the status quo for the (C) Python project, and I personally don't
> see a strong reason to change that policy until those who favor a
> package-manager-based solution have a package manager that satisfies
> them!

Beautifully put :)

And a package based solution that satisfies everyone (or even a
significant majority), be they on Windows, OS X, .deb based Linux, .rpm
based Linux, some other Linux or *nix, be they a mere user or the
machine admin... I think that's a problem that still has a lot of
development work in front of it ;)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ziade.tarek at gmail.com  Wed Mar 25 15:02:13 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 25 Mar 2009 15:02:13 +0100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <loom.20090325T133856-563@post.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<loom.20090325T133856-563@post.gmane.org>
Message-ID: <94bdd2610903250702x6b218369jc7175067a40f39ad@mail.gmail.com>

On Wed, Mar 25, 2009 at 2:45 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
>>
>> But I agree that the sizes of the packages are too small now, and it has gone
>> to far. Installing a web app like Plone is scary (+100 packages)
>
> I am working on a TurboGears2-based app and I just did a count of the .egg
> packages in the virtualenv. There are 45 of them....
>
> People should really stop splitting their work into micro-libraries (with such
> ludicrous names as "AddOns" or "Extremes", I might add (*)), and myriads of
> separately-packaged plugins (the repoze stuff). The Twisted approach is much
> saner, where you have a cohesive whole in a single package.

Yes but this means that you have to wait for the next version of the
"big" package
when a bug is corrected or a feature added, or you need to patch it.
(or maybe use the namespace trick to override it)

Having them as separated package allow different release cycles, which makes it
more "agile".

I think there should be a right middle between a big package and a myriad of
small ones.

In Zope for instance, we could probably reunite a lot of package in a single
package, because they don't evolve much anymore. (things like zope.interface)


Regards
Tarek

From p.f.moore at gmail.com  Wed Mar 25 15:08:28 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 25 Mar 2009 14:08:28 +0000
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C886EF.80203@v.loewis.de> <49C8C9B3.3070403@holdenweb.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
Message-ID: <79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>

2009/3/25 Tarek Ziad? <ziade.tarek at gmail.com>:
> I can't hear that setuptools has divided the Python community. It has provided
> solutions to real problems we had in web development. It's unperfect,
> and it has to be
> fixed and integrated into Python. But it should not be done outside Python imho.

It's quite possible that setuptools has solved real issues in the web
development area. But it did so at the expense of ease of use for
people *not* in that area.

I use Python for systems admin scripts, Windows services, and database
management. In my experience (and I agree, it's only one, limited, use
case) availability of download-and-run bdist_wininst installers for
every package I used was the only significant requirement I had for
Python package distribution (I remember pre-distutils days, when being
able to install a 3rd party package on Windows was a nightmare of
build-it-yourself guesswork).

Since setuptools came on the scene, I can state with some certainty
that many packages which would otherwise have been distributed as
bdist_wininst installers, now aren't. In some cases, only source
packages are provided (on the basis that easy_install will build what
you need). In those cases, I can accept that maybe the developer would
not have built Windows installers even before setuptools arrived. But
in a significant number of cases - including setuptools itself!!!! -
binary, version-specific eggs for Windows are provided, but no
bdist_wininst installers. If the developer is willing to build an egg,
he could just as easily have built an installer - but he now has to
choose - build one or the other, or both. And not everyone chooses the
same way.

Hence my comment about "dividing the community". From my limited
perspective, it's about no longer having a standard Windows binary
distribution format used by all, not about some sort of ideological
battles. Sorry for being unclear.

Paul.

From ncoghlan at gmail.com  Wed Mar 25 15:19:58 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Mar 2009 00:19:58 +1000
Subject: [Python-Dev] tracker status options
In-Reply-To: <2d75d7660903241622o13ac9f34j251fb887684448c9@mail.gmail.com>
References: <43c8685c0903182033l4348e445yaa54e219ded5ec2e@mail.gmail.com>
	<49C1EA9E.3040603@v.loewis.de>	<bbaeab100903191414o1710bd87x3d4c9a713c63c832@mail.gmail.com>
	<49C2D040.4080609@v.loewis.de>	<2d75d7660903191653v65640c90pc4df8ddcf60343f1@mail.gmail.com>
	<43c8685c0903222220h6ecc841ar27ce66adf3c3d064@mail.gmail.com>
	<Pine.LNX.4.64.0903232156330.31981@kimball.webabinitio.net>
	<2d75d7660903241622o13ac9f34j251fb887684448c9@mail.gmail.com>
Message-ID: <49CA3D8E.6030400@gmail.com>

Daniel (ajax) Diniz wrote:
> I feel we should make the tracker more useful for core developers,
> volunteers and end-users. I also think having a clear workflow helps a
> lot. Yet, I'd rather have a tracker that allowed users with different
> interpretations to work as they feel most comfortable than one that
> requires a change of assumptions.

One other thing to keep in mind (and something I try to do myself) is to
 include a comment saying *why* a particular status change was made.
Yes, that doesn't help all that much when people are searching for
issues at various stages, but it definitely helps:

a) anyone on the nosy list (since the comment and the field changes will
typically either arrive in the same email or at least close together in
time)
b) anyone reading the issue later (as they will see more than just a
series of status changes at the bottom of the issue page)

If it's a matter of the triage folks wanting to be able to mark
particular kinds of issues to make them easier to find or avoid later,
then keywords may be a better option than messing with the stages.

Heck, you could even go for the internet UI du jour and allow free form
tags rather than being restricted to just the preset list of keywords -
the free form tags would be a little more experimental, with the more
useful ones eventually making their way to the list of properly defined
keywords. The less useful ones could fall into natural disuse without
cluttering up the main keyword selection UI.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ziade.tarek at gmail.com  Wed Mar 25 15:23:51 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 25 Mar 2009 15:23:51 +0100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
Message-ID: <94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>

On Wed, Mar 25, 2009 at 3:08 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>
> Hence my comment about "dividing the community". From my limited
> perspective, it's about no longer having a standard Windows binary
> distribution format used by all, not about some sort of ideological
> battles. Sorry for being unclear.

Are you coming to the Summit tomorrow ? If not, I'll bring up this point.

I whish we will be able in the near future to build a team of people
from various area of
expertise, to work on these problems.

Tarek

>
> Paul.
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From pje at telecommunity.com  Wed Mar 25 15:26:27 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 25 Mar 2009 10:26:27 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49C9C947.3080608@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
Message-ID: <20090325142405.C68463A40A7@sparrow.telecommunity.com>

At 06:03 PM 3/25/2009 +1200, Greg Ewing wrote:
>I wanted a way of writing suspendable functions that
>can call each other easily. (You may remember I
>originally wanted to call it "call".) Then I noticed
>that it would also happen to provide the functionality
>of earlier "yield from" suggestions, so I adopted that
>name.

I still don't see what you gain from making this syntax, vs. putting 
something like this in the stdlib (rough sketch):

class Task(object):
     def __init__(self, geniter):
         self.stack = [geniter]

     def __iter__(self):
         return self

     def send(self, value=None):
         if not self.stack:
             raise RuntimeError("Can't resume completed task")
         return self._step(value)

     send = next

     def _step(self, value=None, exc_info=()):
         while self.stack:
             try:
                 it = self.stack[-1]
                 if exc_info:
                     try:
                         rv = it.throw(*exc_info)
                     finally:
                         exc_info = ()
                 elif value is not None:
                     rv = it.send(value)
                 else:
                     rv = it.next()
             except:
                 value = None
                 exc_info = sys.exc_info()
                 if exc_info[0] is StopIteration:
                     exc_info = ()   # not really an error
                 self.pop()
             else:
                 value, exc_info = yield_to(rv, self)
         else:
             if exc_info:
                 raise exc_info[0], exc_info[1], exc_info[2]
             else:
                 return value

     def throw(self, *exc_info):
         if not self.stack:
             raise RuntimeError("Can't resume completed task")
         return self._step(None, exc_info)

     def push(self, geniter):
         self.stack.append(geniter)
         return None, ()

     def pop(self, value=None):
         if self.stack:
             it = self.stack.pop()
             if hasattr(it, 'close'):
                 try:
                     it.close()
                 except:
                     return None, sys.exc_info()
         return value, ()

     @classmethod
     def factory(cls, func):
         def decorated(*args, **kw):
             return cls(func(*args, **kw))
         return decorated


def yield_to(rv, task):
     # This could/should be a generic function, to allow yielding to
     # deferreds, sockets, timers, and other custom objects
     if hasattr(rv, 'next'):
         return task.push(rv)
     elif isinstance(rv, Return):
         return task.pop(rv.value)
     else:
         return rv, ()

class Return(object):
     def __init__(self, value=None):
         self.value = value


@Task.factory
def sample_task(arg1, another_arg):
     # blah blah
     something = (yield subtask(...))

     yield Return(result)

def subtask(...):
     ...
     yield Return(myvalue)


The trampoline (the _step() method) handles the co-operative aspects, 
and modifying the yield_to() function allows you to define how 
yielded values are processed.  By default, they're sent back into the 
generator that yields them, but you can pass a Return() to terminate 
the generator and pass the value up to the calling 
generator.  Yielding another generator, on the other hand, "calls" 
that generator within the current task, and the same rules apply.

Is there some reason why this won't do what you want, and can't be 
modified to do so?  If so, that should be part of the PEP, as IMO it 
otherwise lacks motivation for a language feature vs. say, a stdlib 
module.  If 'yield_to' is a generic function or at least supports 
registration of some kind, a feature like this would be interoperable 
with a wide variety of frameworks -- you could register deferreds and 
delayed calls and IO objects from Twisted, for example.  So it's not 
like the feature would be creating an entire new framework of its 
own.  Rather, it'd be a front-end to whatever framework (or no 
framework) you're using.


From pje at telecommunity.com  Wed Mar 25 15:28:04 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 25 Mar 2009 10:28:04 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <loom.20090325T121432-793@post.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<87ab7bh5fb.fsf@xemacs.org> <49C87004.2030807@holdenweb.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
Message-ID: <20090325142541.56C163A40A5@sparrow.telecommunity.com>

At 12:25 PM 3/25/2009 +0000, Antoine Pitrou wrote:
>Paul Moore <p.f.moore <at> gmail.com> writes:
> >
> > 3. Setuptools, unfortunately, has divided the Python distribution
> > community quite badly.
>
>Wait a little bit, and it's gonna be even worse, now that buildout 
>and pip seem
>to become popular. For example, the TurboGears people are 
>considering switching
>from setuptools to pip...

You mean from *easy_install* to pip.  pip still uses and supports setuptools.


From p.f.moore at gmail.com  Wed Mar 25 15:28:05 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 25 Mar 2009 14:28:05 +0000
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <94bdd2610903250702x6b218369jc7175067a40f39ad@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<loom.20090325T133856-563@post.gmane.org>
	<94bdd2610903250702x6b218369jc7175067a40f39ad@mail.gmail.com>
Message-ID: <79990c6b0903250728i6e734baap3ada1ed15e3b13ff@mail.gmail.com>

2009/3/25 Tarek Ziad? <ziade.tarek at gmail.com>:
>> People should really stop splitting their work into micro-libraries (with such
>> ludicrous names as "AddOns" or "Extremes", I might add (*)), and myriads of
>> separately-packaged plugins (the repoze stuff). The Twisted approach is much
>> saner, where you have a cohesive whole in a single package.
>
> Yes but this means that you have to wait for the next version of the
> "big" package
> when a bug is corrected or a feature added, or you need to patch it.
> (or maybe use the namespace trick to override it)
>
> Having them as separated package allow different release cycles, which makes it
> more "agile".

Another division :-) (Not one I'll try to blame on setuptools, though :-))

Some people find larger, stable, unified packages more useful. Others
find fine-grained, rapidly developing packages more useful.

It sounds like Antoine and I fall into the former camp. At the risk of
generalising, maybe the latter philosophy fits the "web developer"
mental model better?

I doubt this is related to the current discussion, though - except in
that setuptools, with its claim to make managing large numbers of
dependencies relatively painless, encourages people to forget the cost
involved in micro-packages.

Paul.

From pje at telecommunity.com  Wed Mar 25 15:31:18 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 25 Mar 2009 10:31:18 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <24ea26600903250632r69d34c3bifede13fdbeb83736@mail.gmail.co
 m>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<24ea26600903250632r69d34c3bifede13fdbeb83736@mail.gmail.com>
Message-ID: <20090325142855.BBC713A40A5@sparrow.telecommunity.com>

At 08:32 AM 3/25/2009 -0500, Olemis Lang wrote:
>Sometimes it also happens that, once one such build/packaging systems
>is adopted, it is difficult to switch to using another one, since apps
>(... and plugins systems ...) are frequently hard-coupled to the pkg
>sys ?exotic features? and support (... what comes to my mind right now
>is Trac plugins which are a little bit dependant on setuptools entry
>points and so on ...).

Please note that entry points are not coupled to easy_install.  They 
have a documented file format and API that's *distributed* with 
setuptools, but is not dependent on it and does not require .egg 
files, either.  There's nothing stopping an independent 
implementation in the stdlib, ala PEP 376's metadata API, which is 
designed to be backward compatible with setuptools.  It could be 
extended to support entry points as well.


From p.f.moore at gmail.com  Wed Mar 25 15:35:22 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 25 Mar 2009 14:35:22 +0000
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
Message-ID: <79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>

2009/3/25 Tarek Ziad? <ziade.tarek at gmail.com>:
> On Wed, Mar 25, 2009 at 3:08 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>>
>> Hence my comment about "dividing the community". From my limited
>> perspective, it's about no longer having a standard Windows binary
>> distribution format used by all, not about some sort of ideological
>> battles. Sorry for being unclear.
>
> Are you coming to the Summit tomorrow ? If not, I'll bring up this point.

Sorry, no. I'm on the wrong continent :-)

If you would, I'd appreciate it. Sometimes I feel that the
distutils/setuptools discussions need better input from the
non-web-developer community. And even more so from the "not a
developer, just a user" community! But the various communities
involved seem to struggle to appreciate each others' viewpoints, to
such a great extent that it's hard to make any progress.

Good luck with the Summit!

Paul.

From guido at python.org  Wed Mar 25 15:40:44 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Mar 2009 07:40:44 -0700
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <20090325142855.BBC713A40A5@sparrow.telecommunity.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<24ea26600903250632r69d34c3bifede13fdbeb83736@mail.gmail.com>
	<20090325142855.BBC713A40A5@sparrow.telecommunity.com>
Message-ID: <ca471dc20903250740w40edc466h937be86e80e30928@mail.gmail.com>

On Wed, Mar 25, 2009 at 7:31 AM, P.J. Eby <pje at telecommunity.com> wrote:
> Please note that entry points are not coupled to easy_install. ?They have a
> documented file format and API that's *distributed* with setuptools, but is
> not dependent on it and does not require .egg files, either. ?There's
> nothing stopping an independent implementation in the stdlib, ala PEP 376's
> metadata API, which is designed to be backward compatible with setuptools.
> ?It could be extended to support entry points as well.

Well, sorry, but this complex layered interdependent architecture is
one of the *causes* of confusion -- apart from you nobody knows what
is what exactly, and the names are often used "incorrectly". BTW this
also makes it hard to find downloadable versions of these tools if you
don't already know the magic Google query (and I happen to get that
wrong each time I want to learn about these).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From olemis at gmail.com  Wed Mar 25 15:41:29 2009
From: olemis at gmail.com (Olemis Lang)
Date: Wed, 25 Mar 2009 09:41:29 -0500
Subject: [Python-Dev]  "setuptools has divided the Python community"
In-Reply-To: <24ea26600903250733x4d0092e9ydd56e68662bd33a5@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<24ea26600903250733x4d0092e9ydd56e68662bd33a5@mail.gmail.com>
Message-ID: <24ea26600903250741g4351c29dx94cb75ed9803179@mail.gmail.com>

On Wed, Mar 25, 2009 at 8:36 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Wed, Mar 25, 2009 at 1:25 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Paul Moore <p.f.moore <at> gmail.com> writes:
>>>
>>> 3. Setuptools, unfortunately, has divided the Python distribution
>>> community quite badly.
>>
>> Wait a little bit, and it's gonna be even worse, now that buildout and pip seem
>> to become popular. For example, the TurboGears people are considering switching
>> from setuptools to pip...
>>
>> Combined with
>> the current trend that everything must be exploded into lots of interdependent
>> but separately packaged libraries (the paste/pylons syndrome... try pulling
>> something like TurboGears2 and see how many third-party packages it installs), I
>> fear this is going to generate a very painful user/developer experience :-(
>>
>
> I think we are in a point in Python development where we need to clearly define
> how dependencies work.

+1

> And this has to be defined in Python (in Distutils)
> for the sake of standardization.
>

Well, I mentionned I did it using apt + insinstall support ... but I'd
love to have all these things in Python ... with or without relying on
OS-specific infrastructure for managing pkgs, deps and so on ... It is
possible to have two approaches :

- a thin layer on top of OS-specific (dpkg + apt, rpm + yum,
wininstall, mac os ... ) pkg managers so that distutils be able to use
them directly whille still keeping a single model from the end-user
perspective ... At least there exists some kind of python-debian pkg
...

{{{
$ apt-cache show python-debian
Package: python-debian
Priority: optional
Section: universe/devel
Installed-Size: 268
Maintainer: Ubuntu MOTU Developers <ubuntu-motu at lists.ubuntu.com>
Original-Maintainer: Debian python-debian Maintainers
<pkg-python-debian-maint at lists.alioth.debian.org>
Architecture: all
Version: 0.1.9
[...]
Depends: python (>= 2.4), python-support (>= 0.7.1)
Recommends: python-apt
[...]
Size: 43662
[...]
Description: python modules to work with Debian-related data formats
?This package provides python modules that abstract many formats of Debian
?related files. Currently handled are:
?* Debtags information (debian_bundle.debtags module)
?* debian/changelog (debian_bundle.changelog module)
?* Packages files, pdiffs (debian_bundle.debian_support module)
?* Control files of single or multple RFC822-style paragraphs, e.g
? ?debian/control, .changes, .dsc, Packages, Sources, Release, etc.
? ?(debian_bundle.deb822 module)
?* Raw .deb and .ar files, with (read-only) access to contained
? ?files and meta-information
[...]
}}}

Since there are a lot such systems (at least for Linux ...) ... this
could also mean that it could be very complex to handle all this
diversity ... However there are a few which are highly influential ...
but this is certainly a concern ...

- A fully fledged implementation from scratch so that distutils be in
charge, all apps be consistent with Py-std ... and users have
potentially two pkg systems (.. the built-in and the Py-specific ;) ;
and I say potentially since it's quite sudden so far and perhaps there
is a way to smoothly integrate both systems (which are potentially
many of them ;)

> I think the Language Summit tomorrow is a good place to discuss about
> these problems,

Hope it be available (recorded ?) some day so as to hear some opinions ... ;)

> and to make sure pip, setuptools and zc.buildout rely on the same
> standards and pieces.
>

Oh yes !
+1

> PEP 376 is my first attempt to make it happen, and my goal is to see another
> pre-PEP coming out of thea language summit, adressing the dependencies problem.
>

;)

> I can't hear that setuptools has divided the Python community.

Said like this ... some might think that setuptools is responsible for
someone else's choices and actions ... and I dont think so ...

> It has provided
> solutions to real problems we had in web development. It's unperfect,
> and it has to be
> fixed and integrated into Python. But it should not be done outside Python imho.
>

I mostly agree with all this, but I would like to know something ...
in case a single pkg management sys be ready for Py (... I mean
integrated into Py ...) and so on ... Why will we need distutils
comands like :

bdist_msi
bdist_wininst
bdist_rpm
<... more ?non-standard? candidates like py2exe, stdeb, ...>

? Will all this still be necessary and maintained ? As long as
installing /uninstalling them be handled correctly in built-in as well
as Py systems (considering the duplicate ?package repositories? ...)
... I think this could be fine ... but details are details ... ;)

> If you had worked with Zope 5 years ago, you would understand what
> setuptools and
> zc.buildout brought to these communities. And today's experience is a
> whole less painful trust me.
>

Yes ... setuptools has undoubtedly its own value ... :o)

> But I agree that the sizes of the packages are too small now, and it has gone
> to far. Installing a web app like Plone is scary (+100 packages)
>
> But this is the responsability of Zope, TG, etc to distribute their packages in
> bigger pieces I guess.
>

I do think so ... The same happens with Trac plugins. In case of
browsing the archive of trac-users mailing list anyone can realize
that there are many related plugins people'd like to install at once
to get the whole functionality. Nowadays in some cases they have to
install ... *a loooot* and then configure ... *a loooot* in order to
get the whole thing working out ... and there are tiny but useful
plugins ... Besides further contributions seldom end-up with potential
interoperability issues and related features might not be tested as a
whole ...

And things that should be in core (e.g. AccountManager plugin ...
since it seems there is no way to log out if it's not installed ... ;)
are not still there (... took some time until trac-devs decided to
incorporate it during PyCon'09 sprint ... ;)

But this unification is definitely up to the plugin & framew devs ... IMO

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From barry at python.org  Wed Mar 25 15:52:03 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 25 Mar 2009 09:52:03 -0500
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49CA3953.1080506@gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>
	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>
	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>
	<87ab7bh5fb.fsf@xemacs.org>	<49C87004.2030807@holdenweb.com>
	<873ad3gxdd.fsf@xemacs.org> <49CA3953.1080506@gmail.com>
Message-ID: <638F8377-6967-44B9-8ECF-F5DF80CD32BC@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 25, 2009, at 9:01 AM, Nick Coghlan wrote:

> And a package based solution that satisfies everyone (or even a
> significant majority), be they on Windows, OS X, .deb based  
> Linux, .rpm
> based Linux, some other Linux or *nix, be they a mere user or the
> machine admin... I think that's a problem that still has a lot of
> development work in front of it ;)

I don't think that will ever happen.  I regularly use three *nix  
systems and each has a different package management systems and I  
don't think I even want a unified package manager.

Tools like setuptools, zc.buildout, etc. seem great for developers but  
not very good for distributions.  At last year's Pycon I think there  
was agreement from the Linux distributors that distutils, etc. just  
wasn't very useful for them.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScpFE3EjvBPtnXfVAQJ6FwP9GdT9gVyfDp1aCLlfXsw61hcwU+6E3sX2
kvuqRYyXeTlecvRWnhc0YKzLjIke+4K+z7DlKxKazWaOsgkLTxxKrB1HTOwqKsiA
+PxIrHK5tSKbvoBLphFQVgtDcqzb07xxxsz0KsTxWR/mskvR0eYf1/XZl3+b+9sQ
M1POnOxXCN0=
=8VhZ
-----END PGP SIGNATURE-----

From solipsis at pitrou.net  Wed Mar 25 15:53:29 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 25 Mar 2009 14:53:29 +0000 (UTC)
Subject: [Python-Dev] "setuptools has divided the Python community"
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<loom.20090325T133856-563@post.gmane.org>
	<94bdd2610903250702x6b218369jc7175067a40f39ad@mail.gmail.com>
	<79990c6b0903250728i6e734baap3ada1ed15e3b13ff@mail.gmail.com>
Message-ID: <loom.20090325T143347-105@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:
> 
> Another division  (Not one I'll try to blame on setuptools, though )
> 
> Some people find larger, stable, unified packages more useful. Others
> find fine-grained, rapidly developing packages more useful.
> 
> It sounds like Antoine and I fall into the former camp. At the risk of
> generalising, maybe the latter philosophy fits the "web developer"
> mental model better?

I don't think it's a question of mental model. Whatever your mentality, having
O(N) libraries means you have O(N**2) potential interdependencies to worry about
(especially when those libraries are socially and functionally part of a larger
ensemble). When N is 30 or 100, it's easy to see where it leads: loss of
control, unmanageability, irresponsibility.

The argument that you can individually upgrade each micro-library is a
double-edged sword: individual upgrades can break the interaction with other
micro-libraries because of subtle changes in API or behaviour. If you post on
the framework's mailing-list to get help with the problem, the framework authors
are unable to tell you how to solve it because /even they/ can't master the
O(N**2) interdependencies. Then you must upgrade the other micro-libraries as
well, which is no better than having a large cohesive package where everything
evolves in lockstep by construction.

(by the way, I don't think it's setuptools' fault... that's why I've used quotes
in the title. People have simply abused a tool, the same way "DLL hell" is an
abuse of the shared library system)

Regards

Antoine.



From barry at python.org  Wed Mar 25 15:54:06 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 25 Mar 2009 09:54:06 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <94bdd2610903250702x6b218369jc7175067a40f39ad@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<loom.20090325T133856-563@post.gmane.org>
	<94bdd2610903250702x6b218369jc7175067a40f39ad@mail.gmail.com>
Message-ID: <397DDCBC-D282-454A-897A-A999381D4324@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 25, 2009, at 9:02 AM, Tarek Ziad? wrote:

> Yes but this means that you have to wait for the next version of the
> "big" package
> when a bug is corrected or a feature added, or you need to patch it.
> (or maybe use the namespace trick to override it)
>
> Having them as separated package allow different release cycles,  
> which makes it
> more "agile".
>
> I think there should be a right middle between a big package and a  
> myriad of
> small ones.
>
> In Zope for instance, we could probably reunite a lot of package in  
> a single
> package, because they don't evolve much anymore. (things like  
> zope.interface)

Maybe there's a difference between being a Zope user and using zope  
packages?  I think it's great that I can pick and choose  
zope.interfaces and other packages in my not-Zope project.  But if I'm  
deploying actual Zope-the-web-thing I just want to install it from my  
distro and be done with it.  It's up to my distro to determine  
compatibility, handle bug and security fixing, etc.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScpFj3EjvBPtnXfVAQJakwP/djOrhpyZZrrqCFNG7rAi3qpZIaVLXiwk
U/td3R6OyugcDrnf26+xpaWGjr5HiJsftusRdVPbZmR9kFNmOc15lmJ1b3QElpv5
dofRiQ8hMXqe+sgCm9QmRj5RyszNlALdbYKkjCMODrcKqzC1N6fPjNi03FloWKmd
N89NYb75CUo=
=dmRY
-----END PGP SIGNATURE-----

From pje at telecommunity.com  Wed Mar 25 16:00:48 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 25 Mar 2009 11:00:48 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <ca471dc20903250740w40edc466h937be86e80e30928@mail.gmail.co
 m>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<24ea26600903250632r69d34c3bifede13fdbeb83736@mail.gmail.com>
	<20090325142855.BBC713A40A5@sparrow.telecommunity.com>
	<ca471dc20903250740w40edc466h937be86e80e30928@mail.gmail.com>
Message-ID: <20090325145826.6601B3A40A5@sparrow.telecommunity.com>

At 07:40 AM 3/25/2009 -0700, Guido van Rossum wrote:
>Well, sorry, but this complex layered interdependent architecture is
>one of the *causes* of confusion -- apart from you nobody knows what
>is what exactly,

I'll pick a minor nit here...  buildout, pip, and a wide variety of 
other tools and frameworks out there wouldn't exist if I were really 
the only person who knew "what's what".  And I believe most of those 
people will be at the summit.

That having been said, the setuptools documentation definitely sucks 
for the casual reader or curious observer, as opposed to those who 
have serious itches to scratch in the area of dependencies or 
plugins.  Combine that with the practical-but-impure choices I made 
in easy_install to get *something* working, and you have a recipe for 
the current situation.

pkg_resources, for example, is only bundled with setuptools because 
it couldn't go in the stdlib when it was written.  easy_install, 
OTOH, is bundled with setuptools because *setuptools* isn't in the 
stdlib!  (And of course, both use pkg_resources.)

So ironically, setuptools is bundled in the way that it is, precisely 
*because* there's no support for dependencies in the stdlib...  and 
nicely illustrates why smaller libraries (and less bundling) is a *good* thing.


From olemis at gmail.com  Wed Mar 25 16:11:47 2009
From: olemis at gmail.com (Olemis Lang)
Date: Wed, 25 Mar 2009 10:11:47 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <20090325142855.BBC713A40A5@sparrow.telecommunity.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<24ea26600903250632r69d34c3bifede13fdbeb83736@mail.gmail.com>
	<20090325142855.BBC713A40A5@sparrow.telecommunity.com>
Message-ID: <24ea26600903250811p6f67f25ag5b6f1ff15dcafdd2@mail.gmail.com>

On Wed, Mar 25, 2009 at 9:31 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 08:32 AM 3/25/2009 -0500, Olemis Lang wrote:
>>
>> Sometimes it also happens that, once one such build/packaging systems
>> is adopted, it is difficult to switch to using another one, since apps
>> (... and plugins systems ...) are frequently hard-coupled to the pkg
>> sys ?exotic features? and support (... what comes to my mind right now
>> is Trac plugins which are a little bit dependant on setuptools entry
>> points and so on ...).
>
> Please note that entry points are not coupled to easy_install.

Yes u'r right and I know ... perhaps there are better examples to
illustrate what I was saying ... which is that ... distutils
architecture is great since I can code ?my own? extension (e.g.
setuptools ;) with ?my? potentially non-standard | conflicting
features ... (e.g.  entry points ... AFAIK they'r only de-facto
standards, not in approved final PEPs ;)

Note: ... I'm not saying that I've contributed to setuptools (the ?my?
thing ...), it's just a way of talking ... hope you dont mind ... ;)

> ?They have a
> documented file format and API that's *distributed* with setuptools, but is
> not dependent on it and does not require .egg files, either.

... but Trac plugins *do require* egg files ... (AFAIK after reading
Trac docs and implementation of plugin upload from /admin/plugins, egg
cache for plugins ... and so on ...) and this is what I was talking
about ... apps (e.g. Trac) depending *today* on setuptools (e.g. egg
files ...) ... not easy_install since, in fact I could install plugins
with apt (... or similar ;) too ... although I always build egg files,
but this is my own decision BTW since I prefer to inherit configs &
plugins for all envs ... but this is a little bit OT here since its
specific to Trac ... ;)

> ?There's
> nothing stopping an independent implementation in the stdlib, ala PEP 376's
> metadata API, which is designed to be backward compatible with setuptools.

Of course there is nothing ?stopping an independent implementation in
the stdlib [...] designed to be backward compatible with setuptools.?,
but it is still not there ... AFAIK ... and that's what I was talking
about ... considering my example ...

I'd like it in there ... 2 ;)

Like I said I like the idea behind easy_install, but not the ?holes? I
mentioned ... but I wanna see filled out ... maybe its just a matter
of time ... :)

> ?It could be extended to support entry points as well.
>

... and then Trac plugins (falling-back to my example ;) will rely on
std features ... in a near future ... ok ...

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From curt at hagenlocher.org  Wed Mar 25 16:12:30 2009
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Wed, 25 Mar 2009 08:12:30 -0700
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <loom.20090325T121432-793@post.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
Message-ID: <d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>

On Wed, Mar 25, 2009 at 5:25 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> I'm not a Windows user, but I suppose it boils down to whether people are
> comfortable with the command-line or not (which even many Windows /developers/
> aren't). Since having GUIs for everything is part of the Windows philosophy,
> it's a fair expectation that Python libraries come with graphical Windows
> installers.

MSIs aren't only about the GUI; they're also about management.  For
instance, an MSI can be pushed out to clients using Active Directory.
This simplifies not only initial deployment but also subsequent
updates -- particularly when you're dealing with many clients.

--
Curt Hagenlocher
curt at hagenlocher.org

From tseaver at palladion.com  Wed Mar 25 16:34:43 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 25 Mar 2009 11:34:43 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <loom.20090325T133856-563@post.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49C88503.2030902@v.loewis.de>
	<49C886EF.80203@v.loewis.de>	<49C8C9B3.3070403@holdenweb.com>
	<49C939BA.8040206@v.loewis.de>	<49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de>	<49C99C7B.3080805@holdenweb.com>	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<loom.20090325T133856-563@post.gmane.org>
Message-ID: <gqdiul$1e5$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Antoine Pitrou wrote:
> Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
>> But I agree that the sizes of the packages are too small now, and it has gone
>> to far. Installing a web app like Plone is scary (+100 packages)
> 
> I am working on a TurboGears2-based app and I just did a count of the .egg
> packages in the virtualenv. There are 45 of them....
> 
> People should really stop splitting their work into micro-libraries (with such
> ludicrous names as "AddOns" or "Extremes", I might add (*)), and myriads of
> separately-packaged plugins (the repoze stuff). The Twisted approach is much
> saner, where you have a cohesive whole in a single package.

Monoliths have downsides:  consider that fact that the WSGI-complieant
HTTP server for Twisted languished for *years* outside the released
versions of Twisted:  IIRC, the server was released as a separate
distribution, but it was not compatible with the released versions of
the main Twisted distribution:  you had to install a snapshot / alpha of
Twisted to get the 'web2' server to work.

Repoze ships a number of pieces which may not be interesting to all users:

 - repoze.bfg is a web-application framework, loosely Zope-like, which
   reuses a fairly small subset of the Zope libraries.

 - repoze.who is an web authentication framework, which can be used
   in conjunction with *any* web application which respsects at least
   the REMOTE_USER key in the WSGI environment.  TurboGears is now
   using this piece, even though they don't use any other repoze.*
   packages.

 - The various "middleware" packages are tiny, conherent, and
   can be used ad libidem in a WSGI pipeline.

WSGI itself, like Unix pipes, favors composition of smaller pieces via a
simple API:  a given deployment can add stuff like profiling, etc. just
be adding a new "layer" in the pipeline.  Putting the profiler into a
larger package would make it less likely to be used across frameworks.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJyk8T+gerLs4ltQ4RAiVkAKCJbtyDpYYww30P2SupovVTNx39jgCgtPBF
XUEJbfXz5LHqBgpHs+hXrqk=
=cNYq
-----END PGP SIGNATURE-----


From hall.jeff at gmail.com  Wed Mar 25 16:37:08 2009
From: hall.jeff at gmail.com (Jeff Hall)
Date: Wed, 25 Mar 2009 11:37:08 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C886EF.80203@v.loewis.de> <49C8C9B3.3070403@holdenweb.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
Message-ID: <1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>

Someone asked for the input of "ordinary users" (i.e. non developers) which,
unfortunately, most of the people on this list don't fall in. My experience
with setuptools is that it's poorly documented and assumes a level of
sophistication that isn't always there. While this is fine in a lot of
arenas, installation of things shouldn't be an area where I have to guess
between multiple interpretations of documentation AND have a thorough
knowledge of some of the nuts and bolts of interdependencies. Maybe having
an "easily extensible" language is not consistent with having an "easy to
extend" language, but the current arrangement seems untenable.

-- 
Haikus are easy
Most make very little sense
Refrigerator
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090325/22de0082/attachment.htm>

From tseaver at palladion.com  Wed Mar 25 16:37:59 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 25 Mar 2009 11:37:59 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <397DDCBC-D282-454A-897A-A999381D4324@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49C8C9B3.3070403@holdenweb.com>
	<49C939BA.8040206@v.loewis.de>	<49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de>	<49C99C7B.3080805@holdenweb.com>	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>	<loom.20090325T133856-563@post.gmane.org>	<94bdd2610903250702x6b218369jc7175067a40f39ad@mail.gmail.com>
	<397DDCBC-D282-454A-897A-A999381D4324@python.org>
Message-ID: <gqdj4o$1e5$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Barry Warsaw wrote:

> Maybe there's a difference between being a Zope user and using zope  
> packages?  I think it's great that I can pick and choose  
> zope.interfaces and other packages in my not-Zope project.  But if I'm  
> deploying actual Zope-the-web-thing I just want to install it from my  
> distro and be done with it.  It's up to my distro to determine  
> compatibility, handle bug and security fixing, etc.

Historically, the distros have done a less than stellar job here, too.
They change the deployment model to such an extent that the core Zope
developers can't support the users of the distro-packaged Zopes (this is
a common complaint among *lot* of upstream projects, not just Zope:  try
getting support for a distro-packaged kernel on lkml, for instance).


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJyk/X+gerLs4ltQ4RAsZoAJ4ybFXYzYIclbI7YYHPj0PrlzvxPACZAaEC
B9Wl+B3LKwjldzZ6D2LEyHM=
=PBji
-----END PGP SIGNATURE-----


From a.badger at gmail.com  Wed Mar 25 16:46:11 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Wed, 25 Mar 2009 08:46:11 -0700
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <638F8377-6967-44B9-8ECF-F5DF80CD32BC@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>	<49C87004.2030807@holdenweb.com>	<873ad3gxdd.fsf@xemacs.org>
	<49CA3953.1080506@gmail.com>
	<638F8377-6967-44B9-8ECF-F5DF80CD32BC@python.org>
Message-ID: <49CA51C3.2080102@gmail.com>

Barry Warsaw wrote:

> Tools like setuptools, zc.buildout, etc. seem great for developers but
> not very good for distributions.  At last year's Pycon I think there was
> agreement from the Linux distributors that distutils, etc. just wasn't
> very useful for them.
> 
It's decent for modules but has limitations that we run up against
somewhat frequently.  It's a horror for applications.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090325/6e9b73d8/attachment-0001.pgp>

From pje at telecommunity.com  Wed Mar 25 17:04:52 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 25 Mar 2009 12:04:52 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <24ea26600903250811p6f67f25ag5b6f1ff15dcafdd2@mail.gmail.co
 m>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<24ea26600903250632r69d34c3bifede13fdbeb83736@mail.gmail.com>
	<20090325142855.BBC713A40A5@sparrow.telecommunity.com>
	<24ea26600903250811p6f67f25ag5b6f1ff15dcafdd2@mail.gmail.com>
Message-ID: <20090325160230.39CFA3A40A5@sparrow.telecommunity.com>

At 10:11 AM 3/25/2009 -0500, Olemis Lang wrote:
>... but Trac plugins *do require* egg files ... (AFAIK after reading
>Trac docs and implementation of plugin upload from /admin/plugins, egg
>cache for plugins ... and so on ...) and this is what I was talking
>about ... apps (e.g. Trac) depending *today* on setuptools (e.g. egg
>files ...)

Trac uses entry points to access plugins.  Those plugins do not 
require .egg files, unless Trac specifically wrote code of their own 
that requires .egg files.

If you install a plugin from source using pip or a system package 
manager, no .egg file is involved -- you'll have an .egg-info 
directory instead, alongside the installed code.  The pkg_resources 
API for accessing entry points works with either .egg files or 
.egg-info directories.


From cournape at gmail.com  Wed Mar 25 17:16:29 2009
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 26 Mar 2009 01:16:29 +0900
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <gqdj4o$1e5$2@ger.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<loom.20090325T133856-563@post.gmane.org>
	<94bdd2610903250702x6b218369jc7175067a40f39ad@mail.gmail.com>
	<397DDCBC-D282-454A-897A-A999381D4324@python.org>
	<gqdj4o$1e5$2@ger.gmane.org>
Message-ID: <5b8d13220903250916t302720a8u3f9bfadad198081f@mail.gmail.com>

On Thu, Mar 26, 2009 at 12:37 AM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Barry Warsaw wrote:
>
>> Maybe there's a difference between being a Zope user and using zope
>> packages? ?I think it's great that I can pick and choose
>> zope.interfaces and other packages in my not-Zope project. ?But if I'm
>> deploying actual Zope-the-web-thing I just want to install it from my
>> distro and be done with it. ?It's up to my distro to determine
>> compatibility, handle bug and security fixing, etc.
>
> Historically, the distros have done a less than stellar job here, too.

I don't think that's entirely accurate. For softwares which I don't
care directly as a developer, because I only use them, and don't
intend to change anything in it, the linux distribution is god's
heaven IMO. Being able to update, upgrade everything in a centralized,
predictable manner works very well.

It fails for software I am directly involved in, or maybe the layer
just below: for example, there is no way for me to get a python 2.6 on
my distribution (Ubuntu), so I cannot easily test the python projects
I am involved in for python 2.6. But for layers below, it is almost
perfect. If python, as a platform, could be as reliable as debian, I
would be extremely happy. I just don't think it is possible, because
of the huge amount of work this requires. Tools are just a small part
of it - you need a lot of discipline and work to make sure everything
fits together, and that can't happen for every python lib/application
out there.

I already mention this on the distutils ML, but IMO, the only workable
solution is to have a system which makes it *possible* for OS
distributors to package in whatever they see fit (.deb, .rpm, .dmg,
.exe, .msi). Distutils, as of today, makes it much harder than it is
for non trivial softwares (documentation, controlling what goes where,
etc...). That's something we can hope to improve. Again, I will take
the autoconf example: it has no policy, and can be used for different
kind of situations, because you can (if you want) control things in a
very fine-grained manner. Automatic, 'native' installers which are
well integrated into every system, this seems so far from reach I
don't see how this can even be considered.

cheers,

David

From olemis at gmail.com  Wed Mar 25 17:35:40 2009
From: olemis at gmail.com (Olemis Lang)
Date: Wed, 25 Mar 2009 11:35:40 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <20090325160230.39CFA3A40A5@sparrow.telecommunity.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<24ea26600903250632r69d34c3bifede13fdbeb83736@mail.gmail.com>
	<20090325142855.BBC713A40A5@sparrow.telecommunity.com>
	<24ea26600903250811p6f67f25ag5b6f1ff15dcafdd2@mail.gmail.com>
	<20090325160230.39CFA3A40A5@sparrow.telecommunity.com>
Message-ID: <24ea26600903250935n2ea18123ic23cd2acd4e403@mail.gmail.com>

On Wed, Mar 25, 2009 at 11:04 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 10:11 AM 3/25/2009 -0500, Olemis Lang wrote:
>>
>> ... but Trac plugins *do require* egg files ... (AFAIK after reading
>> Trac docs and implementation of plugin upload from /admin/plugins, egg
>> cache for plugins ... and so on ...) and this is what I was talking
>> about ... apps (e.g. Trac) depending *today* on setuptools (e.g. egg
>> files ...)
>
> Trac uses entry points to access plugins. ?Those plugins do not require .egg
> files, unless Trac specifically wrote code of their own that requires .egg
> files.
>

Yes you're right, Trac requires .egg files for local plugins installs
(... in /plugins folder ;) so that not all environments but only one
be able to use the plugin ... but that's not exactly what I'm saying,
since setuptools AFAIK *MUST* be already there ... so Trac depends on
setuptools.

> If you install a plugin from source using pip

Like you said setuptools is already there ... pip depends on
setuptools ... isn't it?

> or a system package manager,
> no .egg file is involved -- you'll have an .egg-info directory instead,
> alongside the installed code. ?The pkg_resources API for accessing entry
> points works with either .egg files or .egg-info directories.
>

Like I just said setuptools AFAIK *MUST* be there ... isnt it ? Am I
still wrong ?

BTW ... if you (or anyone else ;) suggests me another pkg ?like?
setuptools (supporting entry points and so on ...) so that I can try
how to install and use Trac plugins without setuptools + pkg_* in the
middle ... I'll be open to try it out ... and learn a little ;)

PS: Sorry if I insist ?too much? ... BTW I am not very sure about
whether this getting a little bit OT or not ... hope this isnt the
case ... but if it is, pls let me know ASAP ... ;)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From exarkun at divmod.com  Wed Mar 25 17:37:39 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Wed, 25 Mar 2009 11:37:39 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <gqdiul$1e5$1@ger.gmane.org>
Message-ID: <20090325163739.12853.340427671.divmod.quotient.47942@henry.divmod.com>

On Wed, 25 Mar 2009 11:34:43 -0400, Tres Seaver <tseaver at palladion.com> wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>Antoine Pitrou wrote:
>> Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
>>> But I agree that the sizes of the packages are too small now, and it has gone
>>> to far. Installing a web app like Plone is scary (+100 packages)
>>
>> I am working on a TurboGears2-based app and I just did a count of the .egg
>> packages in the virtualenv. There are 45 of them....
>>
>> People should really stop splitting their work into micro-libraries (with such
>> ludicrous names as "AddOns" or "Extremes", I might add (*)), and myriads of
>> separately-packaged plugins (the repoze stuff). The Twisted approach is much
>> saner, where you have a cohesive whole in a single package.
>
>Monoliths have downsides:  consider that fact that the WSGI-complieant
>HTTP server for Twisted languished for *years* outside the released
>versions of Twisted:  IIRC, the server was released as a separate
>distribution, but it was not compatible with the released versions of
>the main Twisted distribution:  you had to install a snapshot / alpha of
>Twisted to get the 'web2' server to work.

Maybe monoliths have downsides, but please pick a different example to
support this.  The manner in which the WSGI server in Twisted Web2 was
made available has very little to do with large vs small packages and
much to do with our (the Twisted developers) decision /not/ to release
Twisted Web2 at all.  I could go into lots more detail about that
decision, but I don't think any of i  would be relevant to the topic at
hand.

If anything, Twisted's example shows how monolithic packages are easier
all-around than micro-packages.  We basically have the release infrastructure
to release Twisted in many smaller pieces, and we even do - but we only
make releases of all the smaller pieces simultaneously, we encourage people
to use the "Twisted" package which includes all the pieces, and we don't
do any testing of mixed versions of the smaller pieces because it would be
very difficult.

Further, we *have* done point releases of /all/ of Twisted to supply some
critical bug fix (these generally take the form of a X.Y.1 release, we
rarely go to .2 or higher these days).  And we've done these quite rapidly
when the need arises - the monolithic nature of Twisted isn't really a
hindrance here (updating our website, a manual process, is far more time
consuming than actually doing the release - and a big part of that cost
comes from the fact that we have web pages for each smaller piece, even
though we don't encourage people to use these!).

So, as long as we're just talking about the vagueries of "monolithic" vs
"micro", I'll weigh in in favr of monolithic (but I'm open to discussion
about specific issues which are much more interesting than abstractions).

Jean-Paul

From db3l.net at gmail.com  Wed Mar 25 17:50:31 2009
From: db3l.net at gmail.com (David Bolen)
Date: Wed, 25 Mar 2009 12:50:31 -0400
Subject: [Python-Dev] Test failures under Windows?
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net>
	<49C9EEB5.2090804@gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>
	<d2155e360903250553u2e134d61rc005e8817ad82a60@mail.gmail.com>
Message-ID: <m2ab791r8o.fsf@valheru.db3l.homeip.net>

Curt Hagenlocher <curt at hagenlocher.org> writes:

> The variation that goes through assert.c should write to stderr for a
> console-mode application, so it's reasonable to assume that we're
> hitting the other code path -- and that Mark's suggestion to use
> CrtSetReportMode would address the issue.

Which is pretty much what the regrtest.py -n option does/did in the
py3k branch (and release30-maint).  It enables the following code:

        import msvcrt
        msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS|
                msvcrt.SEM_NOALIGNMENTFAULTEXCEPT|
                msvcrt.SEM_NOGPFAULTERRORBOX|
                msvcrt.SEM_NOOPENFILEERRORBOX)
        try:
            msvcrt.CrtSetReportMode
        except AttributeError:
            # release build
            pass
        else:
            for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]:
                msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE)
                msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR)

which seems like it would still be the right sort of code to use.  For
my buildbot the SetErrorMode is redundant because I already do that in the
buildbot code itself (and the setting is inherited) but it won't hurt.

>From what I can see though, the tools/buildbot/test.bat file no longer
adds the -n option that it used to, although I'm unclear on why it
might have been removed.  Perhaps this was just a regression that was
accidentally missed, as it appears to have disappeared during a large
merger from trunk to the py3k branch mid-2008 (r64273) when the batch
file line ending was changed to CRLF.

It would be nice to also have this in the other active release branches.

-- David


From tjreedy at udel.edu  Wed Mar 25 18:29:25 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 25 Mar 2009 13:29:25 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com>	<49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com>	<49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
Message-ID: <gqdplk$rfh$1@ger.gmane.org>

Paul Moore wrote:
> 2009/3/25 Tarek Ziad? <ziade.tarek at gmail.com>:

> Since setuptools came on the scene, I can state with some certainty
> that many packages which would otherwise have been distributed as
> bdist_wininst installers, now aren't. In some cases, only source
> packages are provided (on the basis that easy_install will build what
> you need). In those cases, I can accept that maybe the developer would
> not have built Windows installers even before setuptools arrived. But
> in a significant number of cases - including setuptools itself!!!! -
> binary, version-specific eggs for Windows are provided, but no
> bdist_wininst installers. If the developer is willing to build an egg,
> he could just as easily have built an installer - but he now has to
> choose - build one or the other, or both. And not everyone chooses the
> same way.

Is it possible to write an egg to bdist converter (or vice versa)?


From tjreedy at udel.edu  Wed Mar 25 18:35:14 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 25 Mar 2009 13:35:14 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <loom.20090325T133856-563@post.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49C88503.2030902@v.loewis.de>
	<49C886EF.80203@v.loewis.de>	<49C8C9B3.3070403@holdenweb.com>
	<49C939BA.8040206@v.loewis.de>	<49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de>	<49C99C7B.3080805@holdenweb.com>	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<loom.20090325T133856-563@post.gmane.org>
Message-ID: <gqdq0h$ta3$1@ger.gmane.org>

Antoine Pitrou wrote:
> Tarek Ziad? <ziade.tarek <at> gmail.com> writes:
>> But I agree that the sizes of the packages are too small now, and it has gone
>> to far. Installing a web app like Plone is scary (+100 packages)
> 
> I am working on a TurboGears2-based app and I just did a count of the .egg
> packages in the virtualenv. There are 45 of them....
> 
> People should really stop splitting their work into micro-libraries (with such
> ludicrous names as "AddOns" or "Extremes", I might add (*)), and myriads of
> separately-packaged plugins (the repoze stuff). The Twisted approach is much
> saner, where you have a cohesive whole in a single package.
> 
> (*) The package descriptions are worth their salt too:
> http://pypi.python.org/pypi/AddOns
> http://pypi.python.org/pypi/Extremes

Thoughts:

1. These are modules, not packages, hence should be relatively small.

2. These are general utilities and have nothing in particular to do with 
peak.

3. If I wanted one of these, I would be happy that they are separately 
indexed and downloadable.

4. If I used one of these in an app, I would include them within the app 
and not have them be separate pieces.

5. Much of this discussion reminds me of the debates between lumping and 
splitting of taxonomic categories in biology.  Like that debate, it will 
continue forever.

Terry Jan Reedy



From skip at pobox.com  Wed Mar 25 18:46:22 2009
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 25 Mar 2009 12:46:22 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <loom.20090325T143347-105@post.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<loom.20090325T133856-563@post.gmane.org>
	<94bdd2610903250702x6b218369jc7175067a40f39ad@mail.gmail.com>
	<79990c6b0903250728i6e734baap3ada1ed15e3b13ff@mail.gmail.com>
	<loom.20090325T143347-105@post.gmane.org>
Message-ID: <18890.28142.360107.874577@montanaro.dyndns.org>

Is setuptools/distutils/whatever on the agenda for the tomorrow's language
summit?  Or is there some other get-together at PyCon for this?

Skip

From jnoller at gmail.com  Wed Mar 25 18:48:11 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Wed, 25 Mar 2009 12:48:11 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <18890.28142.360107.874577@montanaro.dyndns.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<loom.20090325T133856-563@post.gmane.org>
	<94bdd2610903250702x6b218369jc7175067a40f39ad@mail.gmail.com>
	<79990c6b0903250728i6e734baap3ada1ed15e3b13ff@mail.gmail.com>
	<loom.20090325T143347-105@post.gmane.org>
	<18890.28142.360107.874577@montanaro.dyndns.org>
Message-ID: <23560D58-91B5-4EA2-B6AB-79BE881B7DB0@gmail.com>

Yes it's on the agenda

On Mar 25, 2009, at 12:46 PM, skip at pobox.com wrote:

> Is setuptools/distutils/whatever on the agenda for the tomorrow's  
> language
> summit?  Or is there some other get-together at PyCon for this?
>
> Skip
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com

From steven.bethard at gmail.com  Wed Mar 25 20:05:19 2009
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 25 Mar 2009 12:05:19 -0700
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
Message-ID: <d11dcfba0903251205n6e59ba0cl54aa456f486b41b7@mail.gmail.com>

On Wed, Mar 25, 2009 at 7:08 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> I use Python for systems admin scripts, Windows services, and database
> management. In my experience (and I agree, it's only one, limited, use
> case) availability of download-and-run bdist_wininst installers for
> every package I used was the only significant requirement I had for
> Python package distribution (I remember pre-distutils days, when being
> able to install a 3rd party package on Windows was a nightmare of
> build-it-yourself guesswork).
>
> Since setuptools came on the scene, I can state with some certainty
> that many packages which would otherwise have been distributed as
> bdist_wininst installers, now aren't. In some cases, only source
> packages are provided (on the basis that easy_install will build what
> you need). In those cases, I can accept that maybe the developer would
> not have built Windows installers even before setuptools arrived. But
> in a significant number of cases - including setuptools itself!!!! -
> binary, version-specific eggs for Windows are provided, but no
> bdist_wininst installers. If the developer is willing to build an egg,
> he could just as easily have built an installer - but he now has to
> choose - build one or the other, or both. And not everyone chooses the
> same way.

I'd just like to chime in and agree with Paul here. I'm a Windows
user, and I won't install a Python module that I can't get as a
wininst (or preferably a .msi), because I prefer to use the Windows
package management system, not some Python specific thing. I can
generally build installers myself for Python-only packages, but binary
ones are harder. And I've seen several projects with exactly the kind
of thing Paul describes - where a good Windows installer would
probably have been available if it weren't for the interference of
setuptools.

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From tleeuwenburg at gmail.com  Wed Mar 25 21:31:14 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Thu, 26 Mar 2009 07:31:14 +1100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <d11dcfba0903251205n6e59ba0cl54aa456f486b41b7@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<d11dcfba0903251205n6e59ba0cl54aa456f486b41b7@mail.gmail.com>
Message-ID: <43c8685c0903251331g6d3f8700ha94365186ba1ef60@mail.gmail.com>

GSOC?

On Thu, Mar 26, 2009 at 6:05 AM, Steven Bethard <steven.bethard at gmail.com>wrote:

> On Wed, Mar 25, 2009 at 7:08 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> > I use Python for systems admin scripts, Windows services, and database
> > management. In my experience (and I agree, it's only one, limited, use
> > case) availability of download-and-run bdist_wininst installers for
> > every package I used was the only significant requirement I had for
> > Python package distribution (I remember pre-distutils days, when being
> > able to install a 3rd party package on Windows was a nightmare of
> > build-it-yourself guesswork).
> >
> > Since setuptools came on the scene, I can state with some certainty
> > that many packages which would otherwise have been distributed as
> > bdist_wininst installers, now aren't. In some cases, only source
> > packages are provided (on the basis that easy_install will build what
> > you need). In those cases, I can accept that maybe the developer would
> > not have built Windows installers even before setuptools arrived. But
> > in a significant number of cases - including setuptools itself!!!! -
> > binary, version-specific eggs for Windows are provided, but no
> > bdist_wininst installers. If the developer is willing to build an egg,
> > he could just as easily have built an installer - but he now has to
> > choose - build one or the other, or both. And not everyone chooses the
> > same way.
>
> I'd just like to chime in and agree with Paul here. I'm a Windows
> user, and I won't install a Python module that I can't get as a
> wininst (or preferably a .msi), because I prefer to use the Windows
> package management system, not some Python specific thing. I can
> generally build installers myself for Python-only packages, but binary
> ones are harder. And I've seen several projects with exactly the kind
> of thing Paul describes - where a good Windows installer would
> probably have been available if it weren't for the interference of
> setuptools.
>
> Steve
> --
> I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
> tiny blip on the distant coast of sanity.
>        --- Bucky Katt, Get Fuzzy
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/tleeuwenburg%40gmail.com
>



-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090326/5afe3f5e/attachment-0001.htm>

From benjamin at python.org  Wed Mar 25 21:34:40 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 25 Mar 2009 15:34:40 -0500
Subject: [Python-Dev] In case I seem to disappear...
In-Reply-To: <ca471dc20903241826r1c022313y6c3acd9030d55c6a@mail.gmail.com>
References: <ca471dc20903241826r1c022313y6c3acd9030d55c6a@mail.gmail.com>
Message-ID: <1afaf6160903251334i1155428ek518c4c7f875baddd@mail.gmail.com>

Sounds better than being hit by a bus, though! :)

2009/3/24 Guido van Rossum <guido at python.org>:
> I've received several emails recently indicating that someone is
> trying to get me unsubscribed from python-dev. I'm sure they're just
> pranksters, but just in case they succeed, if I seem to be absent for
> a really long time, let me know. After all mailman passwords are not
> secure. :-)
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/benjamin%40python.org
>



-- 
Regards,
Benjamin

From ctb at msu.edu  Wed Mar 25 21:35:00 2009
From: ctb at msu.edu (C. Titus Brown)
Date: Wed, 25 Mar 2009 13:35:00 -0700
Subject: [Python-Dev] More GSoC - make sure you list your projects!
Message-ID: <20090325203500.GA13583@idyll.org>

Hi all,

just a quick note -- anyone who has an idea for a core Python Google
Summer of Code project, has not had that project panned on this list ;),
and is willing to act as a CONTACT for students who want to apply (not
necessarily a mentor!) should make sure that those ideas are posted on
the wiki, at

	http://wiki.python.org/moin/SummerOfCode/2009

Without a contact and a link on the wiki page, students won't have any
idea that the project exists...

I will endeavor to find mentors for as many of the core Python projects
as possible, once we have a clearer sense of how many applicants we have
and how many applications are good.

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu

From ziade.tarek at gmail.com  Wed Mar 25 23:15:22 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 25 Mar 2009 23:15:22 +0100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <23560D58-91B5-4EA2-B6AB-79BE881B7DB0@gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<loom.20090325T133856-563@post.gmane.org>
	<94bdd2610903250702x6b218369jc7175067a40f39ad@mail.gmail.com>
	<79990c6b0903250728i6e734baap3ada1ed15e3b13ff@mail.gmail.com>
	<loom.20090325T143347-105@post.gmane.org>
	<18890.28142.360107.874577@montanaro.dyndns.org>
	<23560D58-91B5-4EA2-B6AB-79BE881B7DB0@gmail.com>
Message-ID: <94bdd2610903251515m70ae34aah99dfa8f373ccdc2f@mail.gmail.com>

Yes,

I'll try to blog today the initial list of topics that will be
discussed during the Summit,

Regards
Tarek

2009/3/25 Jesse Noller <jnoller at gmail.com>:
> Yes it's on the agenda
>
> On Mar 25, 2009, at 12:46 PM, skip at pobox.com wrote:
>
>> Is setuptools/distutils/whatever on the agenda for the tomorrow's language
>> summit? ?Or is there some other get-together at PyCon for this?
>>
>> Skip
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From bignose+hates-spam at benfinney.id.au  Wed Mar 25 23:56:21 2009
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Thu, 26 Mar 2009 09:56:21 +1100
Subject: [Python-Dev] "setuptools has divided the Python community"
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>
Message-ID: <87myb9w6sq.fsf@benfinney.id.au>

Paul Moore <p.f.moore at gmail.com> writes:

> If you would, I'd appreciate it. Sometimes I feel that the
> distutils/setuptools discussions need better input from the
> non-web-developer community. And even more so from the "not a
> developer, just a user" community!

And the often-obscured community: those who desperately want the
Python stuff to just behave the same way everything else on their
system does, i.e. be managed approrpiately by the operating system
package manager. A Python-specific packaging system which makes it
harder to fit Python packages into a broader OS policy works very much
at odds with that.

However, I, too, am on a different continent from the conference.

-- 
 \        ?Kill myself? Killing myself is the last thing I'd ever do.? |
  `\                                            ?Homer, _The Simpsons_ |
_o__)                                                                  |
Ben Finney


From p.f.moore at gmail.com  Thu Mar 26 00:01:18 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 25 Mar 2009 23:01:18 +0000
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <gqdplk$rfh$1@ger.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<gqdplk$rfh$1@ger.gmane.org>
Message-ID: <79990c6b0903251601ra348503idb39df75a644a22e@mail.gmail.com>

2009/3/25 Terry Reedy <tjreedy at udel.edu>:
> Is it possible to write an egg to bdist converter (or vice versa)?

No idea. But would it help? Distributors would still only provide one
or the other, so when only an egg is available, I'd still have to
convert it - which is certainly pretty simple, but so is python
setup.py bdist_wininst.

I don't think it's simply about how easy it is to create installers
for yourself. If a developer provides an egg, and I convert it to a
bdist_wininst, who helps me if there's an installation problem?
(Assume "me" is someone just trying to use Python as a tool - say, one
of the scientists David Cournapeau referred to earlier in the thread).

Paul

From barry at python.org  Thu Mar 26 00:04:26 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 25 Mar 2009 18:04:26 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <5b8d13220903250916t302720a8u3f9bfadad198081f@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<loom.20090325T133856-563@post.gmane.org>
	<94bdd2610903250702x6b218369jc7175067a40f39ad@mail.gmail.com>
	<397DDCBC-D282-454A-897A-A999381D4324@python.org>
	<gqdj4o$1e5$2@ger.gmane.org>
	<5b8d13220903250916t302720a8u3f9bfadad198081f@mail.gmail.com>
Message-ID: <8B2DAD3A-4B6F-4EAF-AF72-0AD185CADD71@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 25, 2009, at 11:16 AM, David Cournapeau wrote:

> It fails for software I am directly involved in, or maybe the layer
> just below: for example, there is no way for me to get a python 2.6 on
> my distribution (Ubuntu), so I cannot easily test the python projects
> I am involved in for python 2.6.

At least it's better now so that if you install it from source and  
throw a bunch of modules into site-packages, you won't kill your  
distro. :)

> I already mention this on the distutils ML, but IMO, the only workable
> solution is to have a system which makes it *possible* for OS
> distributors to package in whatever they see fit (.deb, .rpm, .dmg,
> .exe, .msi). Distutils, as of today, makes it much harder than it is
> for non trivial softwares (documentation, controlling what goes where,
> etc...). That's something we can hope to improve. Again, I will take
> the autoconf example: it has no policy, and can be used for different
> kind of situations, because you can (if you want) control things in a
> very fine-grained manner. Automatic, 'native' installers which are
> well integrated into every system, this seems so far from reach I
> don't see how this can even be considered.

I agree.  With my developer's hat on, I can't know all the crazy  
places my package will be used, so distutils is great because at least  
it's easy for me to write a setup.py, push my package to some place,  
and share it with other developers.  My hope of course is that the  
setup.py is useful to distros who want to include my package in their  
offering.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScq4enEjvBPtnXfVAQIOpAP/bJsMn7iKyIOhm8iGxWfeNhtaesCXy2MZ
eU8KyI0xvtACleLzzZd8mX79QyNw02WTfct6BX83NHuBs2Kyu6S0jjNMfz6zzSNs
Se4qjQTu8Iw/JPCyxq6XOiTQaVkbtSNs93xAWjHfeX6PumkKn+qd5Lx3KuBPDE+X
vcTPOlq0X20=
=UsYX
-----END PGP SIGNATURE-----

From tleeuwenburg at gmail.com  Thu Mar 26 00:06:07 2009
From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg)
Date: Thu, 26 Mar 2009 10:06:07 +1100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <87myb9w6sq.fsf@benfinney.id.au>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>
	<87myb9w6sq.fsf@benfinney.id.au>
Message-ID: <43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>

I would suggest there may be three use cases for Python installation tools.
Bonus -- I'm not a web developer! :)
Case One: Developer wishing to install additional functionality into the
system Python interpreter forever
Case Two: Developer wishing to install additional functionality into the
system Python interpreter for a specific task
Case Three: Person wanting to install an application which happens to be
written in Python

The prime limitation of setuptools appears to me to come when there are
conflicts. Other than that, for packages where it is available, I have never
had an issue with the functioning of setuptools. It's Just Fine. For issues
where there are conflicts, where I have been sufficiently motivated, setting
up a virtualenv has more than met my needs. In fact, that's and *awesome*
piece of functionality.

For case one, where I want to install additional functionality into my
system Python interpreter "forever", it would be great to have my system
manage this. On my ubuntu machine, this happens. The Ubuntu packages take
care of things very satisfactorily and I don't see how anyone would have a
problem with it. I don't know what the situation is under Windows, however
systems such as the Enthought Python Suite do provide kitchen-sink-included
Python distributions.

Perhaps an automated updates site could be configured such that there was a
package for a variety of "Python + some extra modules" scenarios.

For installing applications, I would presume that app developers would
generally take care of this themselves, through the use of an MSI or .deb
file in order to protect their dependencies somewhat.

For anyone looking to extend their interpreter for some specific task,
virtualenv and setuptools should do the job, and the developer can just go
the extra mile to do the work to get it right themselves.

Perhaps the most under-served use case is people who want a
kitchen-sink-included distribution, to be managed by their system packages.
Either they are not away of systems such as EPS or are opposed to it in
principle. Were time and effort no obstacle, I would suggest that it may be
worth Python.org offering to host and/or maintain a variety of Python
distributions available for installation which include various standard
extensions. The model that eclipse takes, with a "base" system and plugins,
but with flavoured versions available (i.e. Scientic Python Distribution to
include SciPy, Numpy and MatPlotLib for example).

Thoughts?

On Thu, Mar 26, 2009 at 9:56 AM, Ben Finney <
bignose+hates-spam at benfinney.id.au <bignose%2Bhates-spam at benfinney.id.au>>wrote:

> Paul Moore <p.f.moore at gmail.com> writes:
>
> > If you would, I'd appreciate it. Sometimes I feel that the
> > distutils/setuptools discussions need better input from the
> > non-web-developer community. And even more so from the "not a
> > developer, just a user" community!
>
> And the often-obscured community: those who desperately want the
> Python stuff to just behave the same way everything else on their
> system does, i.e. be managed approrpiately by the operating system
> package manager. A Python-specific packaging system which makes it
> harder to fit Python packages into a broader OS policy works very much
> at odds with that.
>
> However, I, too, am on a different continent from the conference.
>
> --
>  \        ?Kill myself? Killing myself is the last thing I'd ever do.? |
>  `\                                            ?Homer, _The Simpsons_ |
> _o__)                                                                  |
> Ben Finney
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/tleeuwenburg%40gmail.com
>



-- 
--------------------------------------------------
Tennessee Leeuwenburg
http://myownhat.blogspot.com/
"Don't believe everything you think"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090326/b17d469b/attachment.htm>

From barry at python.org  Thu Mar 26 00:08:31 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 25 Mar 2009 18:08:31 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <24ea26600903250935n2ea18123ic23cd2acd4e403@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<24ea26600903250632r69d34c3bifede13fdbeb83736@mail.gmail.com>
	<20090325142855.BBC713A40A5@sparrow.telecommunity.com>
	<24ea26600903250811p6f67f25ag5b6f1ff15dcafdd2@mail.gmail.com>
	<20090325160230.39CFA3A40A5@sparrow.telecommunity.com>
	<24ea26600903250935n2ea18123ic23cd2acd4e403@mail.gmail.com>
Message-ID: <13C52CA7-E694-4F35-9D4A-E20D4894C26A@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 25, 2009, at 11:35 AM, Olemis Lang wrote:

> Yes you're right, Trac requires .egg files for local plugins installs
> (... in /plugins folder ;) so that not all environments but only one
> be able to use the plugin ... but that's not exactly what I'm saying,
> since setuptools AFAIK *MUST* be already there ... so Trac depends on
> setuptools.

I've not looked at Trac's plugin architecture, but I wonder, how much  
of it is generic enough to be useful for any application?  I've found  
setuptools entry points difficult to work with for plugins, and I  
think the Python ecosystem could benefit from more shared wisdom, if  
not code, in designing plugin frameworks for applications.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScq5b3EjvBPtnXfVAQIdDAP+OHWHqbYhpAuokfOQ4+dDp48aFNZMaDDs
F5IzPZA5Hjvv0r/lkdV06FCxmx24tXxkBnPSrFHbR5iK5pwFsORwf8YWib5S5IPh
DBQLAeBfM0LbzlGp1/p6Aa8xK2lAShTcNrjyWsuWYskUHp66quwQ1IzF6ROOEMjz
bUy9hxiwEx8=
=RVyF
-----END PGP SIGNATURE-----

From barry at python.org  Thu Mar 26 00:16:01 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 25 Mar 2009 18:16:01 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <20090325163739.12853.340427671.divmod.quotient.47942@henry.divmod.com>
References: <20090325163739.12853.340427671.divmod.quotient.47942@henry.divmod.com>
Message-ID: <45F53143-C2D5-44B3-9FCC-C054BF492166@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 25, 2009, at 11:37 AM, Jean-Paul Calderone wrote:

> If anything, Twisted's example shows how monolithic packages are  
> easier
> all-around than micro-packages.  We basically have the release  
> infrastructure
> to release Twisted in many smaller pieces, and we even do - but we  
> only
> make releases of all the smaller pieces simultaneously, we encourage  
> people
> to use the "Twisted" package which includes all the pieces, and we  
> don't
> do any testing of mixed versions of the smaller pieces because it  
> would be
> very difficult.

Micro-packages are really useful when you want to share parts of your  
technology without forcing adoption of the whole thing.  Zope's a good  
example of this; e.g. zope.interfaces is fabulously useful even if you  
never use Zope.

One of the things we're doing with Launchpad's open source effort is  
to find those generic useful things that we can split out and  
distribute separately.

https://launchpad.net/~lazr-developers

OTOH, once we've exhausted that opportunity, I think we'll be left  
with a rather large monolithic application.  Still useful (I hope) but  
there won't be much further decomposition into smaller sharable parts.

Does Twisted's monolithic package structure stem from a similar pattern?

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScq7MXEjvBPtnXfVAQIUFwP/efivE5nTcEF/xhL6FgfJ41/vhWiI1jKY
+FNLotOnvNzDVEjwPNBgjLh9AzaX2TMDCBhHXhFJN12bkZ3u1pvbzdc/v/hVJT3X
0tjvJ78JTH/GSu9EBfRPLAbRrTw3JV58HDUlgy1/zETviwerwtU1QN+Tf+v/zH0p
NO+OEVgfGPg=
=Njrr
-----END PGP SIGNATURE-----

From pje at telecommunity.com  Thu Mar 26 00:19:16 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 25 Mar 2009 19:19:16 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <gqdplk$rfh$1@ger.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C886EF.80203@v.loewis.de> <49C8C9B3.3070403@holdenweb.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<gqdplk$rfh$1@ger.gmane.org>
Message-ID: <20090325231754.5357C3A40A5@sparrow.telecommunity.com>

At 01:29 PM 3/25/2009 -0400, Terry Reedy wrote:
>Paul Moore wrote:
>>2009/3/25 Tarek Ziad? <ziade.tarek at gmail.com>:
>
>>Since setuptools came on the scene, I can state with some certainty
>>that many packages which would otherwise have been distributed as
>>bdist_wininst installers, now aren't. In some cases, only source
>>packages are provided (on the basis that easy_install will build what
>>you need). In those cases, I can accept that maybe the developer would
>>not have built Windows installers even before setuptools arrived. But
>>in a significant number of cases - including setuptools itself!!!! -
>>binary, version-specific eggs for Windows are provided, but no
>>bdist_wininst installers. If the developer is willing to build an egg,
>>he could just as easily have built an installer - but he now has to
>>choose - build one or the other, or both. And not everyone chooses the
>>same way.
>
>Is it possible to write an egg to bdist converter (or vice versa)?

easy_install can convert bdist_wininst installers to .egg files; I'm 
not sure if the reverse is possible in practice, although it 
certainly is in principle.


From pje at telecommunity.com  Thu Mar 26 00:24:22 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 25 Mar 2009 19:24:22 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <13C52CA7-E694-4F35-9D4A-E20D4894C26A@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<24ea26600903250632r69d34c3bifede13fdbeb83736@mail.gmail.com>
	<20090325142855.BBC713A40A5@sparrow.telecommunity.com>
	<24ea26600903250811p6f67f25ag5b6f1ff15dcafdd2@mail.gmail.com>
	<20090325160230.39CFA3A40A5@sparrow.telecommunity.com>
	<24ea26600903250935n2ea18123ic23cd2acd4e403@mail.gmail.com>
	<13C52CA7-E694-4F35-9D4A-E20D4894C26A@python.org>
Message-ID: <20090325232204.0B08D3A40A5@sparrow.telecommunity.com>

At 06:08 PM 3/25/2009 -0500, Barry Warsaw wrote:
>I've found setuptools entry points difficult to work with for plugins,

I'd be interested in hearing more about your specific difficulties, 
although it's probably off-topic for Python-Dev.  Perhaps via the 
distutils-sig, since we don't have a plugins sig?  ;-)


From pje at telecommunity.com  Thu Mar 26 00:26:32 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Wed, 25 Mar 2009 19:26:32 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <24ea26600903250935n2ea18123ic23cd2acd4e403@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<24ea26600903250632r69d34c3bifede13fdbeb83736@mail.gmail.com>
	<20090325142855.BBC713A40A5@sparrow.telecommunity.com>
	<24ea26600903250811p6f67f25ag5b6f1ff15dcafdd2@mail.gmail.com>
	<20090325160230.39CFA3A40A5@sparrow.telecommunity.com>
	<24ea26600903250935n2ea18123ic23cd2acd4e403@mail.gmail.com>
Message-ID: <20090325232424.30E973A40A5@sparrow.telecommunity.com>

At 11:35 AM 3/25/2009 -0500, Olemis Lang wrote:
>Yes you're right, Trac requires .egg files for local plugins installs
>(... in /plugins folder ;) so that not all environments but only one
>be able to use the plugin ... but that's not exactly what I'm saying,
>since setuptools AFAIK *MUST* be already there ... so Trac depends on
>setuptools.

It depends on pkg_resources, which you can manually extract from the 
setuptools package if you like.  I believe there may be some distros 
that actually do this, with separate system packages for "setuptools" 
(really pkg_resources) and setuptools-devel (the setuptools.* code) 
-- in which case you can have pkg_resources installed without the 
rest of setuptools being present.


> > If you install a plugin from source using pip
>
>Like you said setuptools is already there ... pip depends on
>setuptools ... isn't it?

Yes, but that doesn't make *Trac* dependent on setuptools, 
particularly in the system package manager case.


> > or a system package manager,
> > no .egg file is involved -- you'll have an .egg-info directory instead,
> > alongside the installed code.  The pkg_resources API for accessing entry
> > points works with either .egg files or .egg-info directories.
> >
>
>Like I just said setuptools AFAIK *MUST* be there ... isnt it ? Am I
>still wrong ?

Not on the system using the package manager, no.  (By 'setuptools' 
I'm refering to the setuptools.* modules, not the setuptools 
distribution...  which is just another byproduct of our globally 
confusing terminology for these things.)


>BTW ... if you (or anyone else ;) suggests me another pkg ?like?
>setuptools (supporting entry points and so on ...) so that I can try
>how to install and use Trac plugins without setuptools + pkg_* in the
>middle ... I'll be open to try it out ... and learn a little ;)

You'd need pkg_resources, and if you're not using setuptools to 
initially generate the files to install, you'd have to create the 
entry point files by hand, following the EggFormats documentation I 
linked to previously.


From barry at python.org  Thu Mar 26 00:25:24 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 25 Mar 2009 18:25:24 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>
	<87myb9w6sq.fsf@benfinney.id.au>
	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
Message-ID: <25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 25, 2009, at 6:06 PM, Tennessee Leeuwenburg wrote:

> For case one, where I want to install additional functionality into my
> system Python interpreter "forever", it would be great to have my  
> system
> manage this.

In fact, I think it /has/ to.  I'll go further and say that I'm very  
wary of using easy_install and the like to install non-distro provided  
packages into the system Python.  Many Linux distros require a  
functioning system Python to operate and the distros (should) take  
great care to make sure that if you install package X for application  
Y, you won't break essential system service Z.  Once you start  
installing your own stuff in the system Python's site-packages, you  
break the warranty.

There was a related problem in previous Ubuntu/Debian releases.  If  
you installed Python from source, say into the default /usr/local and  
started easy_installing stuff into there, it was possible to break  
your system Python.  This was because the system Python hacked /usr/ 
local's site-packages into sys.path.  That's now been fixed.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScq9ZXEjvBPtnXfVAQLKXwP+K1w1eI8/pdGTzNPZp4McYKPqjgOI2qtm
/LzB+g8kHokJNgHDfPu7p6xd5Sc06bQroZ74nqtVhEZKSfSsLkkMKpfH8t5bBeEd
bOC7DuqMTMwkTTtAXATCMvY1VsuYBhJQ4Glkgbv1pbmYXE+ogwDsL8sEfjI+YT4m
BzsaWPMDIXo=
=lvsP
-----END PGP SIGNATURE-----

From barry at python.org  Thu Mar 26 00:27:26 2009
From: barry at python.org (Barry Warsaw)
Date: Wed, 25 Mar 2009 18:27:26 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <20090325232204.0B08D3A40A5@sparrow.telecommunity.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<24ea26600903250632r69d34c3bifede13fdbeb83736@mail.gmail.com>
	<20090325142855.BBC713A40A5@sparrow.telecommunity.com>
	<24ea26600903250811p6f67f25ag5b6f1ff15dcafdd2@mail.gmail.com>
	<20090325160230.39CFA3A40A5@sparrow.telecommunity.com>
	<24ea26600903250935n2ea18123ic23cd2acd4e403@mail.gmail.com>
	<13C52CA7-E694-4F35-9D4A-E20D4894C26A@python.org>
	<20090325232204.0B08D3A40A5@sparrow.telecommunity.com>
Message-ID: <21B8D18A-B79E-44B6-AA97-07237004AF9D@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 25, 2009, at 6:24 PM, P.J. Eby wrote:

> At 06:08 PM 3/25/2009 -0500, Barry Warsaw wrote:
>> I've found setuptools entry points difficult to work with for  
>> plugins,
>
> I'd be interested in hearing more about your specific difficulties,  
> although it's probably off-topic for Python-Dev.  Perhaps via the  
> distutils-sig, since we don't have a plugins sig?  ;-)

Yet...? :)  Yes, distutils-sig is probably the right place for that  
discussion.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScq93nEjvBPtnXfVAQKSJwP/XLfRTGecUBk6S4ar/2ODQ1JZ/nIljjHN
xbOQ8UKV7V1juv8Z7mrunPYVOd5DlUiv+W8jrmbBI5wtPRATGwuAfuokjkOuWjmF
Q93qvzLTs04xhLbKY9JH19YLrsAfV8sZvwxMlit7gGau3lZkySpsrUWFBf0hXEYt
PE7kjfFDymM=
=Csdq
-----END PGP SIGNATURE-----

From skip at pobox.com  Thu Mar 26 01:24:09 2009
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 25 Mar 2009 19:24:09 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>
	<87myb9w6sq.fsf@benfinney.id.au>
	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
	<25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>
Message-ID: <18890.52009.22735.514624@montanaro.dyndns.org>


    Barry> In fact, I think it /has/ to.  I'll go further and say that I'm
    Barry> very wary of using easy_install and the like to install
    Barry> non-distro provided packages into the system Python.  

Give that man a ceegar.  The pyjamas author seems to have a different
opinion about installing stuff into /usr without working with the system's
packaging mechanism:

    http://bugs.python.org/setuptools/issue63

I don't understand how that can possibly be manageable.

Skip

From bignose+hates-spam at benfinney.id.au  Thu Mar 26 02:24:08 2009
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Thu, 26 Mar 2009 12:24:08 +1100
Subject: [Python-Dev] "setuptools has divided the Python community"
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>
	<87myb9w6sq.fsf@benfinney.id.au>
Message-ID: <87iqlxvzyf.fsf@benfinney.id.au>

Ben Finney <bignose+hates-spam at benfinney.id.au> writes:

> And the often-obscured community: those who desperately want the
> Python stuff to just behave the same way everything else on their
> system does, i.e. be managed approrpiately by the operating system
> package manager. A Python-specific packaging system which makes it
> harder to fit Python packages into a broader OS policy works very
> much at odds with that.

? propos, the Debian Python infrastructure team are currently
preparing a transition to Python 2.6, with some changes in packaging
of distributions using distutils and setuptools.

    Message-ID: <20090325212256.GE3765 at piotro.eu>
    <URL:http://lists.debian.org/msgid-search/20090325212256.GE3765 at piotro.eu>

-- 
 \           ?I do not believe in immortality of the individual, and I |
  `\        consider ethics to be an exclusively human concern with no |
_o__)  superhuman authority behind it.? ?Albert Einstein, letter, 1953 |
Ben Finney


From stephen at xemacs.org  Thu Mar 26 03:26:46 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 26 Mar 2009 11:26:46 +0900
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <43c8685c0903251331g6d3f8700ha94365186ba1ef60@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<d11dcfba0903251205n6e59ba0cl54aa456f486b41b7@mail.gmail.com>
	<43c8685c0903251331g6d3f8700ha94365186ba1ef60@mail.gmail.com>
Message-ID: <87ocvpxbmh.fsf@xemacs.org>

Tennessee Leeuwenburg writes:
 > GSOC?

No.  This is territory that nobody knows how to mentor yet, ya know?
Try it now, and the poor student is likely to find him or herself at
the center of a firestorm!

Maybe next year ....

From ziade.tarek at gmail.com  Thu Mar 26 03:29:04 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 26 Mar 2009 03:29:04 +0100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <87ocvpxbmh.fsf@xemacs.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<d11dcfba0903251205n6e59ba0cl54aa456f486b41b7@mail.gmail.com>
	<43c8685c0903251331g6d3f8700ha94365186ba1ef60@mail.gmail.com>
	<87ocvpxbmh.fsf@xemacs.org>
Message-ID: <94bdd2610903251929h6ba81e5gdf37e921fdb8c941@mail.gmail.com>

On Thu, Mar 26, 2009 at 3:26 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Tennessee Leeuwenburg writes:
> ?> GSOC?
>
> No. ?This is territory that nobody knows how to mentor yet, ya know?
> Try it now, and the poor student is likely to find him or herself at
> the center of a firestorm!

No, we wil
>
> Maybe next year ....
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com
>



-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ziade.tarek at gmail.com  Thu Mar 26 03:33:40 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 26 Mar 2009 03:33:40 +0100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <94bdd2610903251929h6ba81e5gdf37e921fdb8c941@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<d11dcfba0903251205n6e59ba0cl54aa456f486b41b7@mail.gmail.com>
	<43c8685c0903251331g6d3f8700ha94365186ba1ef60@mail.gmail.com>
	<87ocvpxbmh.fsf@xemacs.org>
	<94bdd2610903251929h6ba81e5gdf37e921fdb8c941@mail.gmail.com>
Message-ID: <94bdd2610903251933u34ba0f96obdc4e2781a1ea283@mail.gmail.com>

On Thu, Mar 26, 2009 at 3:29 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Thu, Mar 26, 2009 at 3:26 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
>> Tennessee Leeuwenburg writes:
>> ?> GSOC?
>>
>> No. ?This is territory that nobody knows how to mentor yet, ya know?
>> Try it now, and the poor student is likely to find him or herself at
>> the center of a firestorm!
>
> No, we wil

oups, sent to early sorry ->

No, we will hopefully have great PEPs after the language summit
tomorrow so the students will be OK ;)

There's already a GSOC topic for Disutils about building a distributed
test system using PyPI,
and I'm hoping to add more topics on packaging after the Summit.

Cheers,
Tarek

From steve at holdenweb.com  Thu Mar 26 03:53:26 2009
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 25 Mar 2009 22:53:26 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <18890.52009.22735.514624@montanaro.dyndns.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>	<87myb9w6sq.fsf@benfinney.id.au>	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>	<25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>
	<18890.52009.22735.514624@montanaro.dyndns.org>
Message-ID: <gqeqn9$1tq$1@ger.gmane.org>

skip at pobox.com wrote:
>     Barry> In fact, I think it /has/ to.  I'll go further and say that I'm
>     Barry> very wary of using easy_install and the like to install
>     Barry> non-distro provided packages into the system Python.  
> 
> Give that man a ceegar.  The pyjamas author seems to have a different
> opinion about installing stuff into /usr without working with the system's
> packaging mechanism:
> 
>     http://bugs.python.org/setuptools/issue63
> 
> I don't understand how that can possibly be manageable.
> 
Note that the issue contains a broken link.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From ncoghlan at gmail.com  Thu Mar 26 04:02:08 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Mar 2009 13:02:08 +1000
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com>	<49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com>	<49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
Message-ID: <49CAF030.80707@gmail.com>

Jeff Hall wrote:
> Someone asked for the input of "ordinary users" (i.e. non developers)
> which, unfortunately, most of the people on this list don't fall in. My
> experience with setuptools is that it's poorly documented and assumes a
> level of sophistication that isn't always there. While this is fine in a
> lot of arenas, installation of things shouldn't be an area where I have
> to guess between multiple interpretations of documentation AND have a
> thorough knowledge of some of the nuts and bolts of interdependencies.
> Maybe having an "easily extensible" language is not consistent with
> having an "easy to extend" language, but the current arrangement seems
> untenable.

I'm mostly reading this thread as an interested observer, but the
impression I'm getting is that setuptools and friends are really
*packaging tools* (designed to allow developers to communicate
information to packagers and distributors) that have been pressed into
service as *distribution tools* (designed to deliver software to end users).

That is, the full workflow that should really be happening is something
like the following:

                     Developer(s)
                          |
                          V
         (distutils/setuptools/pip/zc.buildout/etc)
                          |
                          V
             <Common Python packaging spec>
                          |
                          V
              Packager(s)/Distributor(s)
                          |
                          V
  (Windows MSI/Mac OS X bundle/Linux RPM/Linux DEB/etc)
                          |
                          V
                 End User Installation

But what appears to be currently happening is that the second half of
that process is being skipped in many cases and the "<common packaging
spec>" is instead being used as a substitute for the appropriate
platform-specific installation process.

This works fine in a centrally hosted web app development world where
the "distribution to end users" step essentially amounts to "point your
web browser at <this URL>". But for other kinds of software that is
actually installed on an end user's machine it is going to provide a
lousy installation experience.

If that perception is accurate, then any changes likely need to focus on
the *opposite* end of the toolchain: the part between the "<common
packaging spec>" and the end users.

In other words: Given an egg, how easy is it for a packager/distributor
to create a platform specific package that places the files in the
correct locations for that particular platform (regardless of how
arbitrary those rules may appear to the original developers)?

Also-not-on-the-right-continent-for-PyCon'ly,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From martin at v.loewis.de  Thu Mar 26 04:26:51 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 25 Mar 2009 22:26:51 -0500
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <638F8377-6967-44B9-8ECF-F5DF80CD32BC@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49BA3154.8080408@simplistix.co.uk>	<49BAA596.5020106@v.loewis.de>	<49C79C1A.8040301@simplistix.co.uk>	<49C7FC85.5000809@v.loewis.de>	<49C80FA0.4020800@simplistix.co.uk>	<87ab7bh5fb.fsf@xemacs.org>	<49C87004.2030807@holdenweb.com>	<873ad3gxdd.fsf@xemacs.org>
	<49CA3953.1080506@gmail.com>
	<638F8377-6967-44B9-8ECF-F5DF80CD32BC@python.org>
Message-ID: <49CAF5FB.80401@v.loewis.de>

> Tools like setuptools, zc.buildout, etc. seem great for developers but 
> not very good for distributions.  At last year's Pycon I think there was 
> agreement from the Linux distributors that distutils, etc. just wasn't 
> very useful for them.

I think distutils is different here - it not only helps creating
distributions (i.e. deployable package files), but also allows
direct installation. This, in turn, is used to build the packages
for Linux distributions. E.g. debian/rules often contains a
"setup.py install" call in its build step (and there is even a
CDBS python-distutils.mk fragment)

In that sense, distutils is for Python what make is for C.

Regards,
Martin


From skip at pobox.com  Thu Mar 26 04:27:11 2009
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 25 Mar 2009 22:27:11 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <gqeqn9$1tq$1@ger.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>
	<87myb9w6sq.fsf@benfinney.id.au>
	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
	<25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>
	<18890.52009.22735.514624@montanaro.dyndns.org>
	<gqeqn9$1tq$1@ger.gmane.org>
Message-ID: <18890.62991.850960.326187@montanaro.dyndns.org>


    >> http://bugs.python.org/setuptools/issue63
    >> 
    >> I don't understand how that can possibly be manageable.
    >> 
    Steve> Note that the issue contains a broken link.

Fixed.  Looks like a Roundup bug.

Skip

From guido at python.org  Thu Mar 26 04:34:31 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Mar 2009 20:34:31 -0700
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
	currently conform
In-Reply-To: <49CA27BD.7080904@gmail.com>
References: <b8ad139e0903231352i66bc38f3v4f88b0914c760df7@mail.gmail.com>
	<b8ad139e0903231400h570d771bt588ccbc71a295db4@mail.gmail.com>
	<ca471dc20903241314t318f2fb2vf864dae094592f53@mail.gmail.com>
	<49CA27BD.7080904@gmail.com>
Message-ID: <ca471dc20903252034k332f7c10nfae0d51a9a852e04@mail.gmail.com>

On Wed, Mar 25, 2009 at 5:46 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Guido van Rossum wrote:
>> Please don't do this. We need stable APIs. Trying to switch the entire
>> community to use CapWord APIs for something as commonly used as
>> datetime sounds like wasting a lot of cycles with no reason except the
>> mythical "PEP 8 conformance". As I said, it's a pity we didn't change
>> this at the 3.0 point, but I think going forward we should try to be
>> more committed to slow change. Additions of new functionality are of
>> course fine. But renamings (even if the old names remain available)
>> are just noise.
>
> Even for 3.0, the only API I can recall doing this for was the threading
> module, and there we had the additional motivation of being able to add
> multiprocessing with only a PEP 8 compliant API while still having it be
> close to a drop-in replacement for the corresponding threading API.
>
> Having helped with that kind of rename once (and for a relatively small
> API at that), I'd want a *really* compelling reason before ever going
> through it again - it's messy, tedious and a really good way to burn
> volunteer time without a great deal to show for it at the end.

My first response was "in hindsight we shouldn't have done this." But
we moved a bunch of other modules around too (urllib, http, db, I
forget what else) and I think those worked out well. Why was threading
particularly unpleasant? (An no, this isn't a rhetorical question or a
retort. I'm just curious -- I have the same feeling but can't pin it
down.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From kevin at bud.ca  Thu Mar 26 04:44:13 2009
From: kevin at bud.ca (Kevin Teague)
Date: Wed, 25 Mar 2009 20:44:13 -0700
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <loom.20090325T121432-793@post.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<87ab7bh5fb.fsf@xemacs.org> <49C87004.2030807@holdenweb.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
Message-ID: <1B310D4A-75AF-4728-8849-4B9D744CC529@bud.ca>


On Mar 25, 2009, at 5:25 AM, Antoine Pitrou wrote:

> Paul Moore <p.f.moore <at> gmail.com> writes:
>>
>> 3. Setuptools, unfortunately, has divided the Python distribution
>> community quite badly.
>
> Wait a little bit, and it's gonna be even worse, now that buildout  
> and pip seem
> to become popular. For example, the TurboGears people are  
> considering switching
> from setuptools to pip...
>
> Tarek is now doing a lot of very useful work on distutils (thanks  
> Tarek!), but
> I'm not sure it'll be enough to stop people from converting to  
> whatever of the
> many build/packaging systems which have been popping up recently.  
> Combined with
> the current trend that everything must be exploded into lots of  
> interdependent
> but separately packaged libraries (the paste/pylons syndrome... try  
> pulling
> something like TurboGears2 and see how many third-party packages it  
> installs), I
> fear this is going to generate a very painful user/developer  
> experience :-(
>


 From my perspective (the Python web community), setuptools has been a  
great uniter of community.

By making it easier to break-up large projects into smaller  
distributions, it's now reasonable to share more code between more  
developers. This is an important philosophy of TurboGears 2, in that  
they're bringing together different pieces of Python web parts and  
composing them into a single framework. I think it's great that when  
I'm working on a Zope-based project I can participate in packages  
produced by the non-Zope ecosystem. This builds much greater bridges  
across sub-communites than smaller barriers put up such as one project  
using pip and another using easy_install. Heck, even when only wanting  
to share code between a couple personal projects, the barrier is now  
low enough that it's reasonable to package up that code and share a  
distribution, when in the past the only way to share that code was  
nasty copy-n-paste.

Yes, there is pain in learning to use these tools, but it's mostly a  
documentation issue. Once you've learned the tools, it can be very  
easy to manage a very large amount of packages. Thanks to Buildout  
(luuuuuv the Buildout!) it's so much easier today to deploy a Zope- 
based application today than it was 5 years ago, even though a Zope- 
based application 5 years ago used a single monolithic Zope  
distribution and a Zope-based application today is composed of many,  
many smaller distributions. I'm very happy that I can so easily  
incorporate 3rd party libraries into my projects, it makes for a very  
pleasant developer experience :-)


From ziade.tarek at gmail.com  Thu Mar 26 04:58:51 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 26 Mar 2009 04:58:51 +0100
Subject: [Python-Dev] Packaging Survey first results + Summit schedule
Message-ID: <94bdd2610903252058n1a15daa7ic87cd32c7570ba81@mail.gmail.com>

Hi,

Sorry for the cross-post, but it seemed appropriate since packaging is
being discussed in python-dev tonight,

- Here are the first results for the packaging survey:
http://tarekziade.wordpress.com/2009/03/26/packaging-survey-first-results/
- And tomorrow's Summit schedule for the packaging part :
http://tarekziade.wordpress.com/2009/03/26/pycon-language-summit-is-tomorrow/

Please comment (in the appropriate list or in my blog) if you have
something you would like to say or see addressed
during the Summit and you are unable to be present.

(I am already trying to summarize what has been said in python-dev
today but I am not sure I'll be able to read everything
 before tomorrow)

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From cournape at gmail.com  Thu Mar 26 05:19:01 2009
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 26 Mar 2009 13:19:01 +0900
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CAF030.80707@gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
Message-ID: <5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>

On Thu, Mar 26, 2009 at 12:02 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> If that perception is accurate, then any changes likely need to focus on
> the *opposite* end of the toolchain: the part between the "<common
> packaging spec>" and the end users.

Yes - but is this part the job of python ?

> In other words: Given an egg, how easy is it for a packager/distributor
> to create a platform specific package that places the files in the
> correct locations for that particular platform (regardless of how
> arbitrary those rules may appear to the original developers)?

Why coming from eggs and not from the build tool provided by python
itself (distutils) ? I don't see what eggs brings - specially since
the format is not even standardized.

cheers,

David

From guido at python.org  Thu Mar 26 05:24:21 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Mar 2009 21:24:21 -0700
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090325142405.C68463A40A7@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<20090325142405.C68463A40A7@sparrow.telecommunity.com>
Message-ID: <ca471dc20903252124o7635e11cvd047fc9d1de63370@mail.gmail.com>

ISTR that the motivation for adding new syntax is that the best you
can do using a trampoline library is still pretty cumbersome to use
when you have to write a lot of tasks and subtasks, and when using
tasks is just a tool for getting things done rather than an end goal
in itself. I agree that the motivation and the comparison should be
added to the PEP (perhaps moving the trampoline sample
*implementation* to a reference or an appendix, since it is only the
appearance of the trampoline-*using* code that matters).

--Guido

On Wed, Mar 25, 2009 at 7:26 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 06:03 PM 3/25/2009 +1200, Greg Ewing wrote:
>>
>> I wanted a way of writing suspendable functions that
>> can call each other easily. (You may remember I
>> originally wanted to call it "call".) Then I noticed
>> that it would also happen to provide the functionality
>> of earlier "yield from" suggestions, so I adopted that
>> name.
>
> I still don't see what you gain from making this syntax, vs. putting
> something like this in the stdlib (rough sketch):
>
> class Task(object):
> ? ?def __init__(self, geniter):
> ? ? ? ?self.stack = [geniter]
>
> ? ?def __iter__(self):
> ? ? ? ?return self
>
> ? ?def send(self, value=None):
> ? ? ? ?if not self.stack:
> ? ? ? ? ? ?raise RuntimeError("Can't resume completed task")
> ? ? ? ?return self._step(value)
>
> ? ?send = next
>
> ? ?def _step(self, value=None, exc_info=()):
> ? ? ? ?while self.stack:
> ? ? ? ? ? ?try:
> ? ? ? ? ? ? ? ?it = self.stack[-1]
> ? ? ? ? ? ? ? ?if exc_info:
> ? ? ? ? ? ? ? ? ? ?try:
> ? ? ? ? ? ? ? ? ? ? ? ?rv = it.throw(*exc_info)
> ? ? ? ? ? ? ? ? ? ?finally:
> ? ? ? ? ? ? ? ? ? ? ? ?exc_info = ()
> ? ? ? ? ? ? ? ?elif value is not None:
> ? ? ? ? ? ? ? ? ? ?rv = it.send(value)
> ? ? ? ? ? ? ? ?else:
> ? ? ? ? ? ? ? ? ? ?rv = it.next()
> ? ? ? ? ? ?except:
> ? ? ? ? ? ? ? ?value = None
> ? ? ? ? ? ? ? ?exc_info = sys.exc_info()
> ? ? ? ? ? ? ? ?if exc_info[0] is StopIteration:
> ? ? ? ? ? ? ? ? ? ?exc_info = () ? # not really an error
> ? ? ? ? ? ? ? ?self.pop()
> ? ? ? ? ? ?else:
> ? ? ? ? ? ? ? ?value, exc_info = yield_to(rv, self)
> ? ? ? ?else:
> ? ? ? ? ? ?if exc_info:
> ? ? ? ? ? ? ? ?raise exc_info[0], exc_info[1], exc_info[2]
> ? ? ? ? ? ?else:
> ? ? ? ? ? ? ? ?return value
>
> ? ?def throw(self, *exc_info):
> ? ? ? ?if not self.stack:
> ? ? ? ? ? ?raise RuntimeError("Can't resume completed task")
> ? ? ? ?return self._step(None, exc_info)
>
> ? ?def push(self, geniter):
> ? ? ? ?self.stack.append(geniter)
> ? ? ? ?return None, ()
>
> ? ?def pop(self, value=None):
> ? ? ? ?if self.stack:
> ? ? ? ? ? ?it = self.stack.pop()
> ? ? ? ? ? ?if hasattr(it, 'close'):
> ? ? ? ? ? ? ? ?try:
> ? ? ? ? ? ? ? ? ? ?it.close()
> ? ? ? ? ? ? ? ?except:
> ? ? ? ? ? ? ? ? ? ?return None, sys.exc_info()
> ? ? ? ?return value, ()
>
> ? ?@classmethod
> ? ?def factory(cls, func):
> ? ? ? ?def decorated(*args, **kw):
> ? ? ? ? ? ?return cls(func(*args, **kw))
> ? ? ? ?return decorated
>
>
> def yield_to(rv, task):
> ? ?# This could/should be a generic function, to allow yielding to
> ? ?# deferreds, sockets, timers, and other custom objects
> ? ?if hasattr(rv, 'next'):
> ? ? ? ?return task.push(rv)
> ? ?elif isinstance(rv, Return):
> ? ? ? ?return task.pop(rv.value)
> ? ?else:
> ? ? ? ?return rv, ()
>
> class Return(object):
> ? ?def __init__(self, value=None):
> ? ? ? ?self.value = value
>
>
> @Task.factory
> def sample_task(arg1, another_arg):
> ? ?# blah blah
> ? ?something = (yield subtask(...))
>
> ? ?yield Return(result)
>
> def subtask(...):
> ? ?...
> ? ?yield Return(myvalue)
>
>
> The trampoline (the _step() method) handles the co-operative aspects, and
> modifying the yield_to() function allows you to define how yielded values
> are processed. ?By default, they're sent back into the generator that yields
> them, but you can pass a Return() to terminate the generator and pass the
> value up to the calling generator. ?Yielding another generator, on the other
> hand, "calls" that generator within the current task, and the same rules
> apply.
>
> Is there some reason why this won't do what you want, and can't be modified
> to do so? ?If so, that should be part of the PEP, as IMO it otherwise lacks
> motivation for a language feature vs. say, a stdlib module. ?If 'yield_to'
> is a generic function or at least supports registration of some kind, a
> feature like this would be interoperable with a wide variety of frameworks
> -- you could register deferreds and delayed calls and IO objects from
> Twisted, for example. ?So it's not like the feature would be creating an
> entire new framework of its own. ?Rather, it'd be a front-end to whatever
> framework (or no framework) you're using.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Mar 26 05:26:03 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Mar 2009 21:26:03 -0700
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CA3021.5050905@gmail.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz> <gqc4id$ud8$1@ger.gmane.org>
	<20090325033249.C8A3F3A4097@sparrow.telecommunity.com>
	<49C9CBFF.3080705@canterbury.ac.nz> <49CA3021.5050905@gmail.com>
Message-ID: <ca471dc20903252126t31b03dib87d47d27c4700ed@mail.gmail.com>

On Wed, Mar 25, 2009 at 6:22 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Greg Ewing wrote:
>> Would you be happier if some syntactic way to do that
>> were provided?
>>
>> It could perhaps be done by enhancing the part of the
>> 'for' loop that gets executed upon normal termination
>> of the iterator.
>>
>> ? for x in my_iter:
>> ? ? do_something_with(x)
>> ? else v:
>> ? ? handle_return_value(v)
>
> I think something like that would actually make the PEP much stronger on
> this front - it would promote the idea of a "final value" for iterators
> as a more fundamental concept that can be worked with in a non-generator
> context.

Hold it right there. Or maybe I should say "in your dreams." Please
don't stretch the scope of the PEP. It's not going to help your cause.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From cournape at gmail.com  Thu Mar 26 05:32:04 2009
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 26 Mar 2009 13:32:04 +0900
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <49CAF5FB.80401@v.loewis.de>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C79C1A.8040301@simplistix.co.uk> <49C7FC85.5000809@v.loewis.de>
	<49C80FA0.4020800@simplistix.co.uk> <87ab7bh5fb.fsf@xemacs.org>
	<49C87004.2030807@holdenweb.com> <873ad3gxdd.fsf@xemacs.org>
	<49CA3953.1080506@gmail.com>
	<638F8377-6967-44B9-8ECF-F5DF80CD32BC@python.org>
	<49CAF5FB.80401@v.loewis.de>
Message-ID: <5b8d13220903252132l746de5cx981d497c349a8c2b@mail.gmail.com>

On Thu, Mar 26, 2009 at 12:26 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Tools like setuptools, zc.buildout, etc. seem great for developers but not
>> very good for distributions. ?At last year's Pycon I think there was
>> agreement from the Linux distributors that distutils, etc. just wasn't very
>> useful for them.
>
> I think distutils is different here - it not only helps creating
> distributions (i.e. deployable package files), but also allows
> direct installation. This, in turn, is used to build the packages
> for Linux distributions. E.g. debian/rules often contains a
> "setup.py install" call in its build step (and there is even a
> CDBS python-distutils.mk fragment)
>
> In that sense, distutils is for Python what make is for C.

It is more like the whole autotools suite (at least for unix), and
that's the problem: distutils does everything quite poorly, and once
you need to do something that distutils can't do out of the box, you
are in a no man's land because distutils is almost impossible to
extend (everything is done internally, with no way to recover data
short of rewriting everything or monkey patching). To take a recent
example, I wanted to add the ability to install a clib extension (pure
C, no python API), so that it can be used by other projects: that
would take 2 minutes with any build tool out there, but is almost
impossible in distutils, unless you rewrite your own build_clib and/or
install commands. Even autotools is more enjoyable, which is quite an
achievement :)

If distutils was split into different modules (one for the build, one
for the compiler/platform configuration, one for the installation),
which could be extended, tweaked, it would be much better. But the
distutils design makes this inherently very difficult (commands).

cheers,

David

From guido at python.org  Thu Mar 26 05:39:29 2009
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Mar 2009 21:39:29 -0700
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49C9C947.3080608@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
Message-ID: <ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>

On Tue, Mar 24, 2009 at 11:03 PM, Greg Ewing
<greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>
>> The way I think of it, that refactoring has nothing to do with
>> yield-from.
>
> I'm not sure what you mean by that. Currently it's
> *impossible* to factor out code containing a yield.

That's stating it a little too strongly. Phillip has shown how with
judicious use of decorators and helper classes you can get a
reasonable approximation, and I think Twisted uses something like
this, so it's not just theory. I think the best you can do without new
syntax though is still pretty cumbersome and brittle, which is why I
have encouraged your PEP.

> Providing a way to do that is what led me to invent
> this particular version of yield-from in the first
> place.
>
> I wanted a way of writing suspendable functions that
> can call each other easily. (You may remember I
> originally wanted to call it "call".) Then I noticed
> that it would also happen to provide the functionality
> of earlier "yield from" suggestions, so I adopted that
> name.
>
> But for me, factorability has always been the fundamental
> idea, and the equivalence, in one particular restricted
> situation, to a for loop containing a yield is just a
> nice bonus.
>
> That's what I've tried to get across in the PEP, and
> it's the reason I've presented things in the way I have.

That's all good. I just don't think that a presentation in terms of
code in-lining is a good idea. That's not how we explain functions
either. We don't say "the function call means the same as when we
wrote the body of the function in-line here." It's perhaps a game with
words, but it's important to me not to give that impression, since
some languages *do* work that way (e.g. macro languages and Algol-60),
but Python *doesn't*.

>> It's not just variable references -- I used "scope" as a
>> shorthand for everything that can be done within a function body,
>> including control flow: try/except/finally,
>> continue/break/raise/return.
>
> Same answer applies -- use the usual techniques.
>
> When I talk about inlining, I mean inlining the
> *functionality* of the code, not its literal text. I'm
> leaving the reader to imagine performing the necessary
> transformations to preserve the semantics.

Yeah, so I'm asking you to use a different word, since "inlining" to
me has pretty strong connotations of textual substitution.

>> Maybe you're confusing motivation with explanation? That feedback
>> seems to tell me that the *motivation* needs more work; but IMO the
>> *explanation* should start with this simple model and then expand upon
>> the edge cases.
>
> Perhaps what I should do is add a Motivation section before
> the Proposal and move some of the material from the beginning
> of the Rationale sectiomn there.

Yeah, I think it can easily be saved by changing the PEP without
changing the specs of the proposal.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ziade.tarek at gmail.com  Thu Mar 26 05:40:16 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 26 Mar 2009 05:40:16 +0100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
Message-ID: <94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>

On Thu, Mar 26, 2009 at 5:19 AM, David Cournapeau <cournape at gmail.com> wrote:
> On Thu, Mar 26, 2009 at 12:02 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> If that perception is accurate, then any changes likely need to focus on
>> the *opposite* end of the toolchain: the part between the "<common
>> packaging spec>" and the end users.
>
> Yes - but is this part the job of python ?
>
>> In other words: Given an egg, how easy is it for a packager/distributor
>> to create a platform specific package that places the files in the
>> correct locations for that particular platform (regardless of how
>> arbitrary those rules may appear to the original developers)?

I think Distutils (and therefore Setuptools) should provide some APIs
to play with special files (like resources) and to mark them as being special,
no matter where they end up in the target system.

So the code inside the package can use these files seamessly no matter
what the system is
and no matter where the files have been placed by the packager.

This has been discussed already but not clearly defined.

>
> Why coming from eggs and not from the build tool provided by python
> itself (distutils) ? I don't see what eggs brings - specially since
> the format is not even standardized.

I don't think the "egg as a format" part has anything to do with this
discussion. It's just a zip file with
a specific layout.

What's important is what we store in the .egg-info directory (or file) which
has two standards today (one in Distutils, one in Setuptools) but wich should
get a common standard soon (PEP 376)

Cheers
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From ncoghlan at gmail.com  Thu Mar 26 05:44:34 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Mar 2009 14:44:34 +1000
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
 currently conform
In-Reply-To: <ca471dc20903252034k332f7c10nfae0d51a9a852e04@mail.gmail.com>
References: <b8ad139e0903231352i66bc38f3v4f88b0914c760df7@mail.gmail.com>	
	<b8ad139e0903231400h570d771bt588ccbc71a295db4@mail.gmail.com>	
	<ca471dc20903241314t318f2fb2vf864dae094592f53@mail.gmail.com>	
	<49CA27BD.7080904@gmail.com>
	<ca471dc20903252034k332f7c10nfae0d51a9a852e04@mail.gmail.com>
Message-ID: <49CB0832.1030400@gmail.com>

Guido van Rossum wrote:
> On Wed, Mar 25, 2009 at 5:46 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Having helped with that kind of rename once (and for a relatively small
>> API at that), I'd want a *really* compelling reason before ever going
>> through it again - it's messy, tedious and a really good way to burn
>> volunteer time without a great deal to show for it at the end.
> 
> My first response was "in hindsight we shouldn't have done this."

Even in hindsight, I think our reasons for providing a PEP 8 compliant
threading API are sound. But the experience still makes me cautious of
doing it again.

> But
> we moved a bunch of other modules around too (urllib, http, db, I
> forget what else) and I think those worked out well.

For everything else, we just changed the module name or location. The
test suite is pretty good at finding any relevant imports that need to
be fixed, and 2to3 is also pretty good at handling them for third party
code. The threading module was different in that we actually wanted to
change the API of the module itself rather than just where to find it.

> Why was threading
> particularly unpleasant? (An no, this isn't a rhetorical question or a
> retort. I'm just curious -- I have the same feeling but can't pin it
> down.)

I think the main thing that may be putting me off is the amount of
energy that went into deciding whether or not to emit Py3k warnings or
DeprecationWarning or PendingDeprecationWarning for use of the old
threading API. The difficulty of that decision strongly flavours my
recollection of the whole process even though the final solution chosen
was quite simple (maintain the two APIs in parallel, with a couple of
notes in the documentation about the situation).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Thu Mar 26 05:51:08 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Mar 2009 14:51:08 +1000
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903252126t31b03dib87d47d27c4700ed@mail.gmail.com>
References: <49C48D03.7030709@gmail.com>	
	<loom.20090321T122452-862@post.gmane.org>	
	<49C5620A.4080908@canterbury.ac.nz>	
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	
	<49C966A0.6090401@canterbury.ac.nz> <gqc4id$ud8$1@ger.gmane.org>	
	<20090325033249.C8A3F3A4097@sparrow.telecommunity.com>	
	<49C9CBFF.3080705@canterbury.ac.nz> <49CA3021.5050905@gmail.com>
	<ca471dc20903252126t31b03dib87d47d27c4700ed@mail.gmail.com>
Message-ID: <49CB09BC.70205@gmail.com>

Guido van Rossum wrote:
> On Wed, Mar 25, 2009 at 6:22 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> It could perhaps be done by enhancing the part of the
>>> 'for' loop that gets executed upon normal termination
>>> of the iterator.
>>>
>>>   for x in my_iter:
>>>     do_something_with(x)
>>>   else v:
>>>     handle_return_value(v)
>> I think something like that would actually make the PEP much stronger on
>> this front - it would promote the idea of a "final value" for iterators
>> as a more fundamental concept that can be worked with in a non-generator
>> context.
> 
> Hold it right there. Or maybe I should say "in your dreams." Please
> don't stretch the scope of the PEP. It's not going to help your cause.

Yes, I now agree your suggestion of comparing and contrasting with PJE's
simple trampoline example is a much better angle of attack.

Although the PEP may still want to mention how one would write *tests*
for these things. Will the test drivers themselves need to be generators
participating in some kind of trampoline setup?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ziade.tarek at gmail.com  Thu Mar 26 06:01:25 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 26 Mar 2009 06:01:25 +0100
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <5b8d13220903252132l746de5cx981d497c349a8c2b@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C7FC85.5000809@v.loewis.de> <49C80FA0.4020800@simplistix.co.uk>
	<87ab7bh5fb.fsf@xemacs.org> <49C87004.2030807@holdenweb.com>
	<873ad3gxdd.fsf@xemacs.org> <49CA3953.1080506@gmail.com>
	<638F8377-6967-44B9-8ECF-F5DF80CD32BC@python.org>
	<49CAF5FB.80401@v.loewis.de>
	<5b8d13220903252132l746de5cx981d497c349a8c2b@mail.gmail.com>
Message-ID: <94bdd2610903252201t3a4977b2qf944de4cef50c9b5@mail.gmail.com>

On Thu, Mar 26, 2009 at 5:32 AM, David Cournapeau <cournape at gmail.com> wrote:
> If distutils was split into different modules (one for the build, one
> for the compiler/platform configuration, one for the installation),
> which could be extended, tweaked, it would be much better. But the
> distutils design makes this inherently very difficult (commands).

I am not sur why the command design is a problem here. And I think
Distutils features are not far from
what you need, if you look at things like customize_compiler, which is
called by build_clib.

I'm ready to discuss your use case in Distutils-SIG, if you have a
complete example etc.

Cheers
Tarek

From ncoghlan at gmail.com  Thu Mar 26 06:01:32 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Mar 2009 15:01:32 +1000
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>	
	<49C99C7B.3080805@holdenweb.com>	
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	
	<loom.20090325T121432-793@post.gmane.org>	
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>	
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>	
	<49CAF030.80707@gmail.com>	
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
Message-ID: <49CB0C2C.9040201@gmail.com>

Tarek Ziad? wrote:
> On Thu, Mar 26, 2009 at 5:19 AM, David Cournapeau <cournape at gmail.com> wrote:
>> Why coming from eggs and not from the build tool provided by python
>> itself (distutils) ? I don't see what eggs brings - specially since
>> the format is not even standardized.
> 
> I don't think the "egg as a format" part has anything to do with this
> discussion. It's just a zip file with
> a specific layout.
> 
> What's important is what we store in the .egg-info directory (or file) which
> has two standards today (one in Distutils, one in Setuptools) but wich should
> get a common standard soon (PEP 376)

Yes, that metadata is what I meant to refer to rather than zipped .egg
files specifically. An egg is just one example of something which
includes that metadata.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From cournape at gmail.com  Thu Mar 26 06:24:59 2009
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 26 Mar 2009 14:24:59 +0900
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <94bdd2610903252201t3a4977b2qf944de4cef50c9b5@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C80FA0.4020800@simplistix.co.uk> <87ab7bh5fb.fsf@xemacs.org>
	<49C87004.2030807@holdenweb.com> <873ad3gxdd.fsf@xemacs.org>
	<49CA3953.1080506@gmail.com>
	<638F8377-6967-44B9-8ECF-F5DF80CD32BC@python.org>
	<49CAF5FB.80401@v.loewis.de>
	<5b8d13220903252132l746de5cx981d497c349a8c2b@mail.gmail.com>
	<94bdd2610903252201t3a4977b2qf944de4cef50c9b5@mail.gmail.com>
Message-ID: <5b8d13220903252224he5f33e5hacf525667a082efe@mail.gmail.com>

On Thu, Mar 26, 2009 at 2:01 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Thu, Mar 26, 2009 at 5:32 AM, David Cournapeau <cournape at gmail.com> wrote:
>> If distutils was split into different modules (one for the build, one
>> for the compiler/platform configuration, one for the installation),
>> which could be extended, tweaked, it would be much better. But the
>> distutils design makes this inherently very difficult (commands).
>
> I am not sur why the command design is a problem here.

For several reasons:
 - options handling cannot be done correctly. If you need to pass some
options specific to the build, you have to pass it to build_clib and
build_ext, etc... Example: I would really like to add options like
--with-libfoo ala autoconf so that the packager can simply say where
to look for a library (headers, .so, etc....). This cannot be done
easily in distutils (no persistence, no easy way to communicate
between commands)
 - the whole concept of commands is bogus for a build tool. The
correct way to do builds is with a DAG, to handle dependencies.

> And I think
> Distutils features are not far from
> what you need, if you look at things like customize_compiler, which is
> called by build_clib.

The whole customize_compiler is awful. You cannot call it when you
want, but only at some arbitrary time in the execution, which is not
documented. You have to create your own command, because you can't
call it in setup.py directly. You may have to call
initialize_something_which_has_nothing_to_do_with_compiler(), which
may break on windows because the MS compiler abstraction is totally
different than the unix one.

It is actually hard to believe if you never had to deal with it: so
many trivial things which are one line of code in every other tool are
difficult, obscure, magic or damn impossible in distutils. Modifiying
compiler flag ? You have to create a new compiler class. Installing
docs ? You have to create your own install class. etc...

>
> I'm ready to discuss your use case in Distutils-SIG, if you have a
> complete example etc.

Ok, I will give you the example on the distutils ML,

David

From cournape at gmail.com  Thu Mar 26 06:31:10 2009
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 26 Mar 2009 14:31:10 +0900
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CB0C2C.9040201@gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<49CB0C2C.9040201@gmail.com>
Message-ID: <5b8d13220903252231q5dfb6279k1023d714ec9c687a@mail.gmail.com>

On Thu, Mar 26, 2009 at 2:01 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Yes, that metadata is what I meant to refer to rather than zipped .egg
> files specifically. An egg is just one example of something which
> includes that metadata.

Ok, my bad. Being able to describe meta-data for installed files is
indeed sorely needed,

David

From stephen at xemacs.org  Thu Mar 26 06:42:05 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 26 Mar 2009 14:42:05 +0900
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C939BA.8040206@v.loewis.de> <49C95E9D.8070604@gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
Message-ID: <87ljqsyh5e.fsf@xemacs.org>

David Cournapeau writes:

 > On Thu, Mar 26, 2009 at 12:02 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
 > 
 > > If that perception is accurate, then any changes likely need to focus on
 > > the *opposite* end of the toolchain: the part between the "<common
 > > packaging spec>" and the end users.
 > 
 > Yes - but is this part the job of python ?

Yes.  In the sense that what I'm hearing from this discussion is that
tools that are very satisfactory for the upstream end of the chain can
often result in formats that are inconvenient for some downstream
end.  Even if the downstream end doesn't use it, some of their users
may mix that into their own installation, and it's difficult for
anyone to sort the resulting mess.  Ie, Python may not be resposible
for that distribution or its format, but Python should take care not
to kick up sand that may blow into downstream's gearbox.

 > Why coming from eggs

Or other complex format?  Well, because downstream consumers will want
them.  Nick's point is that there are multiple consumers of Python
distributions.  So we have a toolchain

                           +-> E ------> downstream developer -+
                           |                                   |
                           |             +----------+          V
source -> build -> A -> B -+-> C -> D -> | END USER | <--------+
                           |             +----------+          A
                           |                                   |
                           +-> F -> OS distro -----------------+

I think the point is to push the B -> C or E or F split point as far
downstream as possible ... but no farther.  As I understand his
request for "focus on the opposite end", it's that.  He's not claiming
that a one-size-fits all linear distribution chain is possible.

Rather, he's asking that we ask that very question: is there an
existing format that would be a better place to split?  If so, then
the stdlib should be built on top of that format, and it should be a
standard part of Python.

But what we have so far is people are saying "of course if you do it
my way everything will work out fine", but then they're left waving
their hands and chanting spells when it comes to being concrete about
how their vaporware PMS will satisfy the needs of the other camps.  So
there's a need to analyze the requirements of downstream library
developers, workstation app developers, web developers, end users, and
OS distros (I hope I haven't left anybody out).

From cournape at gmail.com  Thu Mar 26 07:47:44 2009
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 26 Mar 2009 15:47:44 +0900
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <87ljqsyh5e.fsf@xemacs.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<87ljqsyh5e.fsf@xemacs.org>
Message-ID: <5b8d13220903252347p200b7ad0n42e33a3fca817652@mail.gmail.com>

On Thu, Mar 26, 2009 at 2:42 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:

>
> ? ? ? ? ? ? ? ? ? ? ? ? ? +-> E ------> downstream developer -+
> ? ? ? ? ? ? ? ? ? ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? |
> ? ? ? ? ? ? ? ? ? ? ? ? ? | ? ? ? ? ? ? +----------+ ? ? ? ? ?V
> source -> build -> A -> B -+-> C -> D -> | END USER | <--------+
> ? ? ? ? ? ? ? ? ? ? ? ? ? | ? ? ? ? ? ? +----------+ ? ? ? ? ?A
> ? ? ? ? ? ? ? ? ? ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? |
> ? ? ? ? ? ? ? ? ? ? ? ? ? +-> F -> OS distro -----------------+
>

According to your diagram, the build->A is the only part where
describing meta-data can be possible so that everyone benefit from it
- which is what I believe Tarek is working on,

cheers,

David

From greg.ewing at canterbury.ac.nz  Thu Mar 26 09:32:16 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 26 Mar 2009 20:32:16 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
Message-ID: <49CB3D90.2000205@canterbury.ac.nz>

Guido van Rossum wrote:

> That's all good. I just don't think that a presentation in terms of
> code in-lining is a good idea.

I was trying to describe it in a way that would give
some insight into *why* the various aspects of the
formal definition are the way they are. The inlining
concept seemed like an elegant way of doing that.

However, I've since realized that it's not quite as
unambiguous as I thought it was when a return value
is involved. I'll see if I can find another approach.

> some languages *do* work that way (e.g. macro languages and Algol-60),

Algol-60 doesn't actually work that way, they just used
a similar trick to define certain aspects of the
semantics (although in that case I agree there were
better ways they could have defined it).

I'm asking you to use a different word, since "inlining" to
> me has pretty strong connotations of textual substitution.

That's not what it usually means, as far as I can see.
When you declare a function 'inline' in C, you're not
asking for a blind textual substitution. Rather, you're
asking the compiler to generate whatever code is needed
to get the same effect as an actual call.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Mar 26 09:32:25 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 26 Mar 2009 20:32:25 +1200
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
 currently conform
In-Reply-To: <49CB0832.1030400@gmail.com>
References: <b8ad139e0903231352i66bc38f3v4f88b0914c760df7@mail.gmail.com>
	<b8ad139e0903231400h570d771bt588ccbc71a295db4@mail.gmail.com>
	<ca471dc20903241314t318f2fb2vf864dae094592f53@mail.gmail.com>
	<49CA27BD.7080904@gmail.com>
	<ca471dc20903252034k332f7c10nfae0d51a9a852e04@mail.gmail.com>
	<49CB0832.1030400@gmail.com>
Message-ID: <49CB3D99.8020400@canterbury.ac.nz>

Nick Coghlan wrote:

> I think the main thing that may be putting me off is the amount of
> energy that went into deciding whether or not to emit Py3k warnings or
> DeprecationWarning or PendingDeprecationWarning for use of the old
> threading API.

Having made that decision, though, couldn't the result
just be re-used for any future renaming exercises of
this kind?

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Mar 26 09:34:43 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 26 Mar 2009 20:34:43 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CB09BC.70205@gmail.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz> <gqc4id$ud8$1@ger.gmane.org>
	<20090325033249.C8A3F3A4097@sparrow.telecommunity.com>
	<49C9CBFF.3080705@canterbury.ac.nz> <49CA3021.5050905@gmail.com>
	<ca471dc20903252126t31b03dib87d47d27c4700ed@mail.gmail.com>
	<49CB09BC.70205@gmail.com>
Message-ID: <49CB3E23.2020300@canterbury.ac.nz>

Nick Coghlan wrote:

> Although the PEP may still want to mention how one would write *tests*
> for these things. Will the test drivers themselves need to be generators
> participating in some kind of trampoline setup?

I don't see that tests are fundamentally different
from any other code that wants to call a value-returning
generator and get the value without becoming a generator
itself. So if it's to be mentioned in the PEP at all,
a general solution might as well be given (whether it's
to use a trampoline or just write the necessary next()
and except code).

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Mar 26 09:43:41 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 26 Mar 2009 20:43:41 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CB3E23.2020300@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz> <gqc4id$ud8$1@ger.gmane.org>
	<20090325033249.C8A3F3A4097@sparrow.telecommunity.com>
	<49C9CBFF.3080705@canterbury.ac.nz> <49CA3021.5050905@gmail.com>
	<ca471dc20903252126t31b03dib87d47d27c4700ed@mail.gmail.com>
	<49CB09BC.70205@gmail.com> <49CB3E23.2020300@canterbury.ac.nz>
Message-ID: <49CB403D.4000103@canterbury.ac.nz>

Trying to think of a better usage example that
combines send() with returning values, I've realized
that part of the problem is that I don't actually
know of any realistic uses for send() in the first
place.

Can anyone point me to any? Maybe it will help
to inspire a better example.

-- 
Greg

From victor.stinner at haypocalc.com  Thu Mar 26 09:42:38 2009
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Thu, 26 Mar 2009 09:42:38 +0100
Subject: [Python-Dev] Packaging Survey first results + Summit schedule
In-Reply-To: <94bdd2610903252058n1a15daa7ic87cd32c7570ba81@mail.gmail.com>
References: <94bdd2610903252058n1a15daa7ic87cd32c7570ba81@mail.gmail.com>
Message-ID: <200903260942.38872.victor.stinner@haypocalc.com>

Le Thursday 26 March 2009 04:58:51 Tarek Ziad?, vous avez ?crit?:
> - Here are the first results for the packaging survey:
> http://tarekziade.wordpress.com/2009/03/26/packaging-survey-first-results/

How do you remove a package ?
   Wow, 275 hits for "manually" and 79 for "fail at uninstallation"

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From ncoghlan at gmail.com  Thu Mar 26 10:19:41 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Mar 2009 19:19:41 +1000
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CB3E23.2020300@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>	<loom.20090321T122452-862@post.gmane.org>	<49C5620A.4080908@canterbury.ac.nz>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	<49C966A0.6090401@canterbury.ac.nz>
	<gqc4id$ud8$1@ger.gmane.org>	<20090325033249.C8A3F3A4097@sparrow.telecommunity.com>	<49C9CBFF.3080705@canterbury.ac.nz>
	<49CA3021.5050905@gmail.com>	<ca471dc20903252126t31b03dib87d47d27c4700ed@mail.gmail.com>	<49CB09BC.70205@gmail.com>
	<49CB3E23.2020300@canterbury.ac.nz>
Message-ID: <49CB48AD.6050509@gmail.com>

Greg Ewing wrote:
> Nick Coghlan wrote:
> 
>> Although the PEP may still want to mention how one would write *tests*
>> for these things. Will the test drivers themselves need to be generators
>> participating in some kind of trampoline setup?
> 
> I don't see that tests are fundamentally different
> from any other code that wants to call a value-returning
> generator and get the value without becoming a generator
> itself. So if it's to be mentioned in the PEP at all,
> a general solution might as well be given (whether it's
> to use a trampoline or just write the necessary next()
> and except code).

Agreed the problem is more general than just testing - but a test driver
is potentially interesting in that you probably want the same test suite
to be able to test both normal code and the cooperative multitasking code.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Thu Mar 26 10:24:23 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Mar 2009 19:24:23 +1000
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
 currently conform
In-Reply-To: <49CB3D99.8020400@canterbury.ac.nz>
References: <b8ad139e0903231352i66bc38f3v4f88b0914c760df7@mail.gmail.com>	<b8ad139e0903231400h570d771bt588ccbc71a295db4@mail.gmail.com>	<ca471dc20903241314t318f2fb2vf864dae094592f53@mail.gmail.com>	<49CA27BD.7080904@gmail.com>	<ca471dc20903252034k332f7c10nfae0d51a9a852e04@mail.gmail.com>	<49CB0832.1030400@gmail.com>
	<49CB3D99.8020400@canterbury.ac.nz>
Message-ID: <49CB49C7.8070503@gmail.com>

Greg Ewing wrote:
> Nick Coghlan wrote:
> 
>> I think the main thing that may be putting me off is the amount of
>> energy that went into deciding whether or not to emit Py3k warnings or
>> DeprecationWarning or PendingDeprecationWarning for use of the old
>> threading API.
> 
> Having made that decision, though, couldn't the result
> just be re-used for any future renaming exercises of
> this kind?

Maybe - the problem with taking that decision and trying to get a
general rule out of it is that there were plenty of reasonable arguments
on all sides (there were more than just 2 options, which made the choice
all the more challenging). It wouldn't take many changes in the
specifics of a situation for the "best" answer to be different from what
we ended up doing in the threading case. The precedent would add weight
to the idea of doing the same thing again, but I don't think it would be
enough on its own to completely decide the matter.

So the only general rule I really got out of that experience was
actually "let's not do this again if we can possibly avoid it" :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From greg.ewing at canterbury.ac.nz  Thu Mar 26 11:21:24 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 26 Mar 2009 22:21:24 +1200
Subject: [Python-Dev] Revised**8 PEP on Yield-From
In-Reply-To: <49CB403D.4000103@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz> <gqc4id$ud8$1@ger.gmane.org>
	<20090325033249.C8A3F3A4097@sparrow.telecommunity.com>
	<49C9CBFF.3080705@canterbury.ac.nz> <49CA3021.5050905@gmail.com>
	<ca471dc20903252126t31b03dib87d47d27c4700ed@mail.gmail.com>
	<49CB09BC.70205@gmail.com> <49CB3E23.2020300@canterbury.ac.nz>
	<49CB403D.4000103@canterbury.ac.nz>
Message-ID: <49CB5724.1010403@canterbury.ac.nz>

Here's a new draft of the PEP. I've added a Motivation
section and removed any mention of inlining.

There is a new expansion that incorporates recent ideas,
including the suggested handling of StopIteration raised
by a throw() call (i.e. if it wasn't the one thrown in,
treat it as a return value).

Explicit finalization is performed if the delegating
generator is closed, but not when the subiterator
completes itself normally.

------------------------------------------------------------

PEP: XXX
Title: Syntax for Delegating to a Subgenerator
Version: $Revision$
Last-Modified: $Date$
Author: Gregory Ewing <greg.ewing at canterbury.ac.nz>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 13-Feb-2009
Python-Version: 2.7
Post-History:


Abstract
========

A syntax is proposed for a generator to delegate part of its
operations to another generator. This allows a section of code
containing 'yield' to be factored out and placed in another
generator. Additionally, the subgenerator is allowed to return with a
value, and the value is made available to the delegating generator.

The new syntax also opens up some opportunities for optimisation when
one generator re-yields values produced by another.


Motivation
==========

A Python generator is a form of coroutine, but has the limitation that
it can only yield to its immediate caller.  This means that a piece of
code containing a ``yield`` cannot be factored out and put into a
separate function in the same way as other code.  Performing such a
factoring causes the called function to itself become a generator, and
it is necessary to explicitly iterate over this second generator and
re-yield any values that it produces.

If yielding of values is the only concern, this can be performed without
much difficulty using a loop such as

::

     for v in g:
         yield v

However, if the subgenerator is to interact properly with the caller
in the case of calls to ``send()``, ``throw()`` and ``close()``, things
become considerably more difficult.  As will be seen later, the necessary
code is very complicated, and it is tricky to handle all the corner cases
correctly.

A new syntax will be proposed to address this issue. In the simplest
use cases, it will be equivalent to the above for-loop, but it will also
handle the full range of generator behaviour, and allow generator code
to be refactored in a simple and straightforward way.


Proposal
========

The following new expression syntax will be allowed in the body of a
generator:

::

     yield from <expr>

where <expr> is an expression evaluating to an iterable, from which an
iterator is extracted. The iterator is run to exhaustion, during which
time it yields and receives values directly to or from the caller of
the generator containing the ``yield from`` expression (the
"delegating generator").

Furthermore, when the iterator is another generator, the subgenerator is
allowed to execute a ``return`` statement with a value, and that value
becomes the value of the ``yield from`` expression.

In general, the semantics can be described in terms of the iterator
protocol as follows:

     * Any values that the iterator yields are passed directly to the
       caller.

     * Any values sent to the delegating generator using ``send()``
       are passed directly to the iterator. If the sent value is None,
       the iterator's ``next()`` method is called. If the sent value is
       not None, the iterator's ``send()`` method is called. Any exception
       resulting from attempting to call ``next`` or ``send`` is raised
       in the delegating generator.

     * Exceptions passed to the ``throw()`` method of the delegating
       generator are forwarded to the ``throw()`` method of the iterator.
       If the iterator does not have a  ``throw()`` method, its ``close()``
       method is called if it has one, then the thrown-in exception is
       raised in the delegating generator. Any exception resulting from
       attempting to call these methods (apart from one case noted below)
       is raised in the delegating generator.

     * The value of the ``yield from`` expression is the first argument
       to the ``StopIteration`` exception raised by the iterator when it
       terminates.

     * ``return expr`` in a generator causes ``StopIteration(expr)`` to
       be raised.


Fine Details
------------

The implicit GeneratorExit resulting from closing the delegating
generator is treated as though it were passed in using ``throw()``.
An iterator having a ``throw()`` method is expected to recognize
this as a request to finalize itself.

If a call to the iterator's ``throw()`` method raises a StopIteration
exception, and it is *not* the same exception object that was thrown
in, its value is returned as the value of the ``yield from`` expression
and the delegating generator is resumed.


Enhancements to StopIteration
-----------------------------

For convenience, the ``StopIteration`` exception will be given a
``value`` attribute that holds its first argument, or None if there
are no arguments.


Formal Semantics
----------------

1. The statement

::

     RESULT = yield from EXPR

is semantically equivalent to

::

     _i = iter(EXPR)
     try:
         try:
             _y = _i.next()
         except StopIteration, _e:
             _r = _e.value
         else:
             while 1:
                 try:
                     _s = yield _y
                 except:
                     _m = getattr(_i, 'throw', None)
                     if _m is not None:
                         _x = sys.exc_info()
                         try:
                             _y = _m(*_x)
                         except StopIteration, _e:
                             if _e is _x[1]:
                                 raise
                             else:
                                 _r = _e.value
                                 break
                     else:
                         _m = getattr(_i, 'close', None)
                         if _m is not None:
                             _m()
                         raise
                 else:
                     try:
                         if _s is None:
                             _y = _i.next()
                         else:
                             _y = _i.send(_s)
                     except StopIteration, _e:
                         _r = _e.value
                         break
     finally:
         del _i
     RESULT = _r

except that implementations are free to cache bound methods for the 'next',
'send' and 'throw' methods of the iterator upon first use.

2. In a generator, the statement

::

     return value

is semantically equivalent to

::

     raise StopIteration(value)

except that, as currently, the exception cannot be caught by ``except``
clauses within the returning generator.

3. The StopIteration exception behaves as though defined thusly:

::

    class StopIteration(Exception):

        def __init__(self, *args):
            if len(args) > 0:
                self.value = args[0]
            else:
                self.value = None
            Exception.__init__(self, *args)


Rationale
=========

The Refactoring Principle
-------------------------

The rationale behind most of the semantics presented above stems from
the desire to be able to refactor generator code. It should be possible
to take an section of code containing one or more ``yield`` expressions,
move it into a separate function (using the usual techniques to deal
with references to variables in the surrounding scope, etc.), and
call the new function using a ``yield from`` expression.

The behaviour of the resulting compound generator should be, as far as
possible, exactly the same as the original unfactored generator in all
situations, including calls to ``next()``, ``send()``, ``throw()`` and
``close()``.

The semantics in cases of subiterators other than generators has been
chosen as a reasonable generalization of the generator case.


Finalization
------------

There was some debate as to whether explicitly finalizing the delegating
generator by calling its ``close()`` method while it is suspended at a
``yield from`` should also finalize the subiterator. An argument against
doing so is that it would result in premature finalization of the
subiterator if references to it exist elsewhere.

Consideration of non-refcounting Python implementations led to the
decision that this explicit finalization should be performed, so that
explicitly closing a factored generator has the same effect as doing
so to an unfactored one in all Python implementations.

The assumption made is that, in the majority of use cases, the subiterator
will not be shared. The rare case of a shared subiterator can be
accommodated by means of a wrapper that blocks ``throw()`` and ``send()``
calls, or by using a means other than ``yield from`` to call the
subiterator.


Generators as Threads
---------------------

A motivation for generators being able to return values concerns the
use of generators to implement lightweight threads.  When using
generators in that way, it is reasonable to want to spread the
computation performed by the lightweight thread over many functions.
One would like to be able to call a subgenerator as though it were an
ordinary function, passing it parameters and receiving a returned
value.

Using the proposed syntax, a statement such as

::

     y = f(x)

where f is an ordinary function, can be transformed into a delegation
call

::

     y = yield from g(x)

where g is a generator. One can reason about the behaviour of the
resulting code by thinking of g as an ordinary function that can be
suspended using a ``yield`` statement.

When using generators as threads in this way, typically one is not
interested in the values being passed in or out of the yields.
However, there are use cases for this as well, where the thread is
seen as a producer or consumer of items. The ``yield from``
expression allows the logic of the thread to be spread over as
many functions as desired, with the production or consumption of
items occuring in any subfunction, and the items are automatically
routed to or from their ultimate source or destination.

Concerning ``throw()`` and ``close()``, it is reasonable to expect
that if an exception is thrown into the thread from outside, it should
first be raised in the innermost generator where the thread is suspended,
and propagate outwards from there; and that if the thread is terminated
from outside by calling ``close()``, the chain of active generators
should be finalised from the innermost outwards.


Syntax
------

The particular syntax proposed has been chosen as suggestive of its
meaning, while not introducing any new keywords and clearly standing
out as being different from a plain ``yield``.


Optimisations
-------------

Using a specialised syntax opens up possibilities for optimisation
when there is a long chain of generators.  Such chains can arise, for
instance, when recursively traversing a tree structure.  The overhead
of passing ``next()`` calls and yielded values down and up the chain
can cause what ought to be an O(n) operation to become, in the worst
case, O(n\*\*2).

A possible strategy is to add a slot to generator objects to hold a
generator being delegated to.  When a ``next()`` or ``send()`` call is
made on the generator, this slot is checked first, and if it is
nonempty, the generator that it references is resumed instead.  If it
raises StopIteration, the slot is cleared and the main generator is
resumed.

This would reduce the delegation overhead to a chain of C function
calls involving no Python code execution.  A possible enhancement would
be to traverse the whole chain of generators in a loop and directly
resume the one at the end, although the handling of StopIteration is
more complicated then.


Use of StopIteration to return values
-------------------------------------

There are a variety of ways that the return value from the generator
could be passed back. Some alternatives include storing it as an
attribute of the generator-iterator object, or returning it as the
value of the ``close()`` call to the subgenerator. However, the proposed
mechanism is attractive for a couple of reasons:

* Using the StopIteration exception makes it easy for other kinds
   of iterators to participate in the protocol without having to
   grow an extra attribute or a close() method.

* It simplifies the implementation, because the point at which the
   return value from the subgenerator becomes available is the same
   point at which StopIteration is raised. Delaying until any later
   time would require storing the return value somewhere.


Criticisms
==========

Under this proposal, the value of a ``yield from`` expression would
be derived in a very different way from that of an ordinary ``yield``
expression. This suggests that some other syntax not containing the
word ``yield`` might be more appropriate, but no acceptable alternative
has so far been proposed.

It has been suggested that some mechanism other than ``return`` in
the subgenerator should be used to establish the value returned by
the ``yield from`` expression. However, this would interfere with
the goal of being able to think of the subgenerator as a suspendable
function, since it would not be able to return values in the same way
as other functions.

The use of an argument to StopIteration to pass the return value
has been criticised as an "abuse of exceptions", without any
concrete justification of this claim. In any case, this is only
one suggested implementation; another mechanism could be used
without losing any essential features of the proposal.

It has been suggested that a different exception, such as
GeneratorReturn, should be used instead of StopIteration to return a
value. However, no convincing practical reason for this has been put
forward, and the addition of a ``value`` attribute to StopIteration
mitigates any difficulties in extracting a return value from a
StopIteration exception that may or may not have one. Also, using a
different exception would mean that, unlike ordinary functions,
'return' without a value in a generator would not be equivalent to
'return None'.


Alternative Proposals
=====================

Proposals along similar lines have been made before, some using the
syntax ``yield *`` instead of ``yield from``.  While ``yield *`` is
more concise, it could be argued that it looks too similar to an
ordinary ``yield`` and the difference might be overlooked when reading
code.

To the author's knowledge, previous proposals have focused only on
yielding values, and thereby suffered from the criticism that the
two-line for-loop they replace is not sufficiently tiresome to write
to justify a new syntax.  By dealing with the full generator
protocol, this proposal provides considerably more benefit.


Additional Material
===================

Some examples of the use of the proposed syntax are available, and also a
prototype implementation based on the first optimisation outlined above.

`Examples and Implementation`_

.. _Examples and Implementation: 
http://www.cosc.canterbury.ac.nz/greg.ewing/python/yield-from/


Copyright
=========

This document has been placed in the public domain.



..
    Local Variables:
    mode: indented-text
    indent-tabs-mode: nil
    sentence-end-double-space: t
    fill-column: 70
    coding: utf-8
    End:

-- 
Greg

From solipsis at pitrou.net  Thu Mar 26 11:56:29 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 26 Mar 2009 10:56:29 +0000 (UTC)
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
Message-ID: <loom.20090326T102716-650@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> 
> That's stating it a little too strongly. Phillip has shown how with
> judicious use of decorators and helper classes you can get a
> reasonable approximation, and I think Twisted uses something like
> this, so it's not just theory. I think the best you can do without new
> syntax though is still pretty cumbersome and brittle, which is why I
> have encouraged your PEP.

It remains to be seen whether Twisted and other libraries (Kamaelia?) can
benefit from this PEP. There seems to be a misunderstanding as to how generators
are used in Twisted. There isn't a global "trampoline" to schedule generators
around. Instead, generators are wrapped with a decorator (*) which collects each
yielded value (it's a Deferred object) and attaches to it a callback which
resumes (using send()) the execution of the generator whenever the Deferred
finally gets its value. The wrapped generator, in turn, looks like a normal
Deferred-returning function to outside code.

Therefore, there is no nesting problem and "yield from" doesn't seem to be
useful here. This has been confirmed to me by a Twisted developer on IRC (he
pointed out, however, a streaming XML parser where "yield from" could save a
couple of lines of code).

(*) inlineCallbacks:
http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.defer.html#inlineCallbacks
http://enthusiasm.cozy.org/archives/2009/03/python-twisteds-inlinecallbacks

Regards

Antoine.



From ncoghlan at gmail.com  Thu Mar 26 12:35:47 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Mar 2009 21:35:47 +1000
Subject: [Python-Dev] Revised**8 PEP on Yield-From
In-Reply-To: <49CB5724.1010403@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>	<loom.20090321T122452-862@post.gmane.org>	<49C5620A.4080908@canterbury.ac.nz>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	<49C966A0.6090401@canterbury.ac.nz>
	<gqc4id$ud8$1@ger.gmane.org>	<20090325033249.C8A3F3A4097@sparrow.telecommunity.com>	<49C9CBFF.3080705@canterbury.ac.nz>
	<49CA3021.5050905@gmail.com>	<ca471dc20903252126t31b03dib87d47d27c4700ed@mail.gmail.com>	<49CB09BC.70205@gmail.com>
	<49CB3E23.2020300@canterbury.ac.nz>	<49CB403D.4000103@canterbury.ac.nz>
	<49CB5724.1010403@canterbury.ac.nz>
Message-ID: <49CB6893.10204@gmail.com>

Greg Ewing wrote:
> Here's a new draft of the PEP. I've added a Motivation
> section and removed any mention of inlining.

I like this version a lot better - reading the two examples on your site
helped as well.

> There is a new expansion that incorporates recent ideas,
> including the suggested handling of StopIteration raised
> by a throw() call (i.e. if it wasn't the one thrown in,
> treat it as a return value).

The spec for GeneratorExit handling means that the if statement around
the raise statement needs an extra condition: a thrown in GeneratorExit
should *always* be reraised, even if the subgenerator converts it to
StopIteration (which it is allowed to do by PEP 342 and the relevant
documentation).

> ------------------------------------------------------------
> In general, the semantics can be described in terms of the iterator
> protocol as follows:

"iterator protocol and generator API" or "generator protocol" (which is
a phrase you already use later in the PEP) would be more accurate, as
send() and throw() aren't part of the basic iterator protocol.

> Fine Details
> ------------
> 
> The implicit GeneratorExit resulting from closing the delegating
> generator is treated as though it were passed in using ``throw()``.
> An iterator having a ``throw()`` method is expected to recognize
> this as a request to finalize itself.
> 
> If a call to the iterator's ``throw()`` method raises a StopIteration
> exception, and it is *not* the same exception object that was thrown
> in, its value is returned as the value of the ``yield from`` expression
> and the delegating generator is resumed.

As mentioned above, I believe this should be overruled in the case of
GeneratorExit. Since correctly written generators are permitted to
convert GeneratorExit to StopIteration, the 'yield from' expression
should detect when that has happened and reraise the original exception.

> Finalization
> ------------
> 
> There was some debate as to whether explicitly finalizing the delegating
> generator by calling its ``close()`` method while it is suspended at a
> ``yield from`` should also finalize the subiterator. An argument against
> doing so is that it would result in premature finalization of the
> subiterator if references to it exist elsewhere.
> 
> Consideration of non-refcounting Python implementations led to the
> decision that this explicit finalization should be performed, so that
> explicitly closing a factored generator has the same effect as doing
> so to an unfactored one in all Python implementations.
> 
> The assumption made is that, in the majority of use cases, the subiterator
> will not be shared. The rare case of a shared subiterator can be
> accommodated by means of a wrapper that blocks ``throw()`` and ``send()``
> calls, or by using a means other than ``yield from`` to call the
> subiterator.

With the currently semantics (calling close() if throw() isn't
available), it is also necessary to block close() in order to share an
iterator. Given the conclusion that shared iterators are actually better
handled by looping or explicit next() calls, I'm actually OK with that -
really focusing 'yield from' specifically on the ability to factor
monolithic generator functions into smaller components is probably a
good idea, since mere iteration is already easy.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From zvezdan at zope.com  Thu Mar 26 14:44:29 2009
From: zvezdan at zope.com (Zvezdan Petkovic)
Date: Thu, 26 Mar 2009 09:44:29 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CAF030.80707@gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com>	<49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com>	<49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
Message-ID: <FB4CB958-0315-42DD-9721-DAE040C1F357@zope.com>

On Mar 25, 2009, at 11:02 PM, Nick Coghlan wrote:

> That is, the full workflow that should really be happening is  
> something like the following:
>
>                     Developer(s)
>                          |
>                          V
>         (distutils/setuptools/pip/zc.buildout/etc)
>                          |
>                          V

Here's a small FYI for this point in the diagram.
Perhaps some people are not aware of zc.sourcerelease.

	http://pypi.python.org/pypi/zc.sourcerelease/

It allows you to take a zc.buildout based project and pack it into a  
tarball with all the eggs the project depends on packed with it.

	buildout-source-release projectdir_or_repo_path buildout.cfg

You can drop the resulting tarball into SOURCES directory and a  
corresponding .spec file into SPECS directory and build an RPM package.
The RPM is then used to install the application.

Thus, zc.buildout is used for development (package management) and  
zc.sourcerelease + rpmbuild for distribution.

I know this does not solve problems that many mentioned in this  
thread, but it could solve a distribution problem for people who use  
zc.buildout  but were not aware of zc.sourcerelease.  As I said above,  
this was just an info for anyone interested in this particular workflow.

>
>             <Common Python packaging spec>
>                          |
>                          V
>              Packager(s)/Distributor(s)
>                          |
>                          V
>  (Windows MSI/Mac OS X bundle/Linux RPM/Linux DEB/etc)
>                          |
>                          V
>                 End User Installation


From olemis at gmail.com  Thu Mar 26 15:01:22 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 26 Mar 2009 09:01:22 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>
	<87myb9w6sq.fsf@benfinney.id.au>
	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
Message-ID: <24ea26600903260701tef32a0h904d0e5ee4931ca7@mail.gmail.com>

2009/3/25 Tennessee Leeuwenburg <tleeuwenburg at gmail.com>:
> I would suggest there may be three use cases for Python installation tools.
> Bonus -- I'm not a web developer! :)
> Case One: Developer wishing to install additional functionality into the
> system Python interpreter forever
> Case Two: Developer wishing to install additional functionality into the
> system Python interpreter for a specific task
> Case Three: Person wanting to install an application which happens to be
> written in Python

If none of these includes managing packages (... similar to apt, yum,
and equivalent ;) pls include it ... and I mean it from a higher level
of abstraction; i.e. disregarding the nature of the ?thing? (pkg, mdl,
web framew, plugin, console app, GUI, sci-tool, and so on ... made
with Py ;)

> The prime limitation of setuptools appears to me to come when there are
> conflicts.

IMO the prime limitation of setuptools (for installing pkgs ... and so
on ..) appears to me when I want to control what I've installed (... a
broader situation, i.e. it includes the specific case when there are
conflicts ... and many others ... like removal, etc ... ).

> For issues
> where there are conflicts, where I have been sufficiently motivated, setting
> up a virtualenv has more than met my needs.

I dont think this should be *THE* option ...

> In fact, that's and *awesome*
> piece of functionality.

But shouldnt be the only one ... not everybody (... devs & users ...)
will be open to setup a virtual env in order to install a Py app ...
even myself ... I'd do it if I really need it ... else it is just
painful and non-sense to force me to do it ... and I dont think it'd
be right either ...

> For case one, where I want to install additional functionality into my
> system Python interpreter "forever", it would be great to have my system
> manage this. On my ubuntu machine, this happens. The Ubuntu packages take
> care of things very satisfactorily and I don't see how anyone would have a
> problem with it.

Well I do ... my example is as follows (... it actually happened few
days ago ...) :

- I maintain many Trac instances in many different places ...
- I updated some of them to Trac 0.11.1 ...
- One of these was a host running Ubuntu Hardy ... and AFAIK there are
no official debs for Ubuntu Hardy, including backports ... but only
for Intrepid ;)
- So I removed Trac 0.10 and installed Trac 0.11, this time using
setup.py since I didnt have stdeb (... or equivalent ;) at hand ...
- Few months later there was a problem with that server ...
- Since I'm used to use apt+dpkg I didnt find Trac among the list of
installed soft and I completely forgot about all the previous steps
(... believe me my memory sucjks ... :S ...), I though Trac was
removed and I installed trac 0.10 once again from a local repo ...
- Results ... conflicts everywehere and time spent in finding out what
was wrong ... Besides I hosted multiple Trac instances in that single
machine (... that's why I package plugins as egg files ... most of the
time I face this situation, and I always try to do it as portable as
possible to decrease maintainance effort ... ;)
- ... and the solution was to remove trac (pkg & egg) folder, and this
is ugly (... at least these days in XXI century ... and me thinking in
taking control over the whole Emperor Vader's fleet ... I want all
their spaceships and vessels ... ;) but ... what if the installation
includes some other files placed somewhere else ... they remain there
and they'r useless ... and you possibly wont ever know they'r still
there ... if you dont use apt or similar ... ;)

> Perhaps an automated updates site could be configured such that there was a
> package for a variety of "Python + some extra modules" scenarios.

AFAIK PyPI offers some kwalitee (cheesecake) metrics and this includes
installability ... perhaps these metrics could be extended to consider
whether it is possible or not to automatically build packages for a
particular platform (debs, rpms, win_inst), and ... as a side-effect,
additional service or maybe because of the devs manually uploading the
?platform-specific pkgs? ... include those packages in PyPI ... (no
debs allowed in there ... yet ;)

Perhaps I'm going too far up to this point ... and I'm just dreaming
... but, beyond all these ... perhaps easy_install could just be a
little bit smarter and do things as follows :

1 - Detect the platform of the target host (i.e. the OS | distro in
the ?machine? where the app | framework | ... will be installed )
2 - Find out if there is a specific install package for this OS |
distro in PyPI ( ... or the downloads site ...)
3 - Install the specific install package ...
4 - If either 2 or 3 goes wrong ... fall-back to installing the source
distro as usual (... perhaps notifying ?the user? of what's going on
and asking him about whether to proceed or not ...)

Doing nothing but this (... and allowing to upload debs to PyPI,
together with including (b|s)dist commands for debs ... in order to
get the sdist and build my own DEBs in case I cant find one ... just
like I'd do it today using distutils RPM support ... ;) I'd be the
happiest man in the world ... maybe some others will be that happy too
... ;)

PS: ?Random ideas? ... ;)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From guido at python.org  Thu Mar 26 15:15:41 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Mar 2009 07:15:41 -0700
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
	currently conform
In-Reply-To: <49CB49C7.8070503@gmail.com>
References: <b8ad139e0903231352i66bc38f3v4f88b0914c760df7@mail.gmail.com>
	<b8ad139e0903231400h570d771bt588ccbc71a295db4@mail.gmail.com>
	<ca471dc20903241314t318f2fb2vf864dae094592f53@mail.gmail.com>
	<49CA27BD.7080904@gmail.com>
	<ca471dc20903252034k332f7c10nfae0d51a9a852e04@mail.gmail.com>
	<49CB0832.1030400@gmail.com> <49CB3D99.8020400@canterbury.ac.nz>
	<49CB49C7.8070503@gmail.com>
Message-ID: <ca471dc20903260715g7a830895q879fae3738e50c43@mail.gmail.com>

On Thu, Mar 26, 2009 at 2:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Greg Ewing wrote:
>> Nick Coghlan wrote:
>>
>>> I think the main thing that may be putting me off is the amount of
>>> energy that went into deciding whether or not to emit Py3k warnings or
>>> DeprecationWarning or PendingDeprecationWarning for use of the old
>>> threading API.
>>
>> Having made that decision, though, couldn't the result
>> just be re-used for any future renaming exercises of
>> this kind?
>
> Maybe - the problem with taking that decision and trying to get a
> general rule out of it is that there were plenty of reasonable arguments
> on all sides (there were more than just 2 options, which made the choice
> all the more challenging). It wouldn't take many changes in the
> specifics of a situation for the "best" answer to be different from what
> we ended up doing in the threading case. The precedent would add weight
> to the idea of doing the same thing again, but I don't think it would be
> enough on its own to completely decide the matter.
>
> So the only general rule I really got out of that experience was
> actually "let's not do this again if we can possibly avoid it" :)

I'll gladly take that as an added rationalization of my plea not to
change datetime.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From olemis at gmail.com  Thu Mar 26 15:29:40 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 26 Mar 2009 09:29:40 -0500
Subject: [Python-Dev] Py plugins architecture - Trac [WAS:] "setuptools has
	divided the Python community"
Message-ID: <24ea26600903260729p2754fb07xa2ed2ad23187e7da@mail.gmail.com>

On Wed, Mar 25, 2009 at 6:08 PM, Barry Warsaw <barry at python.org> wrote:
> On Mar 25, 2009, at 11:35 AM, Olemis Lang wrote:
>
>> Yes you're right, Trac requires .egg files for local plugins installs
>> (... in /plugins folder ;) so that not all environments but only one
>> be able to use the plugin ... but that's not exactly what I'm saying,
>> since setuptools AFAIK *MUST* be already there ... so Trac depends on
>> setuptools.
>
> I've not looked at Trac's plugin architecture, but I wonder, how much of it
> is generic enough to be useful for any application? ?I've found setuptools
> entry points difficult to work with for plugins, and I think the Python
> ecosystem could benefit from more shared wisdom, if not code, in designing
> plugin frameworks for applications.
>

Well ... I'm not pretty sure about following this here (py-dev)
however, I'll do my best effort to try to explain a little about it
and provide you some pointers ... ;)

- Trac plugin architecture (... which is a little bit ?meta? ...) is
based on a few classes (... and more of them under the hood ... ;)
  1 - Interface ... includes the protocol used to interact with the
specific plugin
  2 - Component ... The base class for implementing the plugin functionality ...
  3 - ComponentManager ... represents something managing many
components | plugins ... (e.g. Trac environments are ?represented? as
instances of trac.env.Environment class and it is nothing but a direct
descendant of ComponentManager ... ;).
  4 - ExtensionPoint ... to access all the plugins implenting an
specific interface ...
  5 - A few other ?cosmetic? features like Configurable items and
classes (i.e. descriptors ;) to access configuration options ...

In theory people can implement their own descendants of
ComponentManager and do things the way they want to (... at least Noah
K. -trac dev- has mentioned that he has used Trac architecture for
?his own apps? ... and I have, but only for ?my own? *web* apps ... I
have no experience about desktop or GUI or other apps ...) ...

It relies on pkg_resources entry points support in order to ?load? the
plugins ...

Besides you can read Trac docs for further details ... AFAIK they'll
attend to PyCon'09 ... talks (... hopefully recorded ;) + sprint ...

PS: Edgewall guys have also done a great job in building another great
package for i18n ... and AFAICR it is based on a simpler pluggable
architecture relying on setuptools too ... (... CMIIW anyway ;) I am
talking about Babel ...

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

From a.badger at gmail.com  Thu Mar 26 15:47:31 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 26 Mar 2009 07:47:31 -0700
Subject: [Python-Dev] Integrate BeautifulSoup into stdlib?
In-Reply-To: <5b8d13220903241308p6df4fa6bjf71ebcbc8a2ff589@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>	
	<49C8C9B3.3070403@holdenweb.com>	
	<5b8d13220903240700l72ab6e5fk479bd2fa1cdfd958@mail.gmail.com>	
	<gqb4oi$qb9$1@ger.gmane.org>	
	<5b8d13220903241041o3355b8bfk61f66fc9ea1bc68c@mail.gmail.com>	
	<gqb80a$6gb$1@ger.gmane.org>	
	<5b8d13220903241153i6f2276aq24710e629139becc@mail.gmail.com>	
	<49C93061.200@gmail.com>
	<5b8d13220903241308p6df4fa6bjf71ebcbc8a2ff589@mail.gmail.com>
Message-ID: <49CB9583.3010306@gmail.com>

David Cournapeau wrote:
>> I won't argue for setuptools' implementation of multi-version.  It
>> sucks.  But multi-version can be done well.  Sonames in C libraries are
>> a simple system that does this better.
> 
> I would say simplistic instead of simple :) what works for C won't
> necessarily work for python - and even in C, library versioning is not
> used that often except for a few core libraries. Library versioning
> works in C because C model is very simple. It already breaks for C++.

I'm not sure what you're talking about here.  Library versioning is used
for practically every library on a Linux system.  My limited exposure to
the BSDs and Solaris was the same.  (If you're only talking Windows,
well; does windows even have Sonames?) I can name only one library that
isn't versioned in Fedora right now and may have heard of five total.
Perhaps you are thinking of library symbols?  If so, there are only a
few libraries that are using that.  But specifying backwards
compatibility via soname is well known and ubiquitous.

> More high-level languages like C# already have a more complicated
> scheme (GAC) - and my impression is that it did not work that well.
> The SxS for dll on recent windows to handle multiple version is a
> nightmare too in my (limited) experience.
> 
Looking at C#/Mono/.net for examples is perfectly horrid.  They've taken
inferior library versioning and bad development practices and added
technology (the GAC) as the solution.  If you want an idea of what
python should avoid at all costs, look to that arena for your answer.

* Note that setuptools' multi-version implementation shares some things
in common with the GAC.  For instance, using directories to separate
versions instead of filenames.  setuptools' implementation could be made
better by studying the GAC and taking things like caching of lookups
from it but I don't encourage this... I think the design itself is flawed.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090326/4f4b9184/attachment.pgp>

From guido at python.org  Thu Mar 26 17:41:15 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Mar 2009 09:41:15 -0700
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
Message-ID: <ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>

On Wed, Mar 25, 2009 at 9:40 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> I think Distutils (and therefore Setuptools) should provide some APIs
> to play with special files (like resources) and to mark them as being special,
> no matter where they end up in the target system.
>
> So the code inside the package can use these files seamessly no matter
> what the system is
> and no matter where the files have been placed by the packager.
>
> This has been discussed already but not clearly defined.

Yes, this should be done. PEP 302 has some hooks but they are optional
and not available for the default case. A simple wrapper to access a
resource file relative to a given module or package would be easy to
add. It should probably support four APIs:

- Open as a binary stream
- Open as a text stream
- Get contents as a binary string
- Get contents as a text string

(I am proposing these somewhat redundant APIs since for some loader
implementations the stream APIs are "natural" and the string APIs
should be built on top of that, while for other loaders the string
APIs are natural and the stream APIs would have to wrap a StringIO
instance around this. Providing all four APIs means that the
application can use the API that is most natural for *it*, which
avoids double wrapping.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Mar 26 18:07:42 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 26 Mar 2009 13:07:42 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903252124o7635e11cvd047fc9d1de63370@mail.gmail.co
 m>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<20090325142405.C68463A40A7@sparrow.telecommunity.com>
	<ca471dc20903252124o7635e11cvd047fc9d1de63370@mail.gmail.com>
Message-ID: <20090326170521.468E93A4097@sparrow.telecommunity.com>

At 09:24 PM 3/25/2009 -0700, Guido van Rossum wrote:
>ISTR that the motivation for adding new syntax is that the best you
>can do using a trampoline library is still pretty cumbersome to use
>when you have to write a lot of tasks and subtasks, and when using
>tasks is just a tool for getting things done rather than an end goal
>in itself. I agree that the motivation and the comparison should be
>added to the PEP (perhaps moving the trampoline sample
>*implementation* to a reference or an appendix, since it is only the
>appearance of the trampoline-*using* code that matters).

In particular, it should explain why these choices are so costly as 
to justify new syntax and a complex implementation:

* decorator clearly identifying intent to create a task vs. no 
indication of task-ness
* "yield Return(value)" vs. "return value"
* "result = yield subgenerator()" vs. "result = yield from subgenerator()"

Especially since the first two differences arguably make the 
non-"yield from" code clearer, and the third makes it more compact.


>with judicious use of decorators and helper classes you can get a
>reasonable approximation, and I think Twisted uses something like
>this, so it's not just theory. I think the best you can do without new
>syntax though is still pretty cumbersome and brittle, which is why I
>have encouraged your PEP.

On the "cumbersome" side, there are only three differences, as I've 
shown above... and one of them uses less syntax than the PEP.

I'm not sure what you mean by brittle.  Twisted and PEAK have both 
had generator-based tasks for ages, and have been used in production 
for years, even before generators had send()/throw() -- the addition 
of yield expressions made it possible to get rid of the one previous 
brittleness, where you needed to do things like:

       yield subgenerator(); result = resume()

in order to pass values or exceptions in.  Since send()/throw() was 
added, the need to call a function after each yield was eliminated, 
at least from PEAK and my newer Trellis library; haven't looked at 
Twisted's tasks in a while.  I believe there are other 
generator-based task libraries available on PyPI, but can't comment 
on their robustness.

(The existence of throw(), by the way, makes it possible to produce 
tracebacks that look *exactly* as if you had called a series of 
functions, rather than iterating over a bunch of generators.  The 
sample code I gave should do this correctly, as it was created by 
yanking out working, tested code from my Trellis library, and cutting 
out all the Trellis-specific bits.)

If someone can find any semantic differences between the code I 
posted and the yield-from proposal (apart from the absence of the 
"for x in y: yield x" part of the functionality), I'd like to know 
about it...  and it should go in the PEP.


From pje at telecommunity.com  Thu Mar 26 18:12:02 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 26 Mar 2009 13:12:02 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CB403D.4000103@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz> <gqc4id$ud8$1@ger.gmane.org>
	<20090325033249.C8A3F3A4097@sparrow.telecommunity.com>
	<49C9CBFF.3080705@canterbury.ac.nz> <49CA3021.5050905@gmail.com>
	<ca471dc20903252126t31b03dib87d47d27c4700ed@mail.gmail.com>
	<49CB09BC.70205@gmail.com> <49CB3E23.2020300@canterbury.ac.nz>
	<49CB403D.4000103@canterbury.ac.nz>
Message-ID: <20090326170938.D70023A4097@sparrow.telecommunity.com>

At 08:43 PM 3/26/2009 +1200, Greg Ewing wrote:
>Trying to think of a better usage example that
>combines send() with returning values, I've realized
>that part of the problem is that I don't actually
>know of any realistic uses for send() in the first
>place.
>
>Can anyone point me to any? Maybe it will help
>to inspire a better example.

Er, well, I don't know what anybody *else* wanted them for, but I 
wanted them to implement improved trampoline functions, vs. earlier 
Python versions.  ;-)

The trampoline example I gave uses send() in order to pass the return 
values from one generator back into another.  Of course, the task 
object also has a send(), so if you do find another use case for 
send() in a co-operative context, it should be equally doable with 
the trampoline.


From pje at telecommunity.com  Thu Mar 26 18:19:04 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 26 Mar 2009 13:19:04 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <loom.20090326T102716-650@post.gmane.org>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
Message-ID: <20090326171640.EC27D3A40A5@sparrow.telecommunity.com>

At 10:56 AM 3/26/2009 +0000, Antoine Pitrou wrote:
>Guido van Rossum <guido <at> python.org> writes:
> >
> > That's stating it a little too strongly. Phillip has shown how with
> > judicious use of decorators and helper classes you can get a
> > reasonable approximation, and I think Twisted uses something like
> > this, so it's not just theory. I think the best you can do without new
> > syntax though is still pretty cumbersome and brittle, which is why I
> > have encouraged your PEP.
>
>It remains to be seen whether Twisted and other libraries (Kamaelia?) can
>benefit from this PEP.

They don't get any new features, and would require (possibly 
significant) changes in order to be able to take advantage of the syntax.

And they *still* wouldn't be able to do away with their trampolines 
-- the new trampolines would just be able to avoid the need for a 
generator stack, if they previously had one to begin with.  From your 
description, it sounds like Twisted's version of this doesn't even use a stack.

(Note: by "trampoline" I mean, "thing that processes yielded values 
and manages the resumption of the generator", which need not be 
global.  The example trampoline I posted earlier is also implemented 
as a decorator, and could be trivially extended via a lookup table to 
handle deferreds, delayed calls, or whatever else you wanted it to 
support as yield targets.)


From a.badger at gmail.com  Thu Mar 26 19:44:17 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Thu, 26 Mar 2009 11:44:17 -0700
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>	<49CAF030.80707@gmail.com>	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
Message-ID: <49CBCD01.80500@gmail.com>

Guido van Rossum wrote:
> On Wed, Mar 25, 2009 at 9:40 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>> I think Distutils (and therefore Setuptools) should provide some APIs
>> to play with special files (like resources) and to mark them as being special,
>> no matter where they end up in the target system.
>>
>> So the code inside the package can use these files seamessly no matter
>> what the system is
>> and no matter where the files have been placed by the packager.
>>
>> This has been discussed already but not clearly defined.
> 
> Yes, this should be done. PEP 302 has some hooks but they are optional
> and not available for the default case. A simple wrapper to access a
> resource file relative to a given module or package would be easy to
> add. It should probably support four APIs:
> 
> - Open as a binary stream
> - Open as a text stream
> - Get contents as a binary string
> - Get contents as a text string
> 
Depending on the definition of a "resource" there's additional
information that could be needed.  For instance, if resource includes
message catalogs, then being able to get the base directory that the
catalogs reside in is needed for passing to gettext.

I'd be very happy if "resource" didn't encompass that type of thing,
though... then we could have a separate interface that addressed the
issues with them.  I'll be at PyCon (flying in late tonight, though, and
leaving Sunday) if Tarek and others want to get ahold of me to discuss
possible ways to address what's a resource, what's not, and what we
would need to handle the different cases.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090326/fce22801/attachment.pgp>

From guido at python.org  Thu Mar 26 19:54:14 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Mar 2009 11:54:14 -0700
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CBCD01.80500@gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
Message-ID: <ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>

2009/3/26 Toshio Kuratomi <a.badger at gmail.com>:
> Guido van Rossum wrote:
>> On Wed, Mar 25, 2009 at 9:40 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>> I think Distutils (and therefore Setuptools) should provide some APIs
>>> to play with special files (like resources) and to mark them as being special,
>>> no matter where they end up in the target system.
>>>
>>> So the code inside the package can use these files seamessly no matter
>>> what the system is
>>> and no matter where the files have been placed by the packager.
>>>
>>> This has been discussed already but not clearly defined.
>>
>> Yes, this should be done. PEP 302 has some hooks but they are optional
>> and not available for the default case. A simple wrapper to access a
>> resource file relative to a given module or package would be easy to
>> add. It should probably support four APIs:
>>
>> - Open as a binary stream
>> - Open as a text stream
>> - Get contents as a binary string
>> - Get contents as a text string
>>
> Depending on the definition of a "resource" there's additional
> information that could be needed. ?For instance, if resource includes
> message catalogs, then being able to get the base directory that the
> catalogs reside in is needed for passing to gettext.

Well the whole point is that for certain loaders (e.g. zip files)
there *is* no base directory. If you do need directories you won't be
able to use PEP-302 loaders, and you can just use
os.path.dirname(<some_module>.__file__).

> I'd be very happy if "resource" didn't encompass that type of thing,
> though... then we could have a separate interface that addressed the
> issues with them. ?I'll be at PyCon (flying in late tonight, though, and
> leaving Sunday) if Tarek and others want to get ahold of me to discuss
> possible ways to address what's a resource, what's not, and what we
> would need to handle the different cases.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Mar 26 20:20:48 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Mar 2009 12:20:48 -0700
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090326170521.468E93A4097@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<20090325142405.C68463A40A7@sparrow.telecommunity.com>
	<ca471dc20903252124o7635e11cvd047fc9d1de63370@mail.gmail.com>
	<20090326170521.468E93A4097@sparrow.telecommunity.com>
Message-ID: <ca471dc20903261220i124a84c6udbabd836509d2524@mail.gmail.com>

On Thu, Mar 26, 2009 at 10:07 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 09:24 PM 3/25/2009 -0700, Guido van Rossum wrote:
>> ISTR that the motivation for adding new syntax is that the best you
>> can do using a trampoline library is still pretty cumbersome to use
>> when you have to write a lot of tasks and subtasks, and when using
>> tasks is just a tool for getting things done rather than an end goal
>> in itself. I agree that the motivation and the comparison should be
>> added to the PEP (perhaps moving the trampoline sample
>> *implementation* to a reference or an appendix, since it is only the
>> appearance of the trampoline-*using* code that matters).
>
> In particular, it should explain why these choices are so costly as to
> justify new syntax and a complex implementation:
>
> * decorator clearly identifying intent to create a task vs. no indication of
> task-ness
> * "yield Return(value)" vs. "return value"
> * "result = yield subgenerator()" vs. "result = yield from subgenerator()"
>
> Especially since the first two differences arguably make the non-"yield
> from" code clearer, and the third makes it more compact.
>
>
>> with judicious use of decorators and helper classes you can get a
>> reasonable approximation, and I think Twisted uses something like
>> this, so it's not just theory. I think the best you can do without new
>> syntax though is still pretty cumbersome and brittle, which is why I
>> have encouraged your PEP.
>
> On the "cumbersome" side, there are only three differences, as I've shown
> above... and one of them uses less syntax than the PEP.

I wasn't talking about keystrokes so much as to having to remember to
use thes incantations and understand what they do and what their
limitations are.

> I'm not sure what you mean by brittle. ?Twisted and PEAK have both had
> generator-based tasks for ages, and have been used in production for years,
> even before generators had send()/throw() -- the addition of yield
> expressions made it possible to get rid of the one previous brittleness,
> where you needed to do things like:
>
> ? ? ?yield subgenerator(); result = resume()
>
> in order to pass values or exceptions in. ?Since send()/throw() was added,
> the need to call a function after each yield was eliminated, at least from
> PEAK and my newer Trellis library; haven't looked at Twisted's tasks in a
> while. ?I believe there are other generator-based task libraries available
> on PyPI, but can't comment on their robustness.

(Well here is Greg's requested use case for .send(). :-)

By brittle I meant again having to be aware of those details of the
mechanism that exist because of syntactic limitations, e.g.
accidentally writing "return X" instead of "yield Return(X)".

> (The existence of throw(), by the way, makes it possible to produce
> tracebacks that look *exactly* as if you had called a series of functions,
> rather than iterating over a bunch of generators. ?The sample code I gave
> should do this correctly, as it was created by yanking out working, tested
> code from my Trellis library, and cutting out all the Trellis-specific
> bits.)
>
> If someone can find any semantic differences between the code I posted and
> the yield-from proposal (apart from the absence of the "for x in y: yield x"
> part of the functionality), I'd like to know about it... ?and it should go
> in the PEP.

I'll leave this up to Greg.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Thu Mar 26 20:22:24 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 26 Mar 2009 14:22:24 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
Message-ID: <10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 26, 2009, at 1:54 PM, Guido van Rossum wrote:

> 2009/3/26 Toshio Kuratomi <a.badger at gmail.com>:
>>
>>>
>> Depending on the definition of a "resource" there's additional
>> information that could be needed.  For instance, if resource includes
>> message catalogs, then being able to get the base directory that the
>> catalogs reside in is needed for passing to gettext.
>
> Well the whole point is that for certain loaders (e.g. zip files)
> there *is* no base directory. If you do need directories you won't be
> able to use PEP-302 loaders, and you can just use
> os.path.dirname(<some_module>.__file__).

One thing that /would/ be helpful though is the ability to list all  
the resources under a specific package path.  This is (I think) one  
use case that pkg_resource fails to support and it's the one place  
that I've had to drop down to file system introspection.

Think: the package-y flavor of  
os.listdir(os.path.dirname(package.__file__))

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScvV8HEjvBPtnXfVAQIvdQP/dwzTsPLVtM4eAKLRfzJHG7mToNVDgHNn
4L6Kh99sScVOpeYPw8tltX5YAFwDeysIiDl3L8ynS/iN4/RIVsssO97vK3nENQxE
myQVfapmY0yhYrCJSYLthkaOoADB4D8pBTCbvVRwT7B4DXuVhxGpebWFVi+gSWNi
ZZjDjUQ9fSk=
=REBP
-----END PGP SIGNATURE-----

From tseaver at palladion.com  Thu Mar 26 20:24:07 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 26 Mar 2009 15:24:07 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <gqdq0h$ta3$1@ger.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49C88503.2030902@v.loewis.de>	<49C886EF.80203@v.loewis.de>	<49C8C9B3.3070403@holdenweb.com>	<49C939BA.8040206@v.loewis.de>	<49C95E9D.8070604@gmail.com>	<49C96304.7050207@v.loewis.de>	<49C99C7B.3080805@holdenweb.com>	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>	<loom.20090325T133856-563@post.gmane.org>
	<gqdq0h$ta3$1@ger.gmane.org>
Message-ID: <gqgkoo$p5l$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Terry Reedy wrote:

> 5. Much of this discussion reminds me of the debates between lumping and 
> splitting of taxonomic categories in biology.  Like that debate, it will 
> continue forever.

Funny, I was thinking the same thing, only with respect to
paleoanthropology:  "chunkers" and "splitters" are perpetually at war
there, too.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJy9ZX+gerLs4ltQ4RAsOMAJ4tC4gn8QVGkQmYfEkftKIRJ2PTvACgs614
D6mUasLwphIRTJttkSdZjyA=
=T6Tv
-----END PGP SIGNATURE-----


From guido at python.org  Thu Mar 26 20:27:26 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Mar 2009 12:27:26 -0700
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
Message-ID: <ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>

On Thu, Mar 26, 2009 at 10:19 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 10:56 AM 3/26/2009 +0000, Antoine Pitrou wrote:
>>
>> Guido van Rossum <guido <at> python.org> writes:
>> >
>> > That's stating it a little too strongly. Phillip has shown how with
>> > judicious use of decorators and helper classes you can get a
>> > reasonable approximation, and I think Twisted uses something like
>> > this, so it's not just theory. I think the best you can do without new
>> > syntax though is still pretty cumbersome and brittle, which is why I
>> > have encouraged your PEP.
>>
>> It remains to be seen whether Twisted and other libraries (Kamaelia?) can
>> benefit from this PEP.
>
> They don't get any new features, and would require (possibly significant)
> changes in order to be able to take advantage of the syntax.

Well yes if you want to maintain backwards compatibility there
wouldn't be any advantage.

The point of the new syntax is clearly that (eventually) they can stop
having their own wrappers, decorators and helpers (for this purpose).

> And they *still* wouldn't be able to do away with their trampolines -- the
> new trampolines would just be able to avoid the need for a generator stack,
> if they previously had one to begin with. ?From your description, it sounds
> like Twisted's version of this doesn't even use a stack.

Whether yo need a trampoline or not depends on other needs of a
framework. There is some clear low-hanging fruit for Greg's proposal
where no trampoline or helpers are needed -- but where currently
refactoring complex code containing many yield statements is
cumbersome due to the nee to write each "subroutine" call as "for x in
subroutine(): yield x" -- being able to replace this with "yield from
subroutine()" is a conceptual advantage to me that is not proportional
to the number of characters saved.

> (Note: by "trampoline" I mean, "thing that processes yielded values and
> manages the resumption of the generator", which need not be global. ?The
> example trampoline I posted earlier is also implemented as a decorator, and
> could be trivially extended via a lookup table to handle deferreds, delayed
> calls, or whatever else you wanted it to support as yield targets.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Mar 26 20:31:42 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Mar 2009 12:31:42 -0700
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
Message-ID: <ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>

On Thu, Mar 26, 2009 at 12:22 PM, Barry Warsaw <barry at python.org> wrote:
> On Mar 26, 2009, at 1:54 PM, Guido van Rossum wrote:
>> 2009/3/26 Toshio Kuratomi <a.badger at gmail.com>:
>>> Depending on the definition of a "resource" there's additional
>>> information that could be needed. ?For instance, if resource includes
>>> message catalogs, then being able to get the base directory that the
>>> catalogs reside in is needed for passing to gettext.
>>
>> Well the whole point is that for certain loaders (e.g. zip files)
>> there *is* no base directory. If you do need directories you won't be
>> able to use PEP-302 loaders, and you can just use
>> os.path.dirname(<some_module>.__file__).
>
> One thing that /would/ be helpful though is the ability to list all the
> resources under a specific package path. ?This is (I think) one use case
> that pkg_resource fails to support and it's the one place that I've had to
> drop down to file system introspection.
>
> Think: the package-y flavor of os.listdir(os.path.dirname(package.__file__))

Good idea. Can I suggest that API this takes a glob-style pattern? (Or
to be fully general, a list of patterns and a list of exclusion
patterns.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tseaver at palladion.com  Thu Mar 26 20:33:47 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 26 Mar 2009 15:33:47 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>	<87myb9w6sq.fsf@benfinney.id.au>
	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
Message-ID: <gqglas$und$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tennessee Leeuwenburg wrote:
> I would suggest there may be three use cases for Python installation tools.
> Bonus -- I'm not a web developer! :)
> Case One: Developer wishing to install additional functionality into the
> system Python interpreter forever
> Case Two: Developer wishing to install additional functionality into the
> system Python interpreter for a specific task
> Case Three: Person wanting to install an application which happens to be
> written in Python
> 
> The prime limitation of setuptools appears to me to come when there are
> conflicts. Other than that, for packages where it is available, I have never
> had an issue with the functioning of setuptools. It's Just Fine. For issues
> where there are conflicts, where I have been sufficiently motivated, setting
> up a virtualenv has more than met my needs. In fact, that's and *awesome*
> piece of functionality.

Agreed.  I find the isolation provided by virtualenv to be essential to
the work I do.

> For case one, where I want to install additional functionality into my
> system Python interpreter "forever", it would be great to have my system
> manage this. On my ubuntu machine, this happens. The Ubuntu packages take
> care of things very satisfactorily and I don't see how anyone would have a
> problem with it. I don't know what the situation is under Windows, however
> systems such as the Enthought Python Suite do provide kitchen-sink-included
> Python distributions.
> 
> Perhaps an automated updates site could be configured such that there was a
> package for a variety of "Python + some extra modules" scenarios.
> 
> For installing applications, I would presume that app developers would
> generally take care of this themselves, through the use of an MSI or .deb
> file in order to protect their dependencies somewhat.
> 
> For anyone looking to extend their interpreter for some specific task,
> virtualenv and setuptools should do the job, and the developer can just go
> the extra mile to do the work to get it right themselves.
> 
> Perhaps the most under-served use case is people who want a
> kitchen-sink-included distribution, to be managed by their system packages.
> Either they are not away of systems such as EPS or are opposed to it in
> principle. Were time and effort no obstacle, I would suggest that it may be
> worth Python.org offering to host and/or maintain a variety of Python
> distributions available for installation which include various standard
> extensions. The model that eclipse takes, with a "base" system and plugins,
> but with flavoured versions available (i.e. Scientic Python Distribution to
> include SciPy, Numpy and MatPlotLib for example).
> 
> Thoughts?

I like the summary, and the idea of 'contrib' distributions.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJy9ib+gerLs4ltQ4RAreJAJ938Tf9sQH5T4JikxZUa36MVhp7FwCdH1Qe
GEKtcoFn8GQWlrfU1qmMkGM=
=lKBr
-----END PGP SIGNATURE-----


From barry at python.org  Thu Mar 26 20:37:50 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 26 Mar 2009 14:37:50 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
Message-ID: <487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 26, 2009, at 2:31 PM, Guido van Rossum wrote:

>> One thing that /would/ be helpful though is the ability to list all  
>> the
>> resources under a specific package path.  This is (I think) one use  
>> case
>> that pkg_resource fails to support and it's the one place that I've  
>> had to
>> drop down to file system introspection.
>>
>> Think: the package-y flavor of  
>> os.listdir(os.path.dirname(package.__file__))
>
> Good idea. Can I suggest that API this takes a glob-style pattern? (Or
> to be fully general, a list of patterns and a list of exclusion
> patterns.)

Good idea.  Thinking back on my typical use cases, a glob style  
pattern would be enough.  Usually I'm asking "give me all the .py  
files in this package" or "I'm looking for the .txt files in this path".

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScvZjnEjvBPtnXfVAQKtIQP9HVCkFI8o0bHbUxA+lS3CUa2NzyeKgs9j
MKWjC9yOT+oMhZXa0CZJfzmjwjn7Bum5+7W4TWNsMQPWfCPWl/D6oQlBI/UXa66d
VQpjA05PkxEgMuvmFMGwpMiG2h668kKBKvbKUVsxNIJI3fvBj1TyYpVpQKVNHNep
xnEPyWK9ylM=
=YhoQ
-----END PGP SIGNATURE-----

From olemis at gmail.com  Thu Mar 26 20:39:31 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 26 Mar 2009 14:39:31 -0500
Subject: [Python-Dev] Fwd:  "setuptools has divided the Python community"
In-Reply-To: <24ea26600903261234w36821e90w6aba07bd86e9130c@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<24ea26600903261234w36821e90w6aba07bd86e9130c@mail.gmail.com>
Message-ID: <24ea26600903261239gc944bbfrac38addaf74c734e@mail.gmail.com>

On Thu, Mar 26, 2009 at 1:54 PM, Guido van Rossum <guido at python.org> wrote:
> 2009/3/26 Toshio Kuratomi <a.badger at gmail.com>:
>> Guido van Rossum wrote:
>>> On Wed, Mar 25, 2009 at 9:40 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>>> I think Distutils (and therefore Setuptools) should provide some APIs
>>>> to play with special files (like resources) and to mark them as being special,
>>>> no matter where they end up in the target system.
>>>
>>> Yes, this should be done. PEP 302 has some hooks but they are optional
>>> and not available for the default case. A simple wrapper to access a
>>> resource file relative to a given module or package would be easy to
>>> add. It should probably support four APIs:
>>>
>>> - Open as a binary stream
>>> - Open as a text stream
>>> - Get contents as a binary string
>>> - Get contents as a text string
>>>
>> Depending on the definition of a "resource" there's additional
>> information that could be needed. ?For instance, if resource includes
>> message catalogs, then being able to get the base directory that the
>> catalogs reside in is needed for passing to gettext.
>
> Well the whole point is that for certain loaders (e.g. zip files)
> there *is* no base directory. If you do need directories you won't be
> able to use PEP-302 loaders, and you can just use
> os.path.dirname(<some_module>.__file__).
>

Oops didnt see this msg ... AFAICR ... this is the kind of reasons
ResourceManager is there for in pkg_resources ... CMIIW anyway ...

However GvR was talking about PEP 302 loaders ... well ... the only
thing I can say is that bundling message catalogs in egg files (they
'r zip files anyway ;) and using them with Babel (... similar to
gettext ... I think ...) works fine for me using the aforementioned
functions in pkg_resources ... About whether PEP 302 loaders should
(look like | implement functions in) pkg_resources.ResourceManager or
not ... I'm not very sure ...

... and I'm talking about ...

{{{
>>> [x for x in dir(pkg_resources) if 'resource_' in x]
['resource_exists', 'resource_filename', 'resource_isdir',
'resource_listdir', 'resource_stream', 'resource_string']

}}}

--
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?



-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From tseaver at palladion.com  Thu Mar 26 20:36:25 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 26 Mar 2009 15:36:25 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>	<87myb9w6sq.fsf@benfinney.id.au>	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
	<25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>
Message-ID: <gqglfq$und$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Barry Warsaw wrote:
> On Mar 25, 2009, at 6:06 PM, Tennessee Leeuwenburg wrote:
> 
>> For case one, where I want to install additional functionality into my
>> system Python interpreter "forever", it would be great to have my
>> system
>> manage this.
> 
> In fact, I think it /has/ to.  I'll go further and say that I'm very
> wary of using easy_install and the like to install non-distro provided
> packages into the system Python.  Many Linux distros require a
> functioning system Python to operate and the distros (should) take
> great care to make sure that if you install package X for application
> Y, you won't break essential system service Z.  Once you start
> installing your own stuff in the system Python's site-packages, you
> break the warranty.

Exactly:  I never use easy_isntall to put packages into the system
python.  in fact, I only use it inside a virtalenv-generated isolated
environment.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJy9k5+gerLs4ltQ4RAjOYAKCcGsqsQpcSkbaplvixIiN5gJYp+QCgrq3T
wqnlezbv4wvRNAHzFX8Y1uw=
=fQqU
-----END PGP SIGNATURE-----


From ziade.tarek at gmail.com  Thu Mar 26 20:41:50 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 26 Mar 2009 14:41:50 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
Message-ID: <94bdd2610903261241k66119c9am34e296706df517d4@mail.gmail.com>

On Thu, Mar 26, 2009 at 2:37 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Mar 26, 2009, at 2:31 PM, Guido van Rossum wrote:
>
>>> One thing that /would/ be helpful though is the ability to list all the
>>> resources under a specific package path. ?This is (I think) one use case
>>> that pkg_resource fails to support and it's the one place that I've had
>>> to
>>> drop down to file system introspection.
>>>
>>> Think: the package-y flavor of
>>> os.listdir(os.path.dirname(package.__file__))
>>
>> Good idea. Can I suggest that API this takes a glob-style pattern? (Or
>> to be fully general, a list of patterns and a list of exclusion
>> patterns.)
>
> Good idea. ?Thinking back on my typical use cases, a glob style pattern
> would be enough. ?Usually I'm asking "give me all the .py files in this
> package" or "I'm looking for the .txt files in this path".

I think shutil.copytree new ignore mechanism handles this use case
pretty well (see the ignore_patterns factory in
http://docs.python.org/library/shutil.html)

Maybe we could use the same pattern.

Tarek

From olemis at gmail.com  Thu Mar 26 20:43:32 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 26 Mar 2009 14:43:32 -0500
Subject: [Python-Dev]  "setuptools has divided the Python community"
In-Reply-To: <24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
Message-ID: <24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>

On Thu, Mar 26, 2009 at 2:37 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Mar 26, 2009, at 2:31 PM, Guido van Rossum wrote:
>
>>> One thing that /would/ be helpful though is the ability to list all the
>>> resources under a specific package path. ?This is (I think) one use case
>>> that pkg_resource fails to support and it's the one place that I've had
>>> to
>>> drop down to file system introspection.
>>>

Really ... ? What are all these for ?

{{{
>>> [x for x in dir(pkg_resources) if all(y in x for y in ['dir', 'resource_'])]
['resource_isdir', 'resource_listdir']

}}}

--
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From olemis at gmail.com  Thu Mar 26 20:47:43 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 26 Mar 2009 14:47:43 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <gqglfq$und$2@ger.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>
	<87myb9w6sq.fsf@benfinney.id.au>
	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
	<25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>
	<gqglfq$und$2@ger.gmane.org>
Message-ID: <24ea26600903261247t32af6eb4xde2b957571d44018@mail.gmail.com>

On Thu, Mar 26, 2009 at 2:36 PM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Barry Warsaw wrote:
>> On Mar 25, 2009, at 6:06 PM, Tennessee Leeuwenburg wrote:
>>
>>> For case one, where I want to install additional functionality into my
>>> system Python interpreter "forever", it would be great to have my
>>> system
>>> manage this.
>>
>> In fact, I think it /has/ to. ?I'll go further and say that I'm very
>> wary of using easy_install and the like to install non-distro provided
>> packages into the system Python. ?Many Linux distros require a
>> functioning system Python to operate and the distros (should) take
>> great care to make sure that if you install package X for application
>> Y, you won't break essential system service Z. ?Once you start
>> installing your own stuff in the system Python's site-packages, you
>> break the warranty.
>
> Exactly: ?I never use easy_isntall to put packages into the system
> python. ?in fact, I only use it inside a virtalenv-generated isolated
> environment.
>
>

What about the proposal I made about the smarter easy_install ...
isn't that just enough ?

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From barry at python.org  Thu Mar 26 20:49:31 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 26 Mar 2009 14:49:31 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <94bdd2610903261241k66119c9am34e296706df517d4@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<94bdd2610903261241k66119c9am34e296706df517d4@mail.gmail.com>
Message-ID: <7CB57586-B1A4-4EE8-A7D6-D3346DBDF0FB@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 26, 2009, at 2:41 PM, Tarek Ziad? wrote:

> I think shutil.copytree new ignore mechanism handles this use case
> pretty well (see the ignore_patterns factory in
> http://docs.python.org/library/shutil.html)
>
> Maybe we could use the same pattern.

That works for me!
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScvcTHEjvBPtnXfVAQLEAAP+MmMW3yzIYK5mT+YauzReV10UYFDkLhvT
ALbKI6+NFKu6aMYBWe70QqXYxmau3XpZyUD6qzg32t7J2kTJfoLWRCAzv9XLv6/0
Qu33AVZ7yYasSW/tbpHeDvBYsJAM3ANjYRCI4v1G35gZ8HFgdQwEnviYNW1nB/yt
3lguOFPoT6c=
=/CuU
-----END PGP SIGNATURE-----

From olemis at gmail.com  Thu Mar 26 20:50:11 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 26 Mar 2009 14:50:11 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <24ea26600903261247t32af6eb4xde2b957571d44018@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>
	<87myb9w6sq.fsf@benfinney.id.au>
	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
	<25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>
	<gqglfq$und$2@ger.gmane.org>
	<24ea26600903261247t32af6eb4xde2b957571d44018@mail.gmail.com>
Message-ID: <24ea26600903261250g5459c772i110fbe6851526c4e@mail.gmail.com>

On Thu, Mar 26, 2009 at 2:47 PM, Olemis Lang <olemis at gmail.com> wrote:
> On Thu, Mar 26, 2009 at 2:36 PM, Tres Seaver <tseaver at palladion.com> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Barry Warsaw wrote:
>>> On Mar 25, 2009, at 6:06 PM, Tennessee Leeuwenburg wrote:
>>>
>>>> For case one, where I want to install additional functionality into my
>>>> system Python interpreter "forever", it would be great to have my
>>>> system
>>>> manage this.
>>>
>>> In fact, I think it /has/ to. ?I'll go further and say that I'm very
>>> wary of using easy_install and the like to install non-distro provided
>>> packages into the system Python. ?Many Linux distros require a
>>> functioning system Python to operate and the distros (should) take
>>> great care to make sure that if you install package X for application
>>> Y, you won't break essential system service Z. ?Once you start
>>> installing your own stuff in the system Python's site-packages, you
>>> break the warranty.
>>
>> Exactly: ?I never use easy_isntall to put packages into the system
>> python. ?in fact, I only use it inside a virtalenv-generated isolated
>> environment.
>>
>>
>
> What about the proposal I made about the smarter easy_install ...
> isn't that just enough ?
>
> --
> Regards,
>
> Olemis.
>
> Blog ES: http://simelo-es.blogspot.com/
> Blog EN: http://simelo-en.blogspot.com/
>
> Featured article:
> Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?
>

I apologize but I was sending a few messages before and I didnt post
them to the list ... sorry I'll  do it right now ... sorry if you
receive multiple copies ... :(

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From olemis at gmail.com  Thu Mar 26 20:50:58 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 26 Mar 2009 14:50:58 -0500
Subject: [Python-Dev]  "setuptools has divided the Python community"
In-Reply-To: <24ea26600903250733x4d0092e9ydd56e68662bd33a5@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<24ea26600903250733x4d0092e9ydd56e68662bd33a5@mail.gmail.com>
Message-ID: <24ea26600903261250q7bb574e5n21710bacf809c706@mail.gmail.com>

On Wed, Mar 25, 2009 at 8:36 AM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
> On Wed, Mar 25, 2009 at 1:25 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> Paul Moore <p.f.moore <at> gmail.com> writes:
>>>
>>> 3. Setuptools, unfortunately, has divided the Python distribution
>>> community quite badly.
>>
>> Wait a little bit, and it's gonna be even worse, now that buildout and pip seem
>> to become popular. For example, the TurboGears people are considering switching
>> from setuptools to pip...
>>
>> Combined with
>> the current trend that everything must be exploded into lots of interdependent
>> but separately packaged libraries (the paste/pylons syndrome... try pulling
>> something like TurboGears2 and see how many third-party packages it installs), I
>> fear this is going to generate a very painful user/developer experience :-(
>>
>
> I think we are in a point in Python development where we need to clearly define
> how dependencies work.

+1

> And this has to be defined in Python (in Distutils)
> for the sake of standardization.
>

Well, I mentionned I did it using apt + insinstall support ... but I'd
love to have all these things in Python ... with or without relying on
OS-specific infrastructure for managing pkgs, deps and so on ... It is
possible to have two approaches :

- a thin layer on top of OS-specific (dpkg + apt, rpm + yum,
wininstall, mac os ... ) pkg managers so that distutils be able to use
them directly whille still keeping a single model from the end-user
perspective ... At least there exists some kind of python-debian pkg
...

{{{
$ apt-cache show python-debian
Package: python-debian
Priority: optional
Section: universe/devel
Installed-Size: 268
Maintainer: Ubuntu MOTU Developers <ubuntu-motu at lists.ubuntu.com>
Original-Maintainer: Debian python-debian Maintainers
<pkg-python-debian-maint at lists.alioth.debian.org>
Architecture: all
Version: 0.1.9
[...]
Depends: python (>= 2.4), python-support (>= 0.7.1)
Recommends: python-apt
[...]
Size: 43662
[...]
Description: python modules to work with Debian-related data formats
?This package provides python modules that abstract many formats of Debian
?related files. Currently handled are:
?* Debtags information (debian_bundle.debtags module)
?* debian/changelog (debian_bundle.changelog module)
?* Packages files, pdiffs (debian_bundle.debian_support module)
?* Control files of single or multple RFC822-style paragraphs, e.g
? ?debian/control, .changes, .dsc, Packages, Sources, Release, etc.
? ?(debian_bundle.deb822 module)
?* Raw .deb and .ar files, with (read-only) access to contained
? ?files and meta-information
[...]
}}}

Since there are a lot such systems (at least for Linux ...) ... this
could also mean that it could be very complex to handle all this
diversity ... However there are a few which are highly influential ...
but this is certainly a concern ...

- A fully fledged implementation from scratch so that distutils be in
charge, all apps be consistent with Py-std ... and users have
potentially two pkg systems (.. the built-in and the Py-specific ;) ;
and I say potentially since it's quite sudden so far and perhaps there
is a way to smoothly integrate both systems (which are potentially
many of them ;)

> I think the Language Summit tomorrow is a good place to discuss about
> these problems,

Hope it be available (recorded ?) some day so as to hear some opinions ... ;)

> and to make sure pip, setuptools and zc.buildout rely on the same
> standards and pieces.
>

Oh yes !
+1

> PEP 376 is my first attempt to make it happen, and my goal is to see another
> pre-PEP coming out of thea language summit, adressing the dependencies problem.
>

;)

> I can't hear that setuptools has divided the Python community.

Said like this ... some might think that setuptools is responsible for
someone else's choices and actions ... and I dont think so ...

> It has provided
> solutions to real problems we had in web development. It's unperfect,
> and it has to be
> fixed and integrated into Python. But it should not be done outside Python imho.
>

I mostly agree with all this, but I would like to know something ...
in case a single pkg management sys be ready for Py (... I mean
integrated into Py ...) and so on ... Why will we need distutils
comands like :

bdist_msi
bdist_wininst
bdist_rpm
<... more ?non-standard? candidates like py2exe, stdeb, ...>

? Will all this still be necessary and maintained ? As long as
installing /uninstalling them be handled correctly in built-in as well
as Py systems (considering the duplicate ?package repositories? ...)
... I think this could be fine ... but details are details ... ;)

> If you had worked with Zope 5 years ago, you would understand what
> setuptools and
> zc.buildout brought to these communities. And today's experience is a
> whole less painful trust me.
>

Yes ... setuptools has undoubtedly its own value ... :o)

> But I agree that the sizes of the packages are too small now, and it has gone
> to far. Installing a web app like Plone is scary (+100 packages)
>
> But this is the responsability of Zope, TG, etc to distribute their packages in
> bigger pieces I guess.
>

I do think so ... The same happens with Trac plugins. In case of
browsing the archive of trac-users mailing list anyone can realize
that there are many related plugins people'd like to install at once
to get the whole functionality. Nowadays in some cases they have to
install ... *a loooot* and then configure ... *a loooot* in order to
get the whole thing working out ... and there are tiny but useful
plugins ... Besides further contributions seldom end-up with potential
interoperability issues and related features might not be tested as a
whole ...

And things that should be in core (e.g. AccountManager plugin ...
since it seems there is no way to log out if it's not installed ... ;)
are not still there (... took some time until trac-devs decided to
incorporate it during PyCon'09 sprint ... ;)

But this unification is definitely up to the plugin & framew devs ... IMO

--
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:



-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From barry at python.org  Thu Mar 26 20:52:21 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 26 Mar 2009 14:52:21 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
Message-ID: <3580D947-0674-41D4-BC90-A83C6541095D@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 26, 2009, at 2:43 PM, Olemis Lang wrote:
>>>> One thing that /would/ be helpful though is the ability to list  
>>>> all the
>>>> resources under a specific package path.  This is (I think) one  
>>>> use case
>>>> that pkg_resource fails to support and it's the one place that  
>>>> I've had
>>>> to
>>>> drop down to file system introspection.
>>>>
>
> Really ... ? What are all these for ?

E.g. find all the doctests under foo.bar.docs

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScvc9XEjvBPtnXfVAQJT8AP/QhT6Pm4xEL+Bas96cD9waGOTcy1NIYoj
kiWGGbp3VGlXrEaRqDx0f0J51IIQ+VZJMu2M2tMKGMXX4ePrLoJsEavGduN34jat
KvBN43ZbIlgOFcojSmz9pSjGSUw9T7URHUavnS+psz0SipYceWbCghpML/ORtZml
i+bVUwPvRFg=
=9k5E
-----END PGP SIGNATURE-----

From ziade.tarek at gmail.com  Thu Mar 26 20:52:55 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 26 Mar 2009 14:52:55 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <gqglfq$und$2@ger.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>
	<87myb9w6sq.fsf@benfinney.id.au>
	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
	<25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>
	<gqglfq$und$2@ger.gmane.org>
Message-ID: <94bdd2610903261252w6e4d4b92qe8f148ef82534061@mail.gmail.com>

On Thu, Mar 26, 2009 at 2:36 PM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Barry Warsaw wrote:
>> On Mar 25, 2009, at 6:06 PM, Tennessee Leeuwenburg wrote:
>>
>>> For case one, where I want to install additional functionality into my
>>> system Python interpreter "forever", it would be great to have my
>>> system
>>> manage this.
>>
>> In fact, I think it /has/ to. ?I'll go further and say that I'm very
>> wary of using easy_install and the like to install non-distro provided
>> packages into the system Python. ?Many Linux distros require a
>> functioning system Python to operate and the distros (should) take
>> great care to make sure that if you install package X for application
>> Y, you won't break essential system service Z. ?Once you start
>> installing your own stuff in the system Python's site-packages, you
>> break the warranty.
>
> Exactly: ?I never use easy_isntall to put packages into the system
> python. ?in fact, I only use it inside a virtalenv-generated isolated
> environment.

But how each application use 'Python the interpreter' ?

application =  an interpreter + a list of paths + a script to run your
application.

Isolating everything with virtualenv or zc.buildout is a way to go,
but what is really important is not where each package you use is
located but how
Python see and load them.

So if we, for once, forget about the central site-packages and define some
kind of configuration process that is run when every script is launched
to decide what packages should be loaded, we could seperate
 "python the interpreter" from "python the pile of packages"

If some kind of API + configuration file could adress this need, maybe
things could be simpler. You wouldn't fight against a central site-packages
or a per-user site-packages. You would have to explicitely provide it

Tarek
-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From barry at python.org  Thu Mar 26 20:53:03 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 26 Mar 2009 14:53:03 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
Message-ID: <78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 26, 2009, at 2:43 PM, Olemis Lang wrote:

>
> {{{
>>>> [x for x in dir(pkg_resources) if all(y in x for y in ['dir',  
>>>> 'resource_'])]
> ['resource_isdir', 'resource_listdir']

BTW, under a better name, I would support putting pkg_resources in the  
stdlib.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScvdH3EjvBPtnXfVAQKDvQP7B8/POejfswQ8A+Xl4+XJQiO8xHpyPUbC
wpja12gXhamSvBxgoKnFCeTzMgdEoaeK838CR4yvn9YBM7hzJWW0seODG7JpmxGp
w9RXDaf1BuFuxdtTZ8qsoNzRtFQ0LOEh5HtnHYVcpwDHg9FMOKu4cNX19BpvrerU
xwx60X5lsfg=
=mKi4
-----END PGP SIGNATURE-----

From olemis at gmail.com  Thu Mar 26 20:53:38 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 26 Mar 2009 14:53:38 -0500
Subject: [Python-Dev]  "setuptools has divided the Python community"
In-Reply-To: <24ea26600903261215r440694a4j219dfc56bc312b28@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<24ea26600903261215r440694a4j219dfc56bc312b28@mail.gmail.com>
Message-ID: <24ea26600903261253s5f6cf556i8396036756874329@mail.gmail.com>

2009/3/26 Toshio Kuratomi <a.badger at gmail.com>:
> Guido van Rossum wrote:
>> On Wed, Mar 25, 2009 at 9:40 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>> I think Distutils (and therefore Setuptools) should provide some APIs
>>> to play with special files (like resources) and to mark them as being special,
>>> no matter where they end up in the target system.
>>
>> Yes, this should be done. PEP 302 has some hooks but they are optional
>> and not available for the default case.
[...]
>>
>> - Open as a binary stream
>> - Open as a text stream
>> - Get contents as a binary string
>> - Get contents as a text string
>>
> Depending on the definition of a "resource" there's additional
> information that could be needed. ?For instance, if resource includes
> message catalogs, then being able to get the base directory that the
> catalogs reside in is needed for passing to gettext.
>

Well ... considering my experience with Babel (... gettext is similar
too ;) & bundling message catalogs (.MO ...) for Trac plugins needing
i18n support (... I even wrote some kind of i18n architecture for Trac
< 0.12 ... ;), pkg_resources handles these cases too ... dont remember
very well right now how but I think it is some knid of

{{{
>>> import pkg_resources
>>> help(pkg_resources.resource_filename)

resource_filename(self, package_or_requirement, resource_name) method
of pkg_resources.ResourceManager instance
? ?Return a true filesystem path for specified resource
}}}

Don't know whether this is enough or not ... but anyway ... it's
already there, and many people should be using things like this ... Is
it a bad idea to keep it there too ? I mean together with open/get
features mentioned by GvR ...

Q: Will pkg_resources.ResourceManager or equivalent remain ? AFAIK
this is to load resources from multiple sources ... FS, zip | egg
files and so on ... CMIIW ... ;)

--
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:



-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From tseaver at palladion.com  Thu Mar 26 20:58:30 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 26 Mar 2009 15:58:30 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <24ea26600903261247t32af6eb4xde2b957571d44018@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>	<87myb9w6sq.fsf@benfinney.id.au>	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>	<25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>	<gqglfq$und$2@ger.gmane.org>
	<24ea26600903261247t32af6eb4xde2b957571d44018@mail.gmail.com>
Message-ID: <gqgmp7$3ku$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Olemis Lang wrote:
> On Thu, Mar 26, 2009 at 2:36 PM, Tres Seaver <tseaver at palladion.com> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Barry Warsaw wrote:
>>> On Mar 25, 2009, at 6:06 PM, Tennessee Leeuwenburg wrote:
>>>
>>>> For case one, where I want to install additional functionality into my
>>>> system Python interpreter "forever", it would be great to have my
>>>> system
>>>> manage this.
>>> In fact, I think it /has/ to.  I'll go further and say that I'm very
>>> wary of using easy_install and the like to install non-distro provided
>>> packages into the system Python.  Many Linux distros require a
>>> functioning system Python to operate and the distros (should) take
>>> great care to make sure that if you install package X for application
>>> Y, you won't break essential system service Z.  Once you start
>>> installing your own stuff in the system Python's site-packages, you
>>> break the warranty.
>> Exactly:  I never use easy_isntall to put packages into the system
>> python.  in fact, I only use it inside a virtalenv-generated isolated
>> environment.
>>
>>
> 
> What about the proposal I made about the smarter easy_install ...
> isn't that just enough ?

A shared 'site-packages' directory can't possible be sane for an
arbitrary number of applications:  the versions / dependencies installed
to satisfy one app's requirements are going to conflict eventually with
those needed by another.

I treat the 'system' library stuff as the provicne of the package
manager, and keep my own apps in their own, isolated environments, as
created by one or more of:

 - a separately-compiled Python

 - a virtualenv

 - zc.builtout.

Note that the kind of applications I work on tend to be the sort which
will run as server apps, and which will (in production) be the entire
rasion d'etre for the machine they run on, which makes the cost of
isolation tiny compared to the consequences of failed isolation.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJy95m+gerLs4ltQ4RApPNAJ4oMvr9fLMBvL0/zsUUNsEHXBLZPgCbB+bX
AUmLUGUKJusiH5BhvAtdKy8=
=Snse
-----END PGP SIGNATURE-----


From skip at pobox.com  Thu Mar 26 21:03:37 2009
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 26 Mar 2009 15:03:37 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <gqglfq$und$2@ger.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C96304.7050207@v.loewis.de> <49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>
	<87myb9w6sq.fsf@benfinney.id.au>
	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
	<25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>
	<gqglfq$und$2@ger.gmane.org>
Message-ID: <18891.57241.444084.179034@montanaro.dyndns.org>


    Tres> Exactly: I never use easy_isntall to put packages into the system
    Tres> python.  in fact, I only use it inside a virtalenv-generated
    Tres> isolated environment.

While standing in line for lunch today, someone (don't know his name)
suggested that easy_install needs an --i-am-an-idiot flag which would have
to be used to force it to install packages in to the system's space.

Skip

From olemis at gmail.com  Thu Mar 26 21:07:21 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 26 Mar 2009 15:07:21 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <3580D947-0674-41D4-BC90-A83C6541095D@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<3580D947-0674-41D4-BC90-A83C6541095D@python.org>
Message-ID: <24ea26600903261307x1042a7bfn61d9ce8db02c16c7@mail.gmail.com>

On Thu, Mar 26, 2009 at 2:52 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Mar 26, 2009, at 2:43 PM, Olemis Lang wrote:
>>>>>
>>>>> One thing that /would/ be helpful though is the ability to list all the
>>>>> resources under a specific package path. ?This is (I think) one use
>>>>> case
>>>>> that pkg_resource fails to support and it's the one place that I've had
>>>>> to
>>>>> drop down to file system introspection.
>>>>>
>>
>> Really ... ? What are all these for ?
>
> E.g. find all the doctests under foo.bar.docs
>

Could you explain this a little more pls ? You mean ... using doctest
finders in a module (even inside a zip file ;) that's already loaded ?

Sure ... else I dont understand ...

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From tseaver at palladion.com  Thu Mar 26 21:07:16 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 26 Mar 2009 16:07:16 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <18891.57241.444084.179034@montanaro.dyndns.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>	<87myb9w6sq.fsf@benfinney.id.au>	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>	<25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>	<gqglfq$und$2@ger.gmane.org>
	<18891.57241.444084.179034@montanaro.dyndns.org>
Message-ID: <gqgn9l$3ku$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

skip at pobox.com wrote:
>     Tres> Exactly: I never use easy_isntall to put packages into the system
>     Tres> python.  in fact, I only use it inside a virtalenv-generated
>     Tres> isolated environment.
> 
> While standing in line for lunch today, someone (don't know his name)
> suggested that easy_install needs an --i-am-an-idiot flag which would have
> to be used to force it to install packages in to the system's space.

That was Jacob Kaplan-Moss, I think.  In my case, the fact that you have
to type 'sudo' to make it happen is a sufficient reminder. ;)


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJy+B0+gerLs4ltQ4RAiO6AKCGE5277fUhoXYveYqY4CL8dVE7UQCeKbq8
ehCd8UJovafXT/y6dggLKWY=
=OwUF
-----END PGP SIGNATURE-----


From olemis at gmail.com  Thu Mar 26 21:14:24 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 26 Mar 2009 15:14:24 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
Message-ID: <24ea26600903261314i272e3365u2508c76ef4dc073e@mail.gmail.com>

On Thu, Mar 26, 2009 at 2:53 PM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Mar 26, 2009, at 2:43 PM, Olemis Lang wrote:
>
>> {{{
>>>>>
>>>>> [x for x in dir(pkg_resources) if all(y in x for y in ['dir',
>>>>> 'resource_'])]
>>
>> ['resource_isdir', 'resource_listdir']
>
> BTW, under a better name, I would support putting pkg_resources in the
> stdlib.
>

... or a subset of it ? or integrating its features with PEP 302 ? ...
Sounds interesting to me ...

/me thinking aloud

... perhaps unifying these features with loaders in standard PEP 302
is the way ... :-? ... dont see the point of having an isolated module
for doing what loaders should do ... including that there are plans to
do it already ... thereby duplicating functionality in loaders and
ResourceManager, and so on ...

And'u ? What d'u think ?

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From olemis at gmail.com  Thu Mar 26 21:25:48 2009
From: olemis at gmail.com (Olemis Lang)
Date: Thu, 26 Mar 2009 15:25:48 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <18891.57241.444084.179034@montanaro.dyndns.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>
	<87myb9w6sq.fsf@benfinney.id.au>
	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
	<25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>
	<gqglfq$und$2@ger.gmane.org>
	<18891.57241.444084.179034@montanaro.dyndns.org>
Message-ID: <24ea26600903261325t3161d63dq5e4e28db39f02a53@mail.gmail.com>

On Thu, Mar 26, 2009 at 3:03 PM,  <skip at pobox.com> wrote:
>
> ? ?Tres> Exactly: I never use easy_isntall to put packages into the system
> ? ?Tres> python. ?in fact, I only use it inside a virtalenv-generated
> ? ?Tres> isolated environment.
>
> While standing in line for lunch today, someone (don't know his name)
> suggested that easy_install needs an --i-am-an-idiot flag which would have

... since I am feeling like an idiot right now ... perhaps I'd name it
some kind of --let-me-do-the-best-i-can-do-for-you-else-i-will-do-it-my-own-way-even-if-it-is-a-pain-in-your-s-since-i-am-not-smart-enought-yet
...

... well ... it is too long ... :-? ... perhaps it is better this way ...
--lmdtbicdfyeiwdimoweiiiapiyssiansey ... :P

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From guido at python.org  Thu Mar 26 21:28:19 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Mar 2009 15:28:19 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
Message-ID: <ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>

2009/3/26 Barry Warsaw <barry at python.org>:
> BTW, under a better name, I would support putting pkg_resources in the
> stdlib.

Last time I looked it was an incredibly complicated piece of code that
would have to be refactored considerably before it would be
maintainable by the core developers. I never did manage to get a good
understanding of the code, but I expect that a lot of the complexity
exists so that it works for all Python versions. The stdlib version
shouldn't need this -- it should only care about providing a stable
API that works with the current version.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Thu Mar 26 21:58:04 2009
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 26 Mar 2009 15:58:04 -0500 (CDT)
Subject: [Python-Dev] Python svn post-commit hook?
Message-ID: <20090326205804.2E80BE93155@montanaro.dyndns.org>

I'm trying to get the powers that be at work to enable post-commit hooks for
our Subversion repositories.  Here's the default, sans comments:

    REPOS="$1"
    REV="$2"

    commit-email.pl "$REPOS" "$REV" commit-watchers at example.org
    log-commit.py --repository "$REPOS" --revision "$REV"

Does Python's post-commit.tmpl do more (or less) than this?  If it's
substantially different, can someone mail me a copy of what's installed?

Thanks,

Skip

From pje at telecommunity.com  Thu Mar 26 22:16:58 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 26 Mar 2009 17:16:58 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903261220i124a84c6udbabd836509d2524@mail.gmail.co
 m>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<20090325142405.C68463A40A7@sparrow.telecommunity.com>
	<ca471dc20903252124o7635e11cvd047fc9d1de63370@mail.gmail.com>
	<20090326170521.468E93A4097@sparrow.telecommunity.com>
	<ca471dc20903261220i124a84c6udbabd836509d2524@mail.gmail.com>
Message-ID: <20090326211436.1B4843A4097@sparrow.telecommunity.com>

At 12:20 PM 3/26/2009 -0700, Guido van Rossum wrote:
>By brittle I meant again having to be aware of those details of the
>mechanism that exist because of syntactic limitations, e.g.
>accidentally writing "return X" instead of "yield Return(X)".

In that case, you'd either have a syntax error under the current 
rules (because you're using yield in the same function), or else 
you'd have a normal function (no yields) that worked in the way you 
expect it to.

IIRC, the example I gave would treat a non-Return(), non-generator 
value as a value to be passed back into the current generator, such 
that if you defined a function f, and did:

      g = yield f()

and f was not a generator or did not return one, then the above is 
equivalent to:

      g = f()

and is not an error.  Granted, this can fail if f() can return some 
other sort of iterator, but arguably that's an error in the *caller*, 
which should not use yield to call such a function -- and the need to 
distinguish the type of function you're calling is still present in 
PEP 380, i.e., you still need to know whether the function you're 
calling is a generator with a special return mechanism.


From pje at telecommunity.com  Thu Mar 26 22:19:09 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 26 Mar 2009 17:19:09 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com
 >
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
Message-ID: <20090326211646.3392B3A4097@sparrow.telecommunity.com>

At 12:27 PM 3/26/2009 -0700, Guido van Rossum wrote:
>There is some clear low-hanging fruit for Greg's proposal
>where no trampoline or helpers are needed -- but where currently
>refactoring complex code containing many yield statements is
>cumbersome due to the nee to write each "subroutine" call as "for x in
>subroutine(): yield x" -- being able to replace this with "yield from
>subroutine()" is a conceptual advantage to me that is not proportional
>to the number of characters saved.

Right - I don't object to the concept of "yield from" -- I'm -0 on 
that.  What I don't like is the confusion of adding "return values" 
to generators, at least using the 'return' statement.


From benjamin at python.org  Thu Mar 26 22:17:37 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 26 Mar 2009 16:17:37 -0500
Subject: [Python-Dev] Python svn post-commit hook?
In-Reply-To: <20090326205804.2E80BE93155@montanaro.dyndns.org>
References: <20090326205804.2E80BE93155@montanaro.dyndns.org>
Message-ID: <1afaf6160903261417q48e871e6x79c395ecc8fe5bbb@mail.gmail.com>

2009/3/26  <skip at pobox.com>:
> I'm trying to get the powers that be at work to enable post-commit hooks for
> our Subversion repositories. ?Here's the default, sans comments:
>
> ? ?REPOS="$1"
> ? ?REV="$2"
>
> ? ?commit-email.pl "$REPOS" "$REV" commit-watchers at example.org
> ? ?log-commit.py --repository "$REPOS" --revision "$REV"
>
> Does Python's post-commit.tmpl do more (or less) than this? ?If it's
> substantially different, can someone mail me a copy of what's installed?

It's not that different:

#!/bin/sh
/data/repos/projects/hooks/mailer.py commit "$@"
/data/repos/projects/hooks/svn_buildbot.py --repository "$1"
--revision "$2" --bbport 9020 > /dev/null
/data/repos/projects/hooks/svn_buildbot.py --repository "$1"
--revision "$2" --bbport 9070 > /dev/null

if (svnlook changed -r $2 $1|grep peps)
then
    touch /data/website-build/build/status/pepqueued
fi


-- 
Regards,
Benjamin

From pje at telecommunity.com  Thu Mar 26 22:33:10 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 26 Mar 2009 17:33:10 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.co
 m>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
Message-ID: <20090326213047.6A8413A4097@sparrow.telecommunity.com>

At 03:28 PM 3/26/2009 -0500, Guido van Rossum wrote:
>2009/3/26 Barry Warsaw <barry at python.org>:
> > BTW, under a better name, I would support putting pkg_resources in the
> > stdlib.
>
>Last time I looked it was an incredibly complicated piece of code that
>would have to be refactored considerably before it would be
>maintainable by the core developers. I never did manage to get a good
>understanding of the code, but I expect that a lot of the complexity
>exists so that it works for all Python versions. The stdlib version
>shouldn't need this -- it should only care about providing a stable
>API that works with the current version.

As someone else suggested, moving some of the functionality to PEP 
302 interfaces would also help.  Most of the code, though, deals with 
locating/inspecting installed distributions, resolving version 
requirements, and managing sys.path.  And most of the nastiest 
complexity comes from trying to support true filename access to 
resources -- if that were dropped from the stdlib, there'd be no need 
for egg caches and the like, along with all the complexity entailed.

Application environments such as Chandler, Trac, Zope, etc. that want 
their plugins to live in .egg files wouldn't necessarily be able to 
use such an API, but the independent pkg_resources wouldn't be 
disappearing.  (Of course, they could also implement 
application-specific file extraction, if the stdlib API included the 
ability to inspect and open zipped resources.)

The other significant source of complexity is dynamic management of 
namespace packages; specifically, trying to handle the situation 
where new sys.path entries (e.g. .egg files added as plugins) need to 
have their contents added to existing sys.modules __path__ 
entries.  This is perhaps another feature that could be dropped from 
the stdlib version, given a way to interop with pkg_resources or a replacement.


From barry at python.org  Thu Mar 26 22:48:57 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 26 Mar 2009 16:48:57 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <24ea26600903261307x1042a7bfn61d9ce8db02c16c7@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<3580D947-0674-41D4-BC90-A83C6541095D@python.org>
	<24ea26600903261307x1042a7bfn61d9ce8db02c16c7@mail.gmail.com>
Message-ID: <094EF425-5929-48F6-8024-E61D28D5B0EC@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 26, 2009, at 3:07 PM, Olemis Lang wrote:

> On Thu, Mar 26, 2009 at 2:52 PM, Barry Warsaw <barry at python.org>  
> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On Mar 26, 2009, at 2:43 PM, Olemis Lang wrote:
>>>>>>
>>>>>> One thing that /would/ be helpful though is the ability to list  
>>>>>> all the
>>>>>> resources under a specific package path.  This is (I think) one  
>>>>>> use
>>>>>> case
>>>>>> that pkg_resource fails to support and it's the one place that  
>>>>>> I've had
>>>>>> to
>>>>>> drop down to file system introspection.
>>>>>>
>>>
>>> Really ... ? What are all these for ?
>>
>> E.g. find all the doctests under foo.bar.docs
>>
>
> Could you explain this a little more pls ? You mean ... using doctest
> finders in a module (even inside a zip file ;) that's already loaded ?
>
> Sure ... else I dont understand ...

Here's a concrete example.  I have a bunch of .sql files inside the  
mailman.database package.  Their file names encode a sort order.  I  
want to load each one in order and apply them to my database.  I'd  
like to be able to find all these .sql files (which are intermixed  
with .py files in the same package) without having to use os.listdir()  
because that forces the package to live on the file system.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScv4SnEjvBPtnXfVAQIVXgP/eBYSmfYHkebECPVDmCpewZVC9ydi4WdI
agKW41CKPh6PwFFSq35u41KjXEIBIvAcVM9wbYy3zUieTdY+RVe2ceQM/fGILzny
bsd2uLuNHMh0af2fen8JaY1DIbKfKrO3Ea4puvr7Uw1EGjN5IShA0WSgWpO5Ggs7
OKkhmXlPe0A=
=sLWb
-----END PGP SIGNATURE-----

From barry at python.org  Thu Mar 26 22:50:27 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 26 Mar 2009 16:50:27 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <24ea26600903261314i272e3365u2508c76ef4dc073e@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<24ea26600903261314i272e3365u2508c76ef4dc073e@mail.gmail.com>
Message-ID: <3AED5E1E-D988-496B-B1D2-7C25FB489650@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 26, 2009, at 3:14 PM, Olemis Lang wrote:

> On Thu, Mar 26, 2009 at 2:53 PM, Barry Warsaw <barry at python.org>  
> wrote:
>>
>> BTW, under a better name, I would support putting pkg_resources in  
>> the
>> stdlib.
>>
>
> ... or a subset of it ? or integrating its features with PEP 302 ? ...
> Sounds interesting to me ...
>
> /me thinking aloud
>
> ... perhaps unifying these features with loaders in standard PEP 302
> is the way ... :-? ... dont see the point of having an isolated module
> for doing what loaders should do ... including that there are plans to
> do it already ... thereby duplicating functionality in loaders and
> ResourceManager, and so on ...
>
> And'u ? What d'u think ?

Let me clarify my position: I just want the functionality (preferably  
in the stdlib); I don't really care how it's spelled (except please  
not pkg_resource.whatever() :).

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScv4o3EjvBPtnXfVAQLDQQQAsW6so3MMUgGQHndEAkESZSIFNlyTPPtr
ZfAtRcS3y1v+f6qrab7hxY+GTNKhAJnnWEtKbDwzG5NcFMttCKAJQyFr7HL1tvAA
9dbbskN3qjFtjTQ5tz/DJb87pWtFfuisWGp6DWxaUKCVOYWohXxpE/KyGj8KXjOK
d+YAPJeLtDM=
=Lg0j
-----END PGP SIGNATURE-----

From trentm at activestate.com  Thu Mar 26 23:37:54 2009
From: trentm at activestate.com (Trent Mick)
Date: Thu, 26 Mar 2009 15:37:54 -0700
Subject: [Python-Dev] notes from "2 to 3 porting" session of Python Language
 Summit at PyCon
Message-ID: <49CC03C2.8010109@activestate.com>

My notes from the "2 to 3 porting" session of Python Language Summit at 
PyCon. There were some agreements in this session that people wanted 
recorded. Happy to provide clarification in my notes aren't clear.

----
## python 2 to 3 migration

- 2to3 distutils flag (part of install or sdist)
- 2to3 is slow: lots of room for speedup (esp. on the matching)
- 3to2
   - good, wanted, needed: esp. by module authors
   - brett: volunteers? silence. Benjamin? Google SOC. Jesse volunteered to
     mentor.
   - decision: want to look into it
   - Want target compat 2.x (which "x"). Can be done by selecting 
appropriate
     fixers.
   - some discussion of dict views, etc.
- 2.x branch:
   - decision: 18 to 24 month btwn releases.
   - decision: 2.7 (2.6?) *might* be the end of the 2.x line.
- 3.1:
   - in alpha now
   - a2 during the sprints (Brett channeling Benjamin)
   - MvL still has a couple PEPs he wants to propose
   - Jython: considering targetting 3.2 next, *then* maybe 2.7
   - 3.1 will be the one exception to 18-24 mos.
- on "18-24 mo": start a release date PEP with a feature freeze date 
stake in
   the ground when the prev ver is released
- 3.0: special case, not doing 5 years of security releases...
   - hence 3.1 being released soon
   - linux distro guys are fine with that
- extension docs for porting still missing:
   - brett: Benjamin has a draft
   - PortingToPy3k in the wiki
   - Gerhard: doing it against svn
- the other Python implementations: what ver to target
   - depends on their audiences
   - Jython leaning on the latest 3
   - IronPython doesn't want a moving target
   - skip 3.0 and go for 3.1
- python2 & python3 binaries:
   - make install should do it
   - "python3" for new and forever
   - or "python2" for Python 2.x release? Thomas: bad idea because usage of
     "python2" in the shebang line will only work on newer systems

----


Trent

--
Trent Mick
trentm at gmail.com
trentm at activestate.com

From p.f.moore at gmail.com  Fri Mar 27 00:27:01 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 26 Mar 2009 23:27:01 +0000
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <3AED5E1E-D988-496B-B1D2-7C25FB489650@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<24ea26600903261314i272e3365u2508c76ef4dc073e@mail.gmail.com>
	<3AED5E1E-D988-496B-B1D2-7C25FB489650@python.org>
Message-ID: <79990c6b0903261627w5fa157beh836fb26ec764c74e@mail.gmail.com>

2009/3/26 Barry Warsaw <barry at python.org>:
> Let me clarify my position: I just want the functionality (preferably in the
> stdlib); I don't really care how it's spelled (except please not
> pkg_resource.whatever() :).

Agreed. My one major reservation is that conceptually, the whole
pkg_resource infrastructure seems stunningly complex (at this point
PJE pops up to point out that it's not, really, and I'm confusing
pkg_resource with setuptools, or eggs, or some such - let's just say
it makes my head hurt, and defer judgement on whether that's a comment
about pkg_resource or my head :-))

What I'd really like is essentially some form of "virtual filesystem"
access to stuff addressed relative to a Python package name, rather
than to a filesystem path. How much filesystem functionality is
required is somewhat up for debate, but the basic listdir, get
(text/binary) contents, open (text/binary) stream would be a good
start (as Guido pointed out). I know of one or two packages that might
benefit from write access (clearly not available for all loaders -
zipfiles and URLs couldn't do it, but the filesystem and something
like a database could...) But that's just details - the principle
still applies.

Paul.

From pje at telecommunity.com  Fri Mar 27 00:49:32 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Thu, 26 Mar 2009 19:49:32 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <79990c6b0903261627w5fa157beh836fb26ec764c74e@mail.gmail.co
 m>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<24ea26600903261314i272e3365u2508c76ef4dc073e@mail.gmail.com>
	<3AED5E1E-D988-496B-B1D2-7C25FB489650@python.org>
	<79990c6b0903261627w5fa157beh836fb26ec764c74e@mail.gmail.com>
Message-ID: <20090326234710.1AC0C3A4097@sparrow.telecommunity.com>

At 11:27 PM 3/26/2009 +0000, Paul Moore wrote:
>What I'd really like is essentially some form of "virtual filesystem"
>access to stuff addressed relative to a Python package name,

Note that relative to a *Python package name* isn't quite as useful, 
due to namespace packages.  To be unambiguous as to the targeted 
resource, one needs to be able to reference a specific project, and 
that requires you to go off the name of a module *within* a 
package.  For example, 'zope.somemodule' rather than just 'zope'.  To 
put it another way, you don't want to use a module that comes from an 
__init__.py as your origin point, but rather, some other .py file 
within the package.  Then you can talk to that module's loader 
without fear of ambiguity.


>  rather
>than to a filesystem path. How much filesystem functionality is
>required is somewhat up for debate, but the basic listdir, get
>(text/binary) contents, open (text/binary) stream would be a good
>start (as Guido pointed out).

"isdir" and "exists" would also be nice; pkg_resources provides 
those, plus listdir and string/stream.


From tjreedy at udel.edu  Fri Mar 27 02:05:58 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 26 Mar 2009 21:05:58 -0400
Subject: [Python-Dev] GPython?
Message-ID: <gqh8pm$vve$1@ger.gmane.org>

An ars technica articla just linked to in a python-list post

http://arstechnica.com/open-source/news/2009/03/google-launches-project-to-boost-python-performance-by-5x.ars

calls the following project "Google launched"
http://code.google.com/p/unladen-swallow/wiki/ProjectPlan

(Though the project page does not really claim that.)

I am sure some people here might find this interesting.

I'd love to have a faster CPython, but this note:
"Will probably kill Python Windows support (for now)."
would kill merger back into mainline (for now) without one opposing 
being 'conservative'.

If one adds type annotations so that values can be unboxed, would not 
Cython, etc, do even better for speedup?

tjr


From guido at python.org  Fri Mar 27 04:07:51 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Mar 2009 22:07:51 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <20090326234710.1AC0C3A4097@sparrow.telecommunity.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<24ea26600903261314i272e3365u2508c76ef4dc073e@mail.gmail.com>
	<3AED5E1E-D988-496B-B1D2-7C25FB489650@python.org>
	<79990c6b0903261627w5fa157beh836fb26ec764c74e@mail.gmail.com>
	<20090326234710.1AC0C3A4097@sparrow.telecommunity.com>
Message-ID: <ca471dc20903262007k42c4f9fatc71db8906e4f8047@mail.gmail.com>

On Thu, Mar 26, 2009 at 6:49 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 11:27 PM 3/26/2009 +0000, Paul Moore wrote:
>>
>> What I'd really like is essentially some form of "virtual filesystem"
>> access to stuff addressed relative to a Python package name,
>
> Note that relative to a *Python package name* isn't quite as useful, due to
> namespace packages. ?To be unambiguous as to the targeted resource, one
> needs to be able to reference a specific project, and that requires you to
> go off the name of a module *within* a package. ?For example,
> 'zope.somemodule' rather than just 'zope'. ?To put it another way, you don't
> want to use a module that comes from an __init__.py as your origin point,
> but rather, some other .py file within the package. ?Then you can talk to
> that module's loader without fear of ambiguity.

Hm, aren't you thinking of "top-level package" where I wrote
"package"? I would hope that there's always at least one level of
subpackage under a namespace package. Or we could make that a
requirement when distributing code + data files.

If it really is a common habit to have single-file modules with
associated data files directly rooted under a namespace package, we
could change the API to allow passing in a module and have it be
interpreted as "find the data as sibling files of this module." But
when given a package I would still prefer that it would look inside
the package. But to avoid ambiguities when something that's a module
in one version but a package in another I would really prefer to avoid
this.

>> ?rather
>> than to a filesystem path. How much filesystem functionality is
>> required is somewhat up for debate, but the basic listdir, get
>> (text/binary) contents, open (text/binary) stream would be a good
>> start (as Guido pointed out).
>
> "isdir" and "exists" would also be nice; pkg_resources provides those, plus
> listdir and string/stream.

Yes, I forgot to re-read PEP 302 when I wrote that.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Fri Mar 27 04:08:07 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Mar 2009 16:08:07 +1300
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <loom.20090326T102716-650@post.gmane.org>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
Message-ID: <49CC4317.9040908@canterbury.ac.nz>

Antoine Pitrou wrote:
> There seems to be a misunderstanding as to how generators
> are used in Twisted. There isn't a global "trampoline" to schedule generators
> around. Instead, generators are wrapped with a decorator (*) which collects each
> yielded value (it's a Deferred object) and attaches to it a callback which
> resumes (using send()) the execution of the generator whenever the Deferred
> finally gets its value.

This sounds like an architecture that was developed to
work around the lack of anything like yield-from in the
language.

You can't expect to improve something like that by
stuffing yield-from into the existing framework, because
the point of yield-from is to render the framework
itself unnecessary. To take full advantage of it, you
need to step back and re-design the whole thing in
a different way.

In the case of Twisted, I expect the new design would
look a lot like my generator scheduling example.

-- 
Greg

From guido at python.org  Fri Mar 27 04:19:03 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Mar 2009 22:19:03 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <20090326213047.6A8413A4097@sparrow.telecommunity.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
Message-ID: <ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>

On Thu, Mar 26, 2009 at 4:33 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 03:28 PM 3/26/2009 -0500, Guido van Rossum wrote:
>>
>> 2009/3/26 Barry Warsaw <barry at python.org>:
>> > BTW, under a better name, I would support putting pkg_resources in the
>> > stdlib.
>>
>> Last time I looked it was an incredibly complicated piece of code that
>> would have to be refactored considerably before it would be
>> maintainable by the core developers. I never did manage to get a good
>> understanding of the code, but I expect that a lot of the complexity
>> exists so that it works for all Python versions. The stdlib version
>> shouldn't need this -- it should only care about providing a stable
>> API that works with the current version.
>
> As someone else suggested, moving some of the functionality to PEP 302
> interfaces would also help. ?Most of the code, though, deals with
> locating/inspecting installed distributions, resolving version requirements,
> and managing sys.path. ?And most of the nastiest complexity comes from
> trying to support true filename access to resources -- if that were dropped
> from the stdlib, there'd be no need for egg caches and the like, along with
> all the complexity entailed.
>
> Application environments such as Chandler, Trac, Zope, etc. that want their
> plugins to live in .egg files wouldn't necessarily be able to use such an
> API, but the independent pkg_resources wouldn't be disappearing. ?(Of
> course, they could also implement application-specific file extraction, if
> the stdlib API included the ability to inspect and open zipped resources.)
>
> The other significant source of complexity is dynamic management of
> namespace packages; specifically, trying to handle the situation where new
> sys.path entries (e.g. .egg files added as plugins) need to have their
> contents added to existing sys.modules __path__ entries. ?This is perhaps
> another feature that could be dropped from the stdlib version, given a way
> to interop with pkg_resources or a replacement.

We had a pretty long discussion about this during the packages session
at the language workshop today at PyCon. Tarek should probably report
more extensively on this (hopefully he won't take it straight to
distutils-sig) but in summary, the plan is roughly:

- standardize more metadata, especially including dependencies, and
APIs for accessing the metadata and dependencies
- there should be an API to get metadata for a package without
actually executing any of the package's installation script
- this will go into the stdlib for Python 2.7 and 3.1
- it will be provided as separately downloadable backports for prior
versions (maybe using a bootstrap not unlike ez_install)
- keep distutils, but start deprecating certain higher-level
functionality in it (e.g. bdist_rpm)
- don't try to provide higher-level functionality in the stdlib, but
instead let third party tools built on top of these core APIs compete

It will be a while, but once this is a reality, it should be easy
enough for the existing crop of tools (like setuptools, pip,
zc.buildout) to be retargeted for the new API, and the benefit would
be that for *consumers* of packages it doesn't matter which tool was
used.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Fri Mar 27 04:20:30 2009
From: barry at python.org (Barry Warsaw)
Date: Thu, 26 Mar 2009 22:20:30 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <ca471dc20903262007k42c4f9fatc71db8906e4f8047@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<24ea26600903261314i272e3365u2508c76ef4dc073e@mail.gmail.com>
	<3AED5E1E-D988-496B-B1D2-7C25FB489650@python.org>
	<79990c6b0903261627w5fa157beh836fb26ec764c74e@mail.gmail.com>
	<20090326234710.1AC0C3A4097@sparrow.telecommunity.com>
	<ca471dc20903262007k42c4f9fatc71db8906e4f8047@mail.gmail.com>
Message-ID: <1F2C1631-ADB6-4F50-8BF0-8389B701F2E1@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 26, 2009, at 10:07 PM, Guido van Rossum wrote:

> If it really is a common habit to have single-file modules with
> associated data files directly rooted under a namespace package, we
> could change the API to allow passing in a module and have it be
> interpreted as "find the data as sibling files of this module." But
> when given a package I would still prefer that it would look inside
> the package. But to avoid ambiguities when something that's a module
> in one version but a package in another I would really prefer to avoid
> this.

Yes, it should look inside a package.  Very often, a data directory  
will be made a package through the addition of a __init__.py just so  
the data can be found.  There are often no other py files in such data  
packages.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBScxF/nEjvBPtnXfVAQJSrQP/UutsevR7ZL62snMQP++rb0lC8R7u5Uva
KlZkYnro0Y9jzlld1CnAGxMtmzFHYvo7iwfQ6uHEzpDehwN+dqwxnkefTT42X5OT
3o3prDFSdIsBlQdC4JIUn5wxLfHR2LLEB3RL7RROmPfJmhM0+hzidsI0i9rBb84U
B41I6O2+2jI=
=aZ43
-----END PGP SIGNATURE-----

From brett at python.org  Fri Mar 27 04:21:06 2009
From: brett at python.org (Brett Cannon)
Date: Thu, 26 Mar 2009 20:21:06 -0700
Subject: [Python-Dev] GPython?
In-Reply-To: <gqh8pm$vve$1@ger.gmane.org>
References: <gqh8pm$vve$1@ger.gmane.org>
Message-ID: <bbaeab100903262021j68a73dcds3ec41af52ddd0ca4@mail.gmail.com>

On Thu, Mar 26, 2009 at 18:05, Terry Reedy <tjreedy at udel.edu> wrote:

> An ars technica articla just linked to in a python-list post
>
>
> http://arstechnica.com/open-source/news/2009/03/google-launches-project-to-boost-python-performance-by-5x.ars
>
> calls the following project "Google launched"
> http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
>
> (Though the project page does not really claim that.)


> I am sure some people here might find this interesting.
>
> I'd love to have a faster CPython, but this note:
> "Will probably kill Python Windows support (for now)."
> would kill merger back into mainline (for now) without one opposing being
> 'conservative'.
>

Well, that's "for now". It is brand new and still under active development.
View it as a long term development branch. This is not about to replace
CPython suddenly since work on Unladen just started.


>
> If one adds type annotations so that values can be unboxed, would not
> Cython, etc, do even better for speedup?


Nope as Unladen is planning to re-implement the eval loop, something Cython
doesn't optimize without the need to suddenly start adding type annotations
to code.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090326/2bf1bcef/attachment.htm>

From greg.ewing at canterbury.ac.nz  Fri Mar 27 04:26:35 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Mar 2009 16:26:35 +1300
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
 currently conform
In-Reply-To: <ca471dc20903260715g7a830895q879fae3738e50c43@mail.gmail.com>
References: <b8ad139e0903231352i66bc38f3v4f88b0914c760df7@mail.gmail.com>
	<b8ad139e0903231400h570d771bt588ccbc71a295db4@mail.gmail.com>
	<ca471dc20903241314t318f2fb2vf864dae094592f53@mail.gmail.com>
	<49CA27BD.7080904@gmail.com>
	<ca471dc20903252034k332f7c10nfae0d51a9a852e04@mail.gmail.com>
	<49CB0832.1030400@gmail.com> <49CB3D99.8020400@canterbury.ac.nz>
	<49CB49C7.8070503@gmail.com>
	<ca471dc20903260715g7a830895q879fae3738e50c43@mail.gmail.com>
Message-ID: <49CC476B.7060301@canterbury.ac.nz>

Guido van Rossum wrote:

> I'll gladly take that as an added rationalization of my plea not to
> change datetime.

In the case of datetime, could perhaps just the
module name be changed so that it's not the same
as a name inside the module? Maybe call it
date_time or date_and_time.

-- 
Greg



From guido at python.org  Fri Mar 27 04:34:34 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Mar 2009 22:34:34 -0500
Subject: [Python-Dev] Revised**8 PEP on Yield-From
In-Reply-To: <49CB5724.1010403@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com> <gqc4id$ud8$1@ger.gmane.org>
	<20090325033249.C8A3F3A4097@sparrow.telecommunity.com>
	<49C9CBFF.3080705@canterbury.ac.nz> <49CA3021.5050905@gmail.com>
	<ca471dc20903252126t31b03dib87d47d27c4700ed@mail.gmail.com>
	<49CB09BC.70205@gmail.com> <49CB3E23.2020300@canterbury.ac.nz>
	<49CB403D.4000103@canterbury.ac.nz>
	<49CB5724.1010403@canterbury.ac.nz>
Message-ID: <ca471dc20903262034u2e6916c2jd75f8872ad84cbc5@mail.gmail.com>

On Thu, Mar 26, 2009 at 5:21 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Here's a new draft of the PEP. I've added a Motivation
> section and removed any mention of inlining.
>
> There is a new expansion that incorporates recent ideas,
> including the suggested handling of StopIteration raised
> by a throw() call (i.e. if it wasn't the one thrown in,
> treat it as a return value).
>
> Explicit finalization is performed if the delegating
> generator is closed, but not when the subiterator
> completes itself normally.

Submitted to SVN.

I'll try to critique later.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Mar 27 04:39:48 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Mar 2009 22:39:48 -0500
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090326211646.3392B3A4097@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
Message-ID: <ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>

On Thu, Mar 26, 2009 at 4:19 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 12:27 PM 3/26/2009 -0700, Guido van Rossum wrote:
>> There is some clear low-hanging fruit for Greg's proposal
>> where no trampoline or helpers are needed -- but where currently
>> refactoring complex code containing many yield statements is
>> cumbersome due to the nee to write each "subroutine" call as "for x in
>> subroutine(): yield x" -- being able to replace this with "yield from
>> subroutine()" is a conceptual advantage to me that is not proportional
>> to the number of characters saved.
>
> Right - I don't object to the concept of "yield from" -- I'm -0 on that.
> ?What I don't like is the confusion of adding "return values" to generators,
> at least using the 'return' statement.

Well, after thinking about it some more I think I am going to have to
take an executive decision and override your opposition. :-)

I wasn't keen on this myself at the outset, but it's growing on me,
and I am finding Greg's motivation for wanting to do it this way
pretty strong. I'm +1 on "yield from" and +0 on return values in
generators.

That +0 could turn into a +1 if there was a way to flag this as an
error (at runtime), at least if the return is actually executed:

def g():
    yield 42
    return 43

for x in g():
    print x    # probably expected to print 42 and then 43

Perhaps the exception used in this case could be a different exception
than StopIteration? Regular iteration could either just pass this
exception through or explicitly check for it (a single pointer
comparison could usually suffice), depending on whether it would be a
subclass of StopIteration.

Greg, what do you think? (There could be good reasons not to
complexificate it this way too.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From collinw at gmail.com  Fri Mar 27 04:40:27 2009
From: collinw at gmail.com (Collin Winter)
Date: Thu, 26 Mar 2009 22:40:27 -0500
Subject: [Python-Dev] GPython?
In-Reply-To: <gqh8pm$vve$1@ger.gmane.org>
References: <gqh8pm$vve$1@ger.gmane.org>
Message-ID: <43aa6ff70903262040r41e99681o5c7abd33f744a114@mail.gmail.com>

On Thu, Mar 26, 2009 at 8:05 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> An ars technica articla just linked to in a python-list post
>
> http://arstechnica.com/open-source/news/2009/03/google-launches-project-to-boost-python-performance-by-5x.ars
>
> calls the following project "Google launched"
> http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
>
> (Though the project page does not really claim that.)

Hi, I'm the tech lead for Unladen Swallow. Jeffrey Yasskin and Thomas
Wouters are also working on this project.

Unladen Swallow is Google-sponsored, but not Google-owned. This is an
open-source branch that we're working on, focused on performance, and
we want to move all of our work upstream as quickly as possible. In
fact, right now I'm adding a last few tests before putting our cPickle
patches up on the tracker for further review.

> I am sure some people here might find this interesting.
>
> I'd love to have a faster CPython, but this note:
> "Will probably kill Python Windows support (for now)."
> would kill merger back into mainline (for now) without one opposing being
> 'conservative'.

To clarify, when I wrote 'conservative', I wasn't being disparaging. A
resistance to change can certainly be a good thing, and something that
I think is very healthy in these situations. We certainly have to
prove ourselves, especially given some of the fairly radical things
we're thinking of [1]. We believe we can justify these changes, but I
*do* want to be forced to justify them publicly; I don't think
python-dev would be doing its job if some of these things were merely
accepted without discussion.

In particular, Windows support is one of those things we'll need to
address on our end. LLVM's Windows support may be spotty, or there may
be other Windows issues we inadvertently introduce. None of the three
of us have Windows machines, nor do we particularly want to acquire
them :), and Windows support isn't going to be a big priority. If we
find that some of our patches have Windows issues, we will certainly
fix those before proposing their inclusion in CPython.

> If one adds type annotations so that values can be unboxed, would not
> Cython, etc, do even better for speedup?

Possibly, but we want to see how far we can push the current language
before we even start thinking of tinkering with the language spec.
Assigning meaning to function annotations is something that PEP 3107
explicitly avoids, and I'm not sure Unladen Swallow (or anyone else)
would want to take the plunge into coming up with broadly-acceptable
type systems for Python. That would be a bikeshed discussion of such
magnitude, you'd have to invent new colors to paint the thing.

Collin Winter

[1] - http://code.google.com/p/unladen-swallow/wiki/ProjectPlan

From guido at python.org  Fri Mar 27 04:44:49 2009
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Mar 2009 22:44:49 -0500
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
	currently conform
In-Reply-To: <49CC476B.7060301@canterbury.ac.nz>
References: <b8ad139e0903231352i66bc38f3v4f88b0914c760df7@mail.gmail.com>
	<b8ad139e0903231400h570d771bt588ccbc71a295db4@mail.gmail.com>
	<ca471dc20903241314t318f2fb2vf864dae094592f53@mail.gmail.com>
	<49CA27BD.7080904@gmail.com>
	<ca471dc20903252034k332f7c10nfae0d51a9a852e04@mail.gmail.com>
	<49CB0832.1030400@gmail.com> <49CB3D99.8020400@canterbury.ac.nz>
	<49CB49C7.8070503@gmail.com>
	<ca471dc20903260715g7a830895q879fae3738e50c43@mail.gmail.com>
	<49CC476B.7060301@canterbury.ac.nz>
Message-ID: <ca471dc20903262044g72daf3fdl8c7de1990f5e58f@mail.gmail.com>

On Thu, Mar 26, 2009 at 10:26 PM, Greg Ewing
<greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>
>> I'll gladly take that as an added rationalization of my plea not to
>> change datetime.
>
> In the case of datetime, could perhaps just the
> module name be changed so that it's not the same
> as a name inside the module? Maybe call it
> date_time or date_and_time.

I don't think that's advisable ATM -- again, something we should have
done for 3.0, but now it's too late.

I really don't want to set a trend where 3.1 is backwards incompatible
with 3.0 *except* in cases where we were really planning to kill
something in 3.0 and accidentally forgot to quite remove it completely
(like cmp()).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Fri Mar 27 05:14:01 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 27 Mar 2009 00:14:01 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.co
 m>
References: <49C48D03.7030709@gmail.com>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
Message-ID: <20090327041138.EEF893A4097@sparrow.telecommunity.com>

At 10:39 PM 3/26/2009 -0500, Guido van Rossum wrote:
>That +0 could turn into a +1 if there was a way to flag this as an
>error (at runtime), at least if the return is actually executed:
>
>def g():
>     yield 42
>     return 43
>
>for x in g():
>     print x    # probably expected to print 42 and then 43
>
>Perhaps the exception used in this case could be a different exception
>than StopIteration? Regular iteration could either just pass this
>exception through or explicitly check for it (a single pointer
>comparison could usually suffice), depending on whether it would be a
>subclass of StopIteration.

Could we at least have some syntax like 'return from yield with 43', 
to distinguish it from a regular return, clarify that it's returning 
a value to a yield-from statement, and emphasize that you need a 
yield-from to call it?

If it doesn't have some sort of variant syntax, the error message for 
the return exception is going to need to be rather verbose in order 
to be clear.  However, if there is a variant syntax, then an error 
message like "'return from yield' without 'yield from'" might be 
clear enough, and we can keep the current error for returning values 
in generators.

That way, the paired special syntax is clearly identifiable as 
coroutine/microthread control flow, in a way that's both TOOOWTDI and EIBTI.

One remaining quirk or missing piece: ISTM there needs to be a way to 
extract the return value without using a yield-from statement.  I 
mean, you could write a utility function like:

    def unyield(geniter):
        try:
            while 1: geniter.next()
        except GeneratorReturn as v:
            return v.value

OTOH, I suppose this function is still a trampoline, just one that 
doesn't actually do anything except return an eventual exit value.  I 
suppose you could do a slightly improved one thus:

     def unyield(geniter, value=None, func=lambda v: v)
        try:
            while 1: value=func(geniter.send(value))
        except GeneratorReturn as v:
            return v.value

And drop it into itertools or some such.  It's sort of like an 
all-purpose map/reduce for generators, so that all you need to do is 
pass in a function to do whatever processing you need (e.g. I/O 
waiting) on the values yielded.  You could also use another 
generator's send() method as the function passed in, in which case 
you'd basically have a pair of coroutines...  and whichever returned 
a value first would end up as the return value of the overall 
function.  That'd probably be pretty useful for the sort of simple 
(non I/O) coroutines Greg seems to have in mind.

Or, these could just be examples in the PEP, I suppose.  They're not 
terribly difficult to write...  but then I might be biased since I've 
written a ridiculous number of coroutine trampolines for Python 
generators over the last how-many-ever years Python has had generators.


From pje at telecommunity.com  Fri Mar 27 05:17:39 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 27 Mar 2009 00:17:39 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CC4317.9040908@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<49CC4317.9040908@canterbury.ac.nz>
Message-ID: <20090327041515.CECC73A4097@sparrow.telecommunity.com>

At 04:08 PM 3/27/2009 +1300, Greg Ewing wrote:
>You can't expect to improve something like that by
>stuffing yield-from into the existing framework, because
>the point of yield-from is to render the framework
>itself unnecessary.

But it doesn't.  You still need *something* that processes the 
yielded values, since practical frameworks have various things to 
yield "to" - i/o, time, mouse clicks, whatever.  Correctly dealing 
with the call stack part is tedious to implement, sure, but it's not 
really the focal point of a microthreading framework.

Usually, you need to have some way to control which microthreads are 
actually to be executing, vs. the ones that are waiting for a 
particular time, an I/O operation, or some other sort of event.  None 
of that stuff goes away just by taking care of the call stack.


From greg.ewing at canterbury.ac.nz  Fri Mar 27 05:18:40 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Mar 2009 17:18:40 +1300
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
Message-ID: <49CC53A0.7080005@canterbury.ac.nz>

Guido van Rossum wrote:
> Can I suggest that API this takes a glob-style pattern?

Globs would be nice to have, but the minimum
needed is some kind of listdir-like functionality.
Globbing can be built on that if need be.

-- 
Greg

From alexandre at peadrop.com  Fri Mar 27 05:26:09 2009
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Fri, 27 Mar 2009 00:26:09 -0400
Subject: [Python-Dev] GPython?
In-Reply-To: <43aa6ff70903262040r41e99681o5c7abd33f744a114@mail.gmail.com>
References: <gqh8pm$vve$1@ger.gmane.org>
	<43aa6ff70903262040r41e99681o5c7abd33f744a114@mail.gmail.com>
Message-ID: <acd65fa20903262126h679404dbubc6a82294d580cb7@mail.gmail.com>

On Thu, Mar 26, 2009 at 11:40 PM, Collin Winter <collinw at gmail.com> wrote:
> In fact, right now I'm adding a last few tests before putting our cPickle
> patches up on the tracker for further review.
>

Put me in the nosy list when you do; and when I get some free time, I
will give your patches a complete review. I've already taken a quick
look at cPickle changes you did in Unladen and I think some (i.e., the
custom memo table) are definitely worthy to be merged in the
mainlines.

Cheers,
-- Alexandre

From greg.ewing at canterbury.ac.nz  Fri Mar 27 05:29:50 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Mar 2009 17:29:50 +1300
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <24ea26600903261325t3161d63dq5e4e28db39f02a53@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>
	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>
	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>
	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>
	<87myb9w6sq.fsf@benfinney.id.au>
	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
	<25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>
	<gqglfq$und$2@ger.gmane.org>
	<18891.57241.444084.179034@montanaro.dyndns.org>
	<24ea26600903261325t3161d63dq5e4e28db39f02a53@mail.gmail.com>
Message-ID: <49CC563E.8030201@canterbury.ac.nz>

Olemis Lang wrote:

> ... well ... it is too long ... :-? ... perhaps it is better this way ...
> --lmdtbicdfyeiwdimoweiiiapiyssiansey ... :P

Isn't that the name of a town in Wales somewhere?

-- 
Greg



From collinw at gmail.com  Fri Mar 27 05:31:22 2009
From: collinw at gmail.com (Collin Winter)
Date: Thu, 26 Mar 2009 23:31:22 -0500
Subject: [Python-Dev] GPython?
In-Reply-To: <acd65fa20903262126h679404dbubc6a82294d580cb7@mail.gmail.com>
References: <gqh8pm$vve$1@ger.gmane.org>
	<43aa6ff70903262040r41e99681o5c7abd33f744a114@mail.gmail.com>
	<acd65fa20903262126h679404dbubc6a82294d580cb7@mail.gmail.com>
Message-ID: <43aa6ff70903262131t2ac5960dxcf19d719d8fc6b14@mail.gmail.com>

On Thu, Mar 26, 2009 at 11:26 PM, Alexandre Vassalotti
<alexandre at peadrop.com> wrote:
> On Thu, Mar 26, 2009 at 11:40 PM, Collin Winter <collinw at gmail.com> wrote:
>> In fact, right now I'm adding a last few tests before putting our cPickle
>> patches up on the tracker for further review.
>>
>
> Put me in the nosy list when you do; and when I get some free time, I
> will give your patches a complete review. I've already taken a quick
> look at cPickle changes you did in Unladen and I think some (i.e., the
> custom memo table) are definitely worthy to be merged in the
> mainlines.

Will do, thanks for volunteering! jyasskin has already reviewed them
internally, but it'll be good to put them through another set of eyes.

Collin

From eric at trueblade.com  Fri Mar 27 05:37:16 2009
From: eric at trueblade.com (Eric Smith)
Date: Thu, 26 Mar 2009 23:37:16 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <20090326213047.6A8413A4097@sparrow.telecommunity.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>	<49CBCD01.80500@gmail.com>	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
Message-ID: <49CC57FC.30907@trueblade.com>

P.J. Eby wrote:
 > As someone else suggested, moving some of the functionality to PEP 302
 > interfaces would also help.  Most of the code, though, deals with
 > locating/inspecting installed distributions, resolving version
 > requirements, and managing sys.path.  And most of the nastiest
 > complexity comes from trying to support true filename access to
 > resources -- if that were dropped from the stdlib, there'd be no need
 > for egg caches and the like, along with all the complexity entailed.
 >
 > Application environments such as Chandler, Trac, Zope, etc. that want
 > their plugins to live in .egg files wouldn't necessarily be able to use
 > such an API, but the independent pkg_resources wouldn't be
 > disappearing.  (Of course, they could also implement
 > application-specific file extraction, if the stdlib API included the
 > ability to inspect and open zipped resources.)

Could you comment on why they couldn't use such an API?


From greg.ewing at canterbury.ac.nz  Fri Mar 27 06:00:18 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Mar 2009 18:00:18 +1300
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903261220i124a84c6udbabd836509d2524@mail.gmail.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<20090325142405.C68463A40A7@sparrow.telecommunity.com>
	<ca471dc20903252124o7635e11cvd047fc9d1de63370@mail.gmail.com>
	<20090326170521.468E93A4097@sparrow.telecommunity.com>
	<ca471dc20903261220i124a84c6udbabd836509d2524@mail.gmail.com>
Message-ID: <49CC5D62.5050403@canterbury.ac.nz>

Guido van Rossum wrote:

> (Well here is Greg's requested use case for .send(). :-)

There was a complaint that my return-value-with-send
example was too much of a coroutine scenario, so I
was hoping to find something un-coroutine-like. But
if coroutines are the main uses for send in the first
place, then it seems my example is fine as it is.

BTW, I've thought of an answer as to why one would
want to write a parser using send(). In the non-send()
version I had to make the scanner a global variable
in order to avoid passing it around among all the
parsing functions. The send() technique avoids having
to do either of those things.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Mar 27 06:00:29 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Mar 2009 18:00:29 +1300
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
Message-ID: <49CC5D6D.7060108@canterbury.ac.nz>

P.J. Eby wrote:

> And they *still* wouldn't be able to do away with their trampolines -- 

It's not really about doing away with trampolines anyway.
You still need at least one trampoline-like thing at the
top. What you do away with is the need for creating
special objects to yield, and the attendant syntactic
clumisiness and inefficiencies.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Mar 27 06:00:44 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Mar 2009 18:00:44 +1300
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090326170521.468E93A4097@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<20090325142405.C68463A40A7@sparrow.telecommunity.com>
	<ca471dc20903252124o7635e11cvd047fc9d1de63370@mail.gmail.com>
	<20090326170521.468E93A4097@sparrow.telecommunity.com>
Message-ID: <49CC5D7C.6000804@canterbury.ac.nz>

P.J. Eby wrote:
> In particular, it should explain why these choices are so costly as to 
> justify new syntax and a complex implementation:

If avoiding trampolines was the only reason for
yield-from, that mightn't be enough justification
on its own. But it addresses several other use
cases as well. The justification for yield-from is
the total of all those.

> * decorator clearly identifying intent to create a task vs. no 
> indication of task-ness

Currently the only indication that a function is a
generator rather than an ordinary function is the
presence of a 'yield' buried somewhere in the body.
Some people complained about that back when generators
were being designed, but we seem to have gotten used
to it. I don't anticipate any worse problem here.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Mar 27 06:00:57 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Mar 2009 18:00:57 +1300
Subject: [Python-Dev] Revised**8 PEP on Yield-From
In-Reply-To: <49CB6893.10204@gmail.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz> <gqc4id$ud8$1@ger.gmane.org>
	<20090325033249.C8A3F3A4097@sparrow.telecommunity.com>
	<49C9CBFF.3080705@canterbury.ac.nz> <49CA3021.5050905@gmail.com>
	<ca471dc20903252126t31b03dib87d47d27c4700ed@mail.gmail.com>
	<49CB09BC.70205@gmail.com> <49CB3E23.2020300@canterbury.ac.nz>
	<49CB403D.4000103@canterbury.ac.nz> <49CB5724.1010403@canterbury.ac.nz>
	<49CB6893.10204@gmail.com>
Message-ID: <49CC5D89.6040600@canterbury.ac.nz>

Nick Coghlan wrote:
> Since correctly written generators are permitted to
> convert GeneratorExit to StopIteration, the 'yield from' expression
> should detect when that has happened and reraise the original exception.

I'll have to think about that a bit, but you're
probably right.

> it is also necessary to block close() in order to share an
> iterator.

That's a typo -- I meant to say 'throw' and 'close'
there, I think.

-- 
Greg

From glyph at divmod.com  Fri Mar 27 06:19:35 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Fri, 27 Mar 2009 05:19:35 -0000
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
Message-ID: <20090327051935.12555.739678956.divmod.xquotient.6544@weber.divmod.com>

On 26 Mar, 07:22 pm, barry at python.org wrote:
>One thing that /would/ be helpful though is the ability to list all 
>the resources under a specific package path.  This is (I think) one 
>use case that pkg_resource fails to support and it's the one place 
>that I've had to drop down to file system introspection.
>
>Think: the package-y flavor of 
>os.listdir(os.path.dirname(package.__file__))

FWIW, the Twisted spelling of this is:

from twisted.python.modules import getModule
getModule(__name__).filePath.parent().listdir()

(and yes, this works with zip files)

From jimjjewett at gmail.com  Fri Mar 27 06:25:35 2009
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 27 Mar 2009 01:25:35 -0400
Subject: [Python-Dev] return from a generator [was:PEP 380 (yield from a
	subgenerator) comments]
Message-ID: <fb6fbf560903262225j6e1cf04au70cd34a27fb77767@mail.gmail.com>

On Thu, Mar 26, 2009 at 4:19 PM, P.J. Eby wrote:
>> What I don't like is the confusion of adding "return values" to generators,
>> at least using the 'return' statement.

At Fri Mar 27 04:39:48 CET 2009, Guido van Rossum replied:
> I'm +1 on "yield from" and +0 on return values in generators.

>    def g():
>        yield 42
>        return 43

>    for x in g():
>        print x    # probably expected to print 42 and then 43

I still don't see why it needs to be a return statement.  Why not make
the intent of g explicit, by writing either

    def g():
        yield 42
        yield 43

or

    def g():
        yield 42
        raise StopIteration(43)

-jJ

From greg.ewing at canterbury.ac.nz  Fri Mar 27 06:53:23 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Mar 2009 17:53:23 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
References: <49C48D03.7030709@gmail.com>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
Message-ID: <49CC69D3.4030306@canterbury.ac.nz>

Guido van Rossum wrote:

> That +0 could turn into a +1 if there was a way to flag this as an
> error (at runtime), at least if the return is actually executed:
> 
> def g():
>     yield 42
>     return 43
> 
> for x in g():
>     print x    # probably expected to print 42 and then 43
> 
> Perhaps the exception used in this case could be a different exception
> than StopIteration?

Would checking that the value is None be sufficient? Or do you
want to distinguish between 'return' and 'return None'? That
would feel rather strange to me.

I'm inclined to regard this as an unnecessary complication.
People are already trained to think of 'return' and 'yield'
as quite different things. I don't know why they would suddenly
start using 'return' when they mean 'yield'.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Mar 27 07:06:44 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Mar 2009 18:06:44 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090327041138.EEF893A4097@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<20090327041138.EEF893A4097@sparrow.telecommunity.com>
Message-ID: <49CC6CF4.5080603@canterbury.ac.nz>

P.J. Eby wrote:

> Could we at least have some syntax like 'return from yield with 43', to 
> distinguish it from a regular return, clarify that it's returning a 
> value to a yield-from statement, and emphasize that you need a 
> yield-from to call it?

You don't, though -- yield-from just happens to be a particularly
convenient way.

I suppose what you really mean is that you can't just use an
ordinary call. But generators already have that property,
whether they return values or not, and they're already
syntactically marked as such by containing 'yield'. I don't
see that we need a second syntactic marker.

> If it doesn't have some sort of variant syntax, the error message for 
> the return exception is going to need to be rather verbose

If we're going to treat this as an error at all, I imagine
it would say something like "Return value from generator not
used." RTM to sort out the details.

> One remaining quirk or missing piece: ISTM there needs to be a way to 
> extract the return value without using a yield-from statement.

I did suggest a for-loop variant for doing this, but Guido warned
me not to complicate the PEP any further, so I haven't. A followup
PEP for it might be in order.

-- 
Greg

From stefan_ml at behnel.de  Fri Mar 27 07:12:37 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 27 Mar 2009 07:12:37 +0100
Subject: [Python-Dev] GPython?
In-Reply-To: <bbaeab100903262021j68a73dcds3ec41af52ddd0ca4@mail.gmail.com>
References: <gqh8pm$vve$1@ger.gmane.org>
	<bbaeab100903262021j68a73dcds3ec41af52ddd0ca4@mail.gmail.com>
Message-ID: <gqhqol$4kh$1@ger.gmane.org>

Brett Cannon wrote:
> On Thu, Mar 26, 2009 at 18:05, Terry Reedy wrote:
>> If one adds type annotations so that values can be unboxed, would not
>> Cython, etc, do even better for speedup?
> 
> Nope as Unladen is planning to re-implement the eval loop, something Cython
> doesn't optimize without the need to suddenly start adding type annotations
> to code.

Sorry, but this is not correct.

When you let Cython compile a standard Python module, it will not use
Python's eval loop inside the module. It is only used by module external
code. And you can use "pyximport" to let Cython (try to) also compile
external modules on-the-import. This even works for a couple of stdlib
modules by now.

And you are not required to use type annotations to compile your code into
a faster C module. Cython is often faster than CPython by 10-30% even for
plain Python code. Plus, you can use function decorators for type
annotations to keep your code 100% Python (2.4) compatible, and you can put
a .pxd file with type annotations next to your .py module to provide the
type annotations externally. This can get you a speedup of factors, without
modifying your Python code at all.

Stefan


From greg.ewing at canterbury.ac.nz  Fri Mar 27 07:45:05 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Mar 2009 18:45:05 +1200
Subject: [Python-Dev] return from a generator [was:PEP 380 (yield from a
 subgenerator) comments]
In-Reply-To: <fb6fbf560903262225j6e1cf04au70cd34a27fb77767@mail.gmail.com>
References: <fb6fbf560903262225j6e1cf04au70cd34a27fb77767@mail.gmail.com>
Message-ID: <49CC75F1.7010906@canterbury.ac.nz>

Jim Jewett wrote:

> I still don't see why it needs to be a return statement.  Why not make
> the intent of g explicit
> 
>     def g():
>         yield 42
>         raise StopIteration(43)

Because it would be tedious and ugly, and would actually make
the intent *less* clear in the intended use cases. When
writing a suspendable function, you're not thinking about
iteration. You're thinking about writing a function that
performs some computation and possibly returns a value
when finished.

Have you seen my multithreaded server example? Do you
really think the intent of the code would be any clearer
if all the returns were replaced by raising StopIteration?

-- 
Greg

From cmjohnson.mailinglist at gmail.com  Fri Mar 27 07:59:02 2009
From: cmjohnson.mailinglist at gmail.com (Carl Johnson)
Date: Thu, 26 Mar 2009 20:59:02 -1000
Subject: [Python-Dev] return from a generator [was:PEP 380 (yield from a
	subgenerator) comments]
In-Reply-To: <49CC75F1.7010906@canterbury.ac.nz>
References: <fb6fbf560903262225j6e1cf04au70cd34a27fb77767@mail.gmail.com>
	<49CC75F1.7010906@canterbury.ac.nz>
Message-ID: <3bdda690903262359s389ad0e4q27c144116aad3492@mail.gmail.com>

>    def g():
>        yield 42
>        return 43

I don't see how the allowing return values in generators can fail to
be confusing for people new to Python. Of course, there are limits to
the degree that we should let people who are bad at Python dictate our
code practices, but I really just feel like this is going to confuse
people endlessly. "Where did my 43 go? Why didn't it come up in my
loop?"

On the other hand, raising is a pretty well understood way of doing
control flow, if not value passing. But I can see why "raise
StopIteration(42)" might seem a bit obtuse. What if we just subclass
StopIteration and make something called "ReturnValue" or some-such.
Would a simple rename fix part of the problem:

    def g():
        yield 42
        raise ReturnValue(43)


> Do you really think the intent of the code would be any clearer
> if all the returns were replaced by raising StopIteration?

Doesn't "ReturnValue(43)" pretty clearly indicate what's going on? I
think part of the appeal of using "return" is that return is what's
used in ordinary functions, but if you think about it, you already
have to make your cooperative multitasking mini-thread different from
an ordinary function anyway by sprinkling yields throughout it. If
you're going that far, is it really so bad to also change the "return"
to a "raise ReturnValue"?

-- Carl Johnson

From steve at holdenweb.com  Fri Mar 27 08:07:33 2009
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 27 Mar 2009 03:07:33 -0400
Subject: [Python-Dev] Adding PEP consistent aliases for names that don't
 currently conform
In-Reply-To: <ca471dc20903262044g72daf3fdl8c7de1990f5e58f@mail.gmail.com>
References: <b8ad139e0903231352i66bc38f3v4f88b0914c760df7@mail.gmail.com>	<b8ad139e0903231400h570d771bt588ccbc71a295db4@mail.gmail.com>	<ca471dc20903241314t318f2fb2vf864dae094592f53@mail.gmail.com>	<49CA27BD.7080904@gmail.com>	<ca471dc20903252034k332f7c10nfae0d51a9a852e04@mail.gmail.com>	<49CB0832.1030400@gmail.com>
	<49CB3D99.8020400@canterbury.ac.nz>	<49CB49C7.8070503@gmail.com>	<ca471dc20903260715g7a830895q879fae3738e50c43@mail.gmail.com>	<49CC476B.7060301@canterbury.ac.nz>
	<ca471dc20903262044g72daf3fdl8c7de1990f5e58f@mail.gmail.com>
Message-ID: <gqhtvq$au4$1@ger.gmane.org>

Guido van Rossum wrote:
> On Thu, Mar 26, 2009 at 10:26 PM, Greg Ewing
> <greg.ewing at canterbury.ac.nz> wrote:
>> Guido van Rossum wrote:
>>
>>> I'll gladly take that as an added rationalization of my plea not to
>>> change datetime.
>> In the case of datetime, could perhaps just the
>> module name be changed so that it's not the same
>> as a name inside the module? Maybe call it
>> date_time or date_and_time.
> 
> I don't think that's advisable ATM -- again, something we should have
> done for 3.0, but now it's too late.
> 
> I really don't want to set a trend where 3.1 is backwards incompatible
> with 3.0 *except* in cases where we were really planning to kill
> something in 3.0 and accidentally forgot to quite remove it completely
> (like cmp()).
> 
Right. Otherwise pprint.pprint becomes pprint.p_print and ...

That way madness lies. Besides which, what a terrific opportunity to
castigate the Py3k developers forever. Opportunities like that don't
come by every day ;-)

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Fri Mar 27 08:09:25 2009
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 27 Mar 2009 03:09:25 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CC5D62.5050403@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>	<loom.20090321T122452-862@post.gmane.org>	<49C5620A.4080908@canterbury.ac.nz>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	<49C966A0.6090401@canterbury.ac.nz>	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>	<49C9C947.3080608@canterbury.ac.nz>	<20090325142405.C68463A40A7@sparrow.telecommunity.com>	<ca471dc20903252124o7635e11cvd047fc9d1de63370@mail.gmail.com>	<20090326170521.468E93A4097@sparrow.telecommunity.com>	<ca471dc20903261220i124a84c6udbabd836509d2524@mail.gmail.com>
	<49CC5D62.5050403@canterbury.ac.nz>
Message-ID: <49CC7BA5.1060102@holdenweb.com>

Greg Ewing wrote:
> Guido van Rossum wrote:
> 
>> (Well here is Greg's requested use case for .send(). :-)
> 
> There was a complaint that my return-value-with-send
> example was too much of a coroutine scenario, so I
> was hoping to find something un-coroutine-like. But
> if coroutines are the main uses for send in the first
> place, then it seems my example is fine as it is.
> 
> BTW, I've thought of an answer as to why one would
> want to write a parser using send(). In the non-send()
> version I had to make the scanner a global variable
> in order to avoid passing it around among all the
> parsing functions. The send() technique avoids having
> to do either of those things.
> 
Someone pointed out in my tutorial today that the circuits module makes
heavy use of the generators' two-way communications capability. Maybe of
you looked at the code you would find low-hanging fruit you could pick
for sensible use cases.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Fri Mar 27 08:09:25 2009
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 27 Mar 2009 03:09:25 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CC5D62.5050403@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>	<loom.20090321T122452-862@post.gmane.org>	<49C5620A.4080908@canterbury.ac.nz>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	<49C966A0.6090401@canterbury.ac.nz>	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>	<49C9C947.3080608@canterbury.ac.nz>	<20090325142405.C68463A40A7@sparrow.telecommunity.com>	<ca471dc20903252124o7635e11cvd047fc9d1de63370@mail.gmail.com>	<20090326170521.468E93A4097@sparrow.telecommunity.com>	<ca471dc20903261220i124a84c6udbabd836509d2524@mail.gmail.com>
	<49CC5D62.5050403@canterbury.ac.nz>
Message-ID: <49CC7BA5.1060102@holdenweb.com>

Greg Ewing wrote:
> Guido van Rossum wrote:
> 
>> (Well here is Greg's requested use case for .send(). :-)
> 
> There was a complaint that my return-value-with-send
> example was too much of a coroutine scenario, so I
> was hoping to find something un-coroutine-like. But
> if coroutines are the main uses for send in the first
> place, then it seems my example is fine as it is.
> 
> BTW, I've thought of an answer as to why one would
> want to write a parser using send(). In the non-send()
> version I had to make the scanner a global variable
> in order to avoid passing it around among all the
> parsing functions. The send() technique avoids having
> to do either of those things.
> 
Someone pointed out in my tutorial today that the circuits module makes
heavy use of the generators' two-way communications capability. Maybe of
you looked at the code you would find low-hanging fruit you could pick
for sensible use cases.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Fri Mar 27 08:11:57 2009
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 27 Mar 2009 03:11:57 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
References: <49C48D03.7030709@gmail.com>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	<49C966A0.6090401@canterbury.ac.nz>	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>	<49C9C947.3080608@canterbury.ac.nz>	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>	<loom.20090326T102716-650@post.gmane.org>	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
Message-ID: <49CC7C3D.9090808@holdenweb.com>

Guido van Rossum wrote:
[...] (There could be good reasons not to
> complexificate it this way too.)
> 
You've been 'Mercanized. This is the most worstest English I have ever
seen you write.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Fri Mar 27 08:11:57 2009
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 27 Mar 2009 03:11:57 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
References: <49C48D03.7030709@gmail.com>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	<49C966A0.6090401@canterbury.ac.nz>	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>	<49C9C947.3080608@canterbury.ac.nz>	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>	<loom.20090326T102716-650@post.gmane.org>	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
Message-ID: <49CC7C3D.9090808@holdenweb.com>

Guido van Rossum wrote:
[...] (There could be good reasons not to
> complexificate it this way too.)
> 
You've been 'Mercanized. This is the most worstest English I have ever
seen you write.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From steve at holdenweb.com  Fri Mar 27 08:21:44 2009
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 27 Mar 2009 03:21:44 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CC69D3.4030306@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	<49C966A0.6090401@canterbury.ac.nz>	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>	<49C9C947.3080608@canterbury.ac.nz>	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>	<loom.20090326T102716-650@post.gmane.org>	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>	<20090326211646.3392B3A4097@sparrow.telecommunity.com>	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<49CC69D3.4030306@canterbury.ac.nz>
Message-ID: <49CC7E88.9020102@holdenweb.com>

Greg Ewing wrote:
> Guido van Rossum wrote:
> 
>> That +0 could turn into a +1 if there was a way to flag this as an
>> error (at runtime), at least if the return is actually executed:
>>
>> def g():
>>     yield 42
>>     return 43
>>
>> for x in g():
>>     print x    # probably expected to print 42 and then 43
>>
>> Perhaps the exception used in this case could be a different exception
>> than StopIteration?
> 
> Would checking that the value is None be sufficient? Or do you
> want to distinguish between 'return' and 'return None'? That
> would feel rather strange to me.
> 
> I'm inclined to regard this as an unnecessary complication.
> People are already trained to think of 'return' and 'yield'
> as quite different things. I don't know why they would suddenly
> start using 'return' when they mean 'yield'.
> 
Probably because they just started to use Python and they just don't
understand WT* you are talking about. Reviewing this thread reminds me
of the story about the Princeton math professor who started his lecture
by writing a complex three-line formula on the blackboard and saying
"From this it is obvious that ..." and then proceeded to gaze at the
blackboard in silence for three minutes. He then left the lecture
theater, somewhat to his students' consternation, only to reappear with
a smile on his face in ten minutes to say "Yes, it *is* obvious that ...".

I am a *bit* concerned, without really being able to put my finger on
it, that the "yield from" expression's value comes from inside (the
"return" from the nested generator) while the "yield from" expression's
value comes from "outside" (the value passed to a .send() method call).

Please forgive this bear of very little brain. I suspect the
documentation will need to make this asymmetry more obvious still.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From scott+python-dev at scottdial.com  Fri Mar 27 08:28:55 2009
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Fri, 27 Mar 2009 03:28:55 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090327041138.EEF893A4097@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	<49C966A0.6090401@canterbury.ac.nz>	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>	<49C9C947.3080608@canterbury.ac.nz>	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>	<loom.20090326T102716-650@post.gmane.org>	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>	<20090326211646.3392B3A4097@sparrow.telecommunity.com>	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<20090327041138.EEF893A4097@sparrow.telecommunity.com>
Message-ID: <49CC8037.1010909@scottdial.com>

P.J. Eby wrote:
> One remaining quirk or missing piece: ISTM there needs to be a way to
> extract the return value without using a yield-from statement.  I mean,
> you could write a utility function like:
> 
>    def unyield(geniter):
>        try:
>            while 1: geniter.next()
>        except GeneratorReturn as v:
>            return v.value

My first thought was to ask why it was not equivalent to say:

    x = yield g
    x = yield from g

This would seem like a more obvious lack of parallelism to pick on wrt.
return values.

This unyield() operation seems contrived. Never before have you been
able to write a generator that returns a value, why would these suddenly
become common practice? The only place a return value seems useful is
when refactoring a generator and you need to mend having loss of a
shared scope. What other use is there for a return value?

It would seem unfortunate for it to be considered a runtime error since
this would prevent sharing a generator amongst "yield from" and
non-"yield from" use cases. Although, it would be trivial to do:

    class K:
        ...
        def _f():
            yield 1
            return 2 # used internally
        def f()
            # squelch the runtime error
            yield from self._f()

As Greg has said a number of times, we allow functions to return values
with them silently being ignored all the time.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From techtonik at gmail.com  Fri Mar 27 08:42:56 2009
From: techtonik at gmail.com (anatoly techtonik)
Date: Fri, 27 Mar 2009 09:42:56 +0200
Subject: [Python-Dev] version compare function into main lib
Message-ID: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>

Greetings,

Correct me if I wrong, but shouldn't Python include function for
version comparisons?

The problem of splitting a string like "10.3.40-beta" into tuple for
cmp() seems to be dead simple, but the code gets bulky with excepting
errors arised from converting resulting strings for arithmetic
comparison. No wonder people advise to use ready distutils.version
module, but that's not good to introduce such dependency for main
program logic.

What do you think about adding cmpversions(first, second,
strict=false) based on distutils into main lib?
Will it be more appropriate to isolate the function into "versions" module?
Should it be rewritten to remove re dependency in this case?


Distutils version comparisons:
http://svn.python.org/view/python/branches/release26-maint/Lib/distutils/version.py?view=markup

--anatoly t.

From steve at holdenweb.com  Fri Mar 27 08:49:33 2009
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 27 Mar 2009 03:49:33 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <20090327051935.12555.739678956.divmod.xquotient.6544@weber.divmod.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>	<49CAF030.80707@gmail.com>	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>	<49CBCD01.80500@gmail.com>	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<20090327051935.12555.739678956.divmod.xquotient.6544@weber.divmod.com>
Message-ID: <gqi0ei$gma$1@ger.gmane.org>

glyph at divmod.com wrote:
> On 26 Mar, 07:22 pm, barry at python.org wrote:
>> One thing that /would/ be helpful though is the ability to list all
>> the resources under a specific package path.  This is (I think) one
>> use case that pkg_resource fails to support and it's the one place
>> that I've had to drop down to file system introspection.
>>
>> Think: the package-y flavor of
>> os.listdir(os.path.dirname(package.__file__))
> 
> FWIW, the Twisted spelling of this is:
> 
> from twisted.python.modules import getModule
> getModule(__name__).filePath.parent().listdir()
> 
> (and yes, this works with zip files)

Careful, Glyph. Nobody likes a smart-ass ;-)

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From tjreedy at udel.edu  Fri Mar 27 09:13:44 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 27 Mar 2009 04:13:44 -0400
Subject: [Python-Dev] GPython?
In-Reply-To: <43aa6ff70903262040r41e99681o5c7abd33f744a114@mail.gmail.com>
References: <gqh8pm$vve$1@ger.gmane.org>
	<43aa6ff70903262040r41e99681o5c7abd33f744a114@mail.gmail.com>
Message-ID: <gqi1ro$ktk$1@ger.gmane.org>

Collin Winter wrote:
> On Thu, Mar 26, 2009 at 8:05 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>> An ars technica articla just linked to in a python-list post
>>
>> http://arstechnica.com/open-source/news/2009/03/google-launches-project-to-boost-python-performance-by-5x.ars
>>
>> calls the following project "Google launched"
>> http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
>>
>> (Though the project page does not really claim that.)
> 
> Hi, I'm the tech lead for Unladen Swallow. Jeffrey Yasskin and Thomas
> Wouters are also working on this project.
> 
> Unladen Swallow is Google-sponsored, but not Google-owned. This is an
> open-source branch that we're working on, focused on performance, and
> we want to move all of our work upstream as quickly as possible. In
> fact, right now I'm adding a last few tests before putting our cPickle
> patches up on the tracker for further review.

Thank you for the answers and comments.

Pickle speedups will be welcomed by many.  It comes up on python-list 
from time to time.
> 
>> I am sure some people here might find this interesting.
>>
>> I'd love to have a faster CPython, but this note:
>> "Will probably kill Python Windows support (for now)."
>> would kill merger back into mainline (for now) without one opposing being
>> 'conservative'.
> 
> To clarify, when I wrote 'conservative', I wasn't being disparaging. A

Sorry I was mislead.  Perhaps you might want to rewrite to sound more 
like what you have written here -- something like "There is a risk we 
will be unable to justify the more radical changes that we propose."

> resistance to change can certainly be a good thing, and something that
> I think is very healthy in these situations. We certainly have to
> prove ourselves, especially given some of the fairly radical things
> we're thinking of [1]. We believe we can justify these changes, but I
> *do* want to be forced to justify them publicly; I don't think
> python-dev would be doing its job if some of these things were merely
> accepted without discussion.
[snip]

Terry


From p.f.moore at gmail.com  Fri Mar 27 11:22:00 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 27 Mar 2009 10:22:00 +0000
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
Message-ID: <79990c6b0903270322r13e11969ja64f732b5bc8698f@mail.gmail.com>

2009/3/27 Guido van Rossum <guido at python.org>:
> - keep distutils, but start deprecating certain higher-level
> functionality in it (e.g. bdist_rpm)
> - don't try to provide higher-level functionality in the stdlib, but
> instead let third party tools built on top of these core APIs compete

Please don't move bdist_wininst out of the core, though!

I'd argue that Windows is a special case, as many Windows users don't
have the ability to build their own extensions, so they rely heavily
on binary installers. And there's no "Windows packagers" organisation
to produce such things in the way that Linux has people building debs,
rpms, etc. Making it as easy as possible for a random developer to
build a Windows installer (by including bdist_wininst or equivalent in
the core) is therefore significantly more beneficial than doing the
same for a Linux/Mac/Solaris/... installer (Mac users or anyone else
speak up here if I'm misrepresenting you!)

>From what I understand of the issues, the problems with
bdist_rpm/deb/etc have always been the need to conform to externally
defined standards such as the Linux filesystem policy, or Debian's
policies. As Windows has no such policies (or any that do exist are
routinely ignored :-)) there is no pressure to "fix" bdist_wininst as
policies change.

Sorry if this was discussed at the session already.

Paul.

PS I'm using bdist_wininst as an example here. If someone were to come
up with an alternative (core) means of building Windows installers for
Python packages (maybe based on a common binary format such as eggs)
that would be equally good. It's having no way in the core of building
Windows installers that I'm arguing against.

From p.f.moore at gmail.com  Fri Mar 27 11:50:20 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 27 Mar 2009 10:50:20 +0000
Subject: [Python-Dev] GPython?
In-Reply-To: <43aa6ff70903262040r41e99681o5c7abd33f744a114@mail.gmail.com>
References: <gqh8pm$vve$1@ger.gmane.org>
	<43aa6ff70903262040r41e99681o5c7abd33f744a114@mail.gmail.com>
Message-ID: <79990c6b0903270350g2c2c78eeu5bd9e9f6b78a5bb4@mail.gmail.com>

2009/3/27 Collin Winter <collinw at gmail.com>:
> In particular, Windows support is one of those things we'll need to
> address on our end. LLVM's Windows support may be spotty, or there may
> be other Windows issues we inadvertently introduce. None of the three
> of us have Windows machines, nor do we particularly want to acquire
> them :), and Windows support isn't going to be a big priority. If we
> find that some of our patches have Windows issues, we will certainly
> fix those before proposing their inclusion in CPython.

On the assumption (sorry, I've done little more than read the press
releases so far) that you're starting from the CPython base and
incrementally patching things, you currently have strong Windows
support. It would be a shame if that got gradually chipped away
through neglect, until it became a big job to reinstate it.

If the Unladen Swallow team doesn't include any Windows developers,
you're a bit stuck, I guess, but could you not at least have a Windows
buildbot which keeps tabs on the current status? Then you might
encourage interested Windows bystanders to check in occasionally and
maybe offer fixes.

As things stand, the press releases give me the impression (as a
Windows user without a lot of time to invest in contributing) that
this project is irrelevant to me, and I should ignore it until you
announce "proper" Windows support. By which time, it may have fallen
completely off my radar. (On a smaller scale, this happened with
virtualenv - I found to my surprise that it now supported Windows, and
had for some time without me realising, because when it started it was
Unix-only and I had not bothered to keep track of it). Maybe again
it's something that could be clarified in the announcements.

Paul.

From greg.ewing at canterbury.ac.nz  Fri Mar 27 11:54:03 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Mar 2009 22:54:03 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CC7E88.9020102@holdenweb.com>
References: <49C48D03.7030709@gmail.com>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<49CC69D3.4030306@canterbury.ac.nz> <49CC7E88.9020102@holdenweb.com>
Message-ID: <49CCB04B.9030501@canterbury.ac.nz>

Steve Holden wrote:

> I am a *bit* concerned, without really being able to put my finger on
> it, that the "yield from" expression's value comes from inside (the
> "return" from the nested generator) while the "yield from" expression's
> value comes from "outside" (the value passed to a .send() method call).

The send() calls don't go to the yield-from expression, they
go to the yields inside the subgenerator. Similarly, next()
calls get their values from the yields inside the subgenerator.

The only time the yield-from expression itself receives a
value is right at the very end when the subgenerator
terminates, and that's not a yielding operation, it's a
returning operation.

Yield-from is not a kind of yield. It's a kind of function
call.

-- 
Greg


From list-ener at strank.info  Fri Mar 27 11:53:31 2009
From: list-ener at strank.info (Stefan Rank)
Date: Fri, 27 Mar 2009 11:53:31 +0100
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090327041515.CECC73A4097@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>	<loom.20090321T122452-862@post.gmane.org>	<49C5620A.4080908@canterbury.ac.nz>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	<49C966A0.6090401@canterbury.ac.nz>	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>	<49C9C947.3080608@canterbury.ac.nz>	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>	<loom.20090326T102716-650@post.gmane.org>	<49CC4317.9040908@canterbury.ac.nz>
	<20090327041515.CECC73A4097@sparrow.telecommunity.com>
Message-ID: <49CCB02B.8010705@strank.info>

on 2009-03-27 05:17 P.J. Eby said the following:
> At 04:08 PM 3/27/2009 +1300, Greg Ewing wrote:
>> You can't expect to improve something like that by
>> stuffing yield-from into the existing framework, because
>> the point of yield-from is to render the framework
>> itself unnecessary.
> 
> But it doesn't.  You still need *something* that processes the yielded 
> values, since practical frameworks have various things to yield "to" - 
> i/o, time, mouse clicks, whatever.  Correctly dealing with the call 
> stack part is tedious to implement, sure, but it's not really the focal 
> point of a microthreading framework.

I can chime in here with a use case, if an unusual one. I implemented 
just such a framework based on generator syntax for my thesis work to 
model the behaviour of software agents as a collection of interacting 
activities (microprocesses). The top layer is based on Twisted (similar 
to its _inlineCallbacks) and different schedulers decide on what to do 
with yielded values.
This is really very similar to Philip Eby's code, the main difference 
being that one uses a generic function yield_to as an extension point 
and the other one uses (subclasses of) Deferreds.
You can handle the call stack in the Deferred-based case just as 
clumsily as in the other :-).

And in my system, the "call stack" (i.e. the hierarchy of active 
microprocesses) and how it can be manipulated by the agent is actually 
the interesting part.

> Usually, you need to have some way to control which microthreads are 
> actually to be executing, vs. the ones that are waiting for a particular 
> time, an I/O operation, or some other sort of event.  None of that stuff 
> goes away just by taking care of the call stack.

Yes. However, the valuable addition that an explicit yield from syntax 
would provide for my use case is a way to explicitly distinguish between 
subgenerators just for the sake of refactoring code vs. sub-"processes".
I could remove quite some duplication from my current code.

Additionally, as noted in the PEP, it would open the path for 
optimisations of the refactoring cases.
I also think that a separation of handling the generator call stack and 
handling yielded values improves the situation for scheduling/trampoline 
authors conceptually.

Just my 0.02?
cheers,
stefan


From solipsis at pitrou.net  Fri Mar 27 11:56:36 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 27 Mar 2009 10:56:36 +0000 (UTC)
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<49CC5D6D.7060108@canterbury.ac.nz>
Message-ID: <loom.20090327T105353-433@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> 
> It's not really about doing away with trampolines anyway.
> You still need at least one trampoline-like thing at the
> top. What you do away with is the need for creating
> special objects to yield, and the attendant syntactic
> clumisiness and inefficiencies.

No you don't, not in the Twisted case. The fact that useful library routines
return Deferred objects to which you add callbacks and errbacks is probably not
going away, because it's a fundamental building block in Twisted, not a
convenience for scheduling generators.

As a matter of fact, the people whom this PEP is supposed to benefit haven't
expressed a lot of enthusiasm right now. That's why it looks so academic.

Regards

Antoine.



From eric at trueblade.com  Fri Mar 27 11:57:00 2009
From: eric at trueblade.com (Eric Smith)
Date: Fri, 27 Mar 2009 05:57:00 -0500
Subject: [Python-Dev] version compare function into main lib
In-Reply-To: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>
References: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>
Message-ID: <49CCB0FC.2010609@trueblade.com>

anatoly techtonik wrote:
> Correct me if I wrong, but shouldn't Python include function for
> version comparisons?

<snip>

> What do you think about adding cmpversions(first, second,
> strict=false) based on distutils into main lib?

distutils _is_ already in the "main lib", that is, the standard library.

> Will it be more appropriate to isolate the function into "versions" module?
> Should it be rewritten to remove re dependency in this case?

Given that re is also in the standard library, and this is hardly speed
critical, I'd say no.

> Distutils version comparisons:
> http://svn.python.org/view/python/branches/release26-maint/Lib/distutils/version.py?view=markup

I don't see the point of moving this, unless it's part of some larger,
radical "fix distutils" effort. And even then I'm not convinced.

This probably belongs on the python-ideas mailing list, or on the
distutils SIG list. I expect you'll see a lot of discussion on distutils
SIG list in the coming days.

Eric.



From ncoghlan at gmail.com  Fri Mar 27 12:05:45 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Mar 2009 21:05:45 +1000
Subject: [Python-Dev] GPython?
In-Reply-To: <43aa6ff70903262040r41e99681o5c7abd33f744a114@mail.gmail.com>
References: <gqh8pm$vve$1@ger.gmane.org>
	<43aa6ff70903262040r41e99681o5c7abd33f744a114@mail.gmail.com>
Message-ID: <49CCB309.80404@gmail.com>

Collin Winter wrote:
> That would be a bikeshed discussion of such
> magnitude, you'd have to invent new colors to paint the thing.

Octarine. Definitely octarine :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From greg.ewing at canterbury.ac.nz  Fri Mar 27 12:19:54 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Mar 2009 23:19:54 +1200
Subject: [Python-Dev] Revised**9 PEP on Yield-From
In-Reply-To: <loom.20090327T105353-433@post.gmane.org>
References: <49C48D03.7030709@gmail.com>
	<loom.20090321T122452-862@post.gmane.org>
	<49C5620A.4080908@canterbury.ac.nz>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<49CC5D6D.7060108@canterbury.ac.nz>
	<loom.20090327T105353-433@post.gmane.org>
Message-ID: <49CCB65A.4020407@canterbury.ac.nz>

Draft 10 of the PEP. Removed the outer try-finally
from the expansion and fixed it to re-raise
GeneratorExit if the throw call raises StopIteration.

-- 
Greg

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: yield-from-rev10.txt
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090327/462900b0/attachment-0001.txt>

From greg.ewing at canterbury.ac.nz  Fri Mar 27 12:20:34 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Mar 2009 23:20:34 +1200
Subject: [Python-Dev] version compare function into main lib
In-Reply-To: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>
References: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>
Message-ID: <49CCB682.8050106@canterbury.ac.nz>

anatoly techtonik wrote:

> Correct me if I wrong, but shouldn't Python include function for
> version comparisons?

Can't you just compare sys.version_info tuples?

 >>> sys.version_info
(2, 5, 0, 'final', 0)

Assuming the other possibilities for 'final' are
'alpha' and 'beta', these should compare in the
right order.

-- 
Greg


From foobarmus at gmail.com  Fri Mar 27 09:57:38 2009
From: foobarmus at gmail.com (Mark Donald)
Date: Fri, 27 Mar 2009 16:57:38 +0800
Subject: [Python-Dev] suggestion for try/except program flow
Message-ID: <e5b661850903270157yb43d076jdfd0fdd724902bc4@mail.gmail.com>

I frequently have this situation:

try:
    try:
        raise Thing
    except Thing, e:
        # handle Thing exceptions
        raise
except:
    # handle all exceptions, including Thing

It would be much more readable if there were a way to recatch a named
exception with the generic (catch-all) except clause of its own try,
something like this:

try:
    raise Thing
except Thing, e:
    # handle Thing exceptions
    recatch
except:
    # handle all exceptions, including Thing

From owner at bugs.debian.org  Thu Mar 26 12:39:06 2009
From: owner at bugs.debian.org (Debian Bug Tracking System)
Date: Thu, 26 Mar 2009 11:39:06 +0000
Subject: [Python-Dev] Bug#521275: Acknowledgement (colored prompt conflicts
 with cursor positioning)
References: <200903261236.21657.eckhardt@satorlaser.com>
Message-ID: <handler.521275.B.123806739521183.ack@bugs.debian.org>


Thank you for filing a new Bug report with Debian.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 Matthias Klose <doko at debian.org>

If you wish to submit further information on this problem, please
send it to 521275 at bugs.debian.org, as before.

Please do not send mail to owner at bugs.debian.org unless you wish
to report a problem with the Bug-tracking system.


-- 
521275: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=521275
Debian Bug Tracking System
Contact owner at bugs.debian.org with problems

From bignose+hates-spam at benfinney.id.au  Fri Mar 27 12:28:27 2009
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Fri, 27 Mar 2009 22:28:27 +1100
Subject: [Python-Dev] "setuptools has divided the Python community"
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<79990c6b0903270322r13e11969ja64f732b5bc8698f@mail.gmail.com>
Message-ID: <878wmrtdb8.fsf@benfinney.id.au>

Paul Moore <p.f.moore at gmail.com> writes:

> Please don't move bdist_wininst out of the core, though!
> 
> I'd argue that Windows is a special case, as many Windows users
> don't have the ability to build their own extensions, so they rely
> heavily on binary installers. And there's no "Windows packagers"
> organisation to produce such things in the way that Linux has people
> building debs, rpms, etc.

This is the case for many free-software projects; yet such projects
(depending on popularity) *do* accumulate volunteers capable of
building binary packages for their platform, and then contributing
them back to the project for use by all.

None of this needs explicit special handling in the ?distutils? core,
IIUC.

> Making it as easy as possible for a random developer to build a
> Windows installer (by including bdist_wininst or equivalent in the
> core) is therefore significantly more beneficial than doing the same
> for a Linux/Mac/Solaris/... installer (Mac users or anyone else
> speak up here if I'm misrepresenting you!)

I would argue that the Python community has a wealth of people quite
capable of taking on this particular task, and if it makes the core
architecture and maintenance of ?distutils? simpler to remove special
cases for binary installers, I think that's a pearl of great price.

-- 
 \        ?Holy uncanny photographic mental processes, Batman!? ?Robin |
  `\                                                                   |
_o__)                                                                  |
Ben Finney


From ncoghlan at gmail.com  Fri Mar 27 12:38:35 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Mar 2009 21:38:35 +1000
Subject: [Python-Dev] return from a generator [was:PEP 380 (yield from
 a	subgenerator) comments]
In-Reply-To: <3bdda690903262359s389ad0e4q27c144116aad3492@mail.gmail.com>
References: <fb6fbf560903262225j6e1cf04au70cd34a27fb77767@mail.gmail.com>	<49CC75F1.7010906@canterbury.ac.nz>
	<3bdda690903262359s389ad0e4q27c144116aad3492@mail.gmail.com>
Message-ID: <49CCBABB.8070907@gmail.com>

Carl Johnson wrote:
> I think part of the appeal of using "return" is that return is what's
>  used in ordinary functions, but if you think about it, you already 
> have to make your cooperative multitasking mini-thread different from
>  an ordinary function anyway by sprinkling yields throughout it. If 
> you're going that far, is it really so bad to also change the
> "return" to a "raise ReturnValue"?

The big advantage of "return" over "raise" is that from the point of
view of the current execution frame, "return" is a *non-exceptional* exit.

This means that:
  1. "except" clauses do not execute, but neither do "else" clauses
  2. "with" statements pass the appropriate arguments to __exit__
methods to  indicate that the frame exit is non-exceptional

If you try to use "raise" instead of "return":
  1. All except clauses that match the raised exception will execute
  2. "with" statements will pass the raised exception into their
__exit__ methods

Consider a database access subgenerator that writes sent values to the
database inside a transaction and then returns the number of rows
processed (a *really* rough example just to illustrate the point):

  def process_rows(db, update_cmd):
    rows_processed = 0
    with db.transaction() as cursor:
      while 1:
        values = yield
        if not values:
          return rows_processed
        cursor.run(update_cmd, values)
        rows_processed += 1

That works just fine using return - the changes will be committed
correctly to the database, since this is a non-exceptional exit from the
transaction's point of view. If the result must be returned by raising
an exception instead, then you not only have to change the return
statement into a raise statement, but you have to make sure that it is
moved outside the with statement in order to avoid incorrectly rolling
back the database transaction.

That said, there a couple of fairly straightforward mechanisms I can
think of that allow return statements to be used naturally in
generators, while still picking up cases that are almost certainly errors.

If we don't want to modify existing features (e.g. for loops or
contextlib.contextmanager), then we could add a *peer* exception to
StopIteration called GeneratorReturn. Existing code which only catches
StopIteration would allow the new exception to escape, and the name of
that exception could then easily be Googled or looked up in the
documentation.

The only slight oddity is that a bare return in a generator would still
trigger StopIteration, while a "return None" would probably trigger
GeneratorReturn(None).

Another alternative that avoids the need for a new exception type is to
just modify the for loop handling code in the eval loop to check for a
non-None StopIteration.value and raise RuntimeError if that occurs.
(e.g. RuntimeError("Value set on StopIteration: iterator attempted to
return %r to for loop")). contextlib.contextmanager could easily do
something similar (it already raises RuntimeError whenever the
underlying generator doesn't behave as expected).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From cournape at gmail.com  Fri Mar 27 12:58:06 2009
From: cournape at gmail.com (David Cournapeau)
Date: Fri, 27 Mar 2009 20:58:06 +0900
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <878wmrtdb8.fsf@benfinney.id.au>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<79990c6b0903270322r13e11969ja64f732b5bc8698f@mail.gmail.com>
	<878wmrtdb8.fsf@benfinney.id.au>
Message-ID: <5b8d13220903270458g3646e8eandca0720db7ce0344@mail.gmail.com>

On Fri, Mar 27, 2009 at 8:28 PM, Ben Finney
<bignose+hates-spam at benfinney.id.au> wrote:

>
> I would argue that the Python community has a wealth of people quite
> capable of taking on this particular task, and if it makes the core
> architecture and maintenance of ?distutils? simpler to remove special
> cases for binary installers, I think that's a pearl of great price.

I think there are two points making binary installers pluggable, so
that they are independent of a core distutils, and including such
plugins in the stdlib. Nobody argues against the first case: that's
certainly a common complain that distutils is a big ball of code where
everything is intervened.

Concerning contribution for windows binaries: as the current numpy
developer in charge of windows binaries and windows support for a
while, my experience is that the windows situation for contribution is
very different from the other platforms. The mentality is just
different. At the risk of an overly broad and unfair generalization,
my experience is that on windows, people just want things to work,
complain when they do not, and almost never contribute back to make it
work, or when they do, they are almost never familiar with how things
work on other platforms, so they suggest broken fixes.

To say it differently: I mostly use Linux, the less time I am on
windows, the happier I am, but bdist_wininst is the only distutils
bdist_* command I care about.

cheers,

David

From ncoghlan at gmail.com  Fri Mar 27 13:26:37 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Mar 2009 22:26:37 +1000
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <94bdd2610903261252w6e4d4b92qe8f148ef82534061@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>	<87myb9w6sq.fsf@benfinney.id.au>	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>	<25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>	<gqglfq$und$2@ger.gmane.org>
	<94bdd2610903261252w6e4d4b92qe8f148ef82534061@mail.gmail.com>
Message-ID: <49CCC5FD.3040806@gmail.com>

Tarek Ziad? wrote:
> So if we, for once, forget about the central site-packages and define some
> kind of configuration process that is run when every script is launched
> to decide what packages should be loaded, we could seperate
>  "python the interpreter" from "python the pile of packages"
> 
> If some kind of API + configuration file could adress this need, maybe
> things could be simpler. You wouldn't fight against a central site-packages
> or a per-user site-packages. You would have to explicitely provide it

A rather simplistic version of that was actually the motivation behind
giving the interpreter the ability to execute directories and zipfiles
containing a __main__.py file (since the named directory/zipfile gets
added to sys.path and hence any modules or packages collocated with the
__main__.py file are automatically available to the application).

It doesn't deal with the library sharing or native installer questions,
but it does help with internal distribution of applications.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From hrvoje.niksic at avl.com  Fri Mar 27 13:31:41 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Fri, 27 Mar 2009 13:31:41 +0100
Subject: [Python-Dev] suggestion for try/except program flow
In-Reply-To: <15997572.3965780.1238152899083.JavaMail.xicrypt@atgrzls001>
References: <15997572.3965780.1238152899083.JavaMail.xicrypt@atgrzls001>
Message-ID: <49CCC72D.8070003@avl.com>

Mark Donald wrote:
> I frequently have this situation:
> 
> try:
>     try:
>         raise Thing
>     except Thing, e:
>         # handle Thing exceptions
>         raise
> except:
>     # handle all exceptions, including Thing

How about:

try:
     ... code that can raise Thing or another exception ...
except Exception, e:
     if isinstance(e, Thing):
         # handle thing
     # generic exception handling

From jnoller at gmail.com  Fri Mar 27 13:33:47 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Fri, 27 Mar 2009 07:33:47 -0500
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <loom.20090327T105353-433@post.gmane.org>
References: <49C48D03.7030709@gmail.com>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<49CC5D6D.7060108@canterbury.ac.nz>
	<loom.20090327T105353-433@post.gmane.org>
Message-ID: <4222a8490903270533k104a3a2cv78b66396c1796104@mail.gmail.com>

On Fri, Mar 27, 2009 at 5:56 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
>>
>> It's not really about doing away with trampolines anyway.
>> You still need at least one trampoline-like thing at the
>> top. What you do away with is the need for creating
>> special objects to yield, and the attendant syntactic
>> clumisiness and inefficiencies.
>
> No you don't, not in the Twisted case. The fact that useful library routines
> return Deferred objects to which you add callbacks and errbacks is probably not
> going away, because it's a fundamental building block in Twisted, not a
> convenience for scheduling generators.
>
> As a matter of fact, the people whom this PEP is supposed to benefit haven't
> expressed a lot of enthusiasm right now. That's why it looks so academic.
>
> Regards
>
> Antoine.

That's because most of us who might like this have been patently
avoiding this thread.

I like the syntax, I'm iffy on the exception other than stop iteration
(but I lost track on what was decided on this) and I would like to see
this go in. I think this is going to be a power user feature for
awhile, but I will like to see the libraries that will come of this, I
think this does enhance things.

Also, I know David Beazley did a tutorial here at pycon on
implementing coroutines and I'd be interested to see what he thinks of
this as well. I'll see if I can get his opinion.

-jesse

From ncoghlan at gmail.com  Fri Mar 27 13:41:15 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Mar 2009 22:41:15 +1000
Subject: [Python-Dev] suggestion for try/except program flow
In-Reply-To: <49CCC72D.8070003@avl.com>
References: <15997572.3965780.1238152899083.JavaMail.xicrypt@atgrzls001>
	<49CCC72D.8070003@avl.com>
Message-ID: <49CCC96B.9000700@gmail.com>

Hrvoje Niksic wrote:
> How about:
> 
> try:
>     ... code that can raise Thing or another exception ...
> except Exception, e:
>     if isinstance(e, Thing):
>         # handle thing
>     # generic exception handling

That is indeed the idiomatic way of handling the original poster's use
case without nesting a second try block.

Any proposals for new dedicated syntax should probably go to
python-ideas rather than here though. (It's a rare enough use case that
I don't like the original poster's chances of getting much support for
changes though)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From mal at egenix.com  Fri Mar 27 13:49:55 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 27 Mar 2009 13:49:55 +0100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
Message-ID: <49CCCB73.7060706@egenix.com>

On 2009-03-27 04:19, Guido van Rossum wrote:
> - keep distutils, but start deprecating certain higher-level
> functionality in it (e.g. bdist_rpm)
> - don't try to provide higher-level functionality in the stdlib, but
> instead let third party tools built on top of these core APIs compete

Should this be read as:

- remove bdist_rpm from the stdlib and let it live on PyPI

?

Perhaps I just misunderstand the comment.

I think that esp. the bdist_* commands help developers a lot by
removing the need to know how to build e.g. RPMs or Windows
installers and let distutils deal with it.

The bdist_* commands don't really provide any higher level
functionality. They only provide interfaces to certain packaging
formats commonly used on the various platforms.

Instead of removing such functionality, I think we should add
more support for standard packaging formats to distutils, e.g.
bdist_deb, bdist_pkg, etc.

And for eggs, there should be a standard bdist_egg, written against
the core distutils APIs (*), creating archives which other Python
package managers can then use in whatever way they seem fit.

Just please don't tie eggs to one specific package manager,
e.g. having to install setuptools just to run eggified packages
is just plain wrong. The format itself doesn't require this and
neither should the software shipped with those eggs.

(*) I've had a go at this a few months ago and then found out
that the egg format itself is not documented anywhere. As a result
you have to dig deep into setuptools to find out which files
are needed and where. That's something that needs to change
(Tarek is already working on a PEP for this, AFAIK).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 27 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2009-03-19: Released mxODBC.Connect 1.0.1      http://python.egenix.com/

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From olemis at gmail.com  Fri Mar 27 13:51:08 2009
From: olemis at gmail.com (Olemis Lang)
Date: Fri, 27 Mar 2009 07:51:08 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <094EF425-5929-48F6-8024-E61D28D5B0EC@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<3580D947-0674-41D4-BC90-A83C6541095D@python.org>
	<24ea26600903261307x1042a7bfn61d9ce8db02c16c7@mail.gmail.com>
	<094EF425-5929-48F6-8024-E61D28D5B0EC@python.org>
Message-ID: <24ea26600903270551w36295e9dld8c6eb2343261c14@mail.gmail.com>

On Thu, Mar 26, 2009 at 4:48 PM, Barry Warsaw <barry at python.org> wrote:
> On Mar 26, 2009, at 3:07 PM, Olemis Lang wrote:
>> On Thu, Mar 26, 2009 at 2:52 PM, Barry Warsaw <barry at python.org> wrote:
>>> On Mar 26, 2009, at 2:43 PM, Olemis Lang wrote:
>>>>>>>
>>>>>>> One thing that /would/ be helpful though is the ability to list all
>>>>>>> the
>>>>>>> resources under a specific package path. ?This is (I think) one use
>>>>>>> case
>>>>>>> that pkg_resource fails to support and it's the one place that I've
>>>>>>> had
>>>>>>> to
>>>>>>> drop down to file system introspection.
>>>>>>>
>>>>
>>>> Really ... ? What are all these for ?
>>>
>>> E.g. find all the doctests under foo.bar.docs
>>
>> Could you explain this a little more pls ? You mean ... using doctest
>> finders in a module (even inside a zip file ;) that's already loaded ?
>>
>> Sure ... else I dont understand ...
>
> Here's a concrete example. ?I have a bunch of .sql files inside the
> mailman.database package. ?Their file names encode a sort order. ?I want to
> load each one in order and apply them to my database. ?I'd like to be able
> to find all these .sql files (which are intermixed with .py files in the
> same package) without having to use os.listdir() because that forces the
> package to live on the file system.
>

Ok ... I will assume the following scenario :

- You have many .py together ... so you distribute them in a
mailman.database package (not module ... but it should be almost the
same thing ...) and provide setup.py for this ... therefore you should
have something like this in the later file

{{{
#!python

try:
    from setuptools import setup
except ImportError:
    from distutils.core import setup
# Many incredibly useful statements ...

setup(
        name='mailman.database',
        package_dir = {
                'mailman.database' : './db',
                },
        packages= ['mailman.database'],
        package_data={
                'mailman.database': ['sql/*', 'messages/es/LC_MESSAGES/*']
                },
        **many_incredibly_useful_args
        )
}}}

- Provided you have done this ... I will assume that u need to execute
such scripts at run-time, and not at install-time ... else setup.py is
more than enough ... ;)

{{{
#!python

from pkg_resources import *

for fnm in sorted(resource_listdir('mailman.database', 'sql'), \
                   my_own_cmp ): # Only if needed ... ;)
    exec_script(resource_string('mailman.database', 'sql/' + fnm))
}}}

... perhaps your specific example is a little bit more complex than
that ... but the procedure should be similar to the one ci-dessus ...
:)

ResourceManager instances handle all the details involved disregarding
whether the pkg is in a zip file, an egg file or FS ... ;)

CMIIW anyway ... ;)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From olemis at gmail.com  Fri Mar 27 13:55:16 2009
From: olemis at gmail.com (Olemis Lang)
Date: Fri, 27 Mar 2009 07:55:16 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <79990c6b0903261627w5fa157beh836fb26ec764c74e@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<24ea26600903261314i272e3365u2508c76ef4dc073e@mail.gmail.com>
	<3AED5E1E-D988-496B-B1D2-7C25FB489650@python.org>
	<79990c6b0903261627w5fa157beh836fb26ec764c74e@mail.gmail.com>
Message-ID: <24ea26600903270555k34c5c1ecg847661df0ef958b3@mail.gmail.com>

On Thu, Mar 26, 2009 at 6:27 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/3/26 Barry Warsaw <barry at python.org>:
>> Let me clarify my position: I just want the functionality (preferably in the
>> stdlib); I don't really care how it's spelled (except please not
>> pkg_resource.whatever() :).
>
> Agreed.

+1

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From cournape at gmail.com  Fri Mar 27 13:58:55 2009
From: cournape at gmail.com (David Cournapeau)
Date: Fri, 27 Mar 2009 21:58:55 +0900
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CCCB73.7060706@egenix.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<49CCCB73.7060706@egenix.com>
Message-ID: <5b8d13220903270558v686a0148u9bb84874dc0bc143@mail.gmail.com>

On Fri, Mar 27, 2009 at 9:49 PM, M.-A. Lemburg <mal at egenix.com> wrote:

> I think that esp. the bdist_* commands help developers a lot by
> removing the need to know how to build e.g. RPMs or Windows
> installers and let distutils deal with it.

I think it is a big dangerous to build rpm/deb without knowing how to
build them, because contrary to windows .exe, rpm/deb install things
system-wide, and you could easily break something. I don't think you
can build deb/rpm without knowing quite a lot about them.

> (*) I've had a go at this a few months ago and then found out
> that the egg format itself is not documented anywhere. As a result
> you have to dig deep into setuptools to find out which files
> are needed and where. That's something that needs to change
> (Tarek is already working on a PEP for this, AFAIK).

It is "documented" here:

http://peak.telecommunity.com/DevCenter/EggFormats

But as said in the preambule, people are not supposed to rely on this.
I for once would be really happy if  I could build eggs without
setuptools - for example to build eggs from scons, scripts, etc...

cheers,

David

From eric at trueblade.com  Fri Mar 27 14:02:50 2009
From: eric at trueblade.com (Eric Smith)
Date: Fri, 27 Mar 2009 08:02:50 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CCCB73.7060706@egenix.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	<20090326213047.6A8413A4097@sparrow.telecommunity.com>	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<49CCCB73.7060706@egenix.com>
Message-ID: <49CCCE7A.8090903@trueblade.com>

M.-A. Lemburg wrote:
> On 2009-03-27 04:19, Guido van Rossum wrote:
>> - keep distutils, but start deprecating certain higher-level
>> functionality in it (e.g. bdist_rpm)
>> - don't try to provide higher-level functionality in the stdlib, but
>> instead let third party tools built on top of these core APIs compete
> 
> Should this be read as:
> 
> - remove bdist_rpm from the stdlib and let it live on PyPI
> 
> ?

As one of the people who proposed this, I think it means: move 
bdist_rpm, bdist_msi, etc. out of distutils, but provide some of them 
with the standard Python installation. I'm certain that as part of the 
refactoring and simplification of distutils we'll gradually move the 
existing bdist_* commands into separate, stand-alone "things" (scripts, 
callable modules, or something). We'll need to do this if only for 
testing, so we may as well make them work.

> Instead of removing such functionality, I think we should add
> more support for standard packaging formats to distutils, e.g.
> bdist_deb, bdist_pkg, etc.

Agreed.

> And for eggs, there should be a standard bdist_egg, written against
> the core distutils APIs (*), creating archives which other Python
> package managers can then use in whatever way they seem fit.

Agreed.

> Just please don't tie eggs to one specific package manager,
> e.g. having to install setuptools just to run eggified packages
> is just plain wrong. The format itself doesn't require this and
> neither should the software shipped with those eggs.

I think that whatever we produce will need to be supported by the 
standalone version of the installer portion that will be backported and 
separately available.

From p.f.moore at gmail.com  Fri Mar 27 14:03:19 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 27 Mar 2009 13:03:19 +0000
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <4222a8490903270533k104a3a2cv78b66396c1796104@mail.gmail.com>
References: <49C48D03.7030709@gmail.com> <49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<49CC5D6D.7060108@canterbury.ac.nz>
	<loom.20090327T105353-433@post.gmane.org>
	<4222a8490903270533k104a3a2cv78b66396c1796104@mail.gmail.com>
Message-ID: <79990c6b0903270603s361763c7t5d88575bedad8090@mail.gmail.com>

2009/3/27 Jesse Noller <jnoller at gmail.com>:
> That's because most of us who might like this have been patently
> avoiding this thread.
>
> I like the syntax, I'm iffy on the exception other than stop iteration
> (but I lost track on what was decided on this) and I would like to see
> this go in. I think this is going to be a power user feature for
> awhile, but I will like to see the libraries that will come of this, I
> think this does enhance things.

Agreed on all counts.

Paul.

From olemis at gmail.com  Fri Mar 27 14:37:37 2009
From: olemis at gmail.com (Olemis Lang)
Date: Fri, 27 Mar 2009 08:37:37 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CCCB73.7060706@egenix.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<49CCCB73.7060706@egenix.com>
Message-ID: <24ea26600903270637y5df3b2ddna2f3b7e0bc6e6ce3@mail.gmail.com>

On Fri, Mar 27, 2009 at 7:49 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> On 2009-03-27 04:19, Guido van Rossum wrote:
>> - keep distutils, but start deprecating certain higher-level
>> functionality in it (e.g. bdist_rpm)
>> - don't try to provide higher-level functionality in the stdlib, but
>> instead let third party tools built on top of these core APIs compete
>
> Should this be read as:
>
> - remove bdist_rpm from the stdlib and let it live on PyPI
>
> ?
>
> Perhaps I just misunderstand the comment.
>
> I think that esp. the bdist_* commands help developers a lot by
> removing the need to know how to build e.g. RPMs or Windows
> installers and let distutils deal with it.
>
> The bdist_* commands don't really provide any higher level
> functionality. They only provide interfaces to certain packaging
> formats commonly used on the various platforms.
>
> Instead of removing such functionality, I think we should add
> more support for standard packaging formats to distutils, e.g.
> bdist_deb, bdist_pkg, etc.
>

+1 ... for this ...

> And for eggs, there should be a standard bdist_egg, written against
> the core distutils APIs (*), creating archives which other Python
> package managers can then use in whatever way they seem fit.
>

If not the eggs we have today ... the eggs we may incubate for tomorrow ... XD

> Just please don't tie eggs to one specific package manager,
> e.g. having to install setuptools just to run eggified packages
> is just plain wrong. The format itself doesn't require this and
> neither should the software shipped with those eggs.
>

... partly, because of this ... ;)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From olemis at gmail.com  Fri Mar 27 14:38:08 2009
From: olemis at gmail.com (Olemis Lang)
Date: Fri, 27 Mar 2009 08:38:08 -0500
Subject: [Python-Dev]  "setuptools has divided the Python community"
In-Reply-To: <24ea26600903270629o43227b7ai6bef06c96ddc6018@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<79990c6b0903270322r13e11969ja64f732b5bc8698f@mail.gmail.com>
	<24ea26600903270629o43227b7ai6bef06c96ddc6018@mail.gmail.com>
Message-ID: <24ea26600903270638h27bbacb5nd085db981dbd6bb6@mail.gmail.com>

On Fri, Mar 27, 2009 at 5:22 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/3/27 Guido van Rossum <guido at python.org>:
>> - keep distutils, but start deprecating certain higher-level
>> functionality in it (e.g. bdist_rpm)
>> - don't try to provide higher-level functionality in the stdlib, but
>> instead let third party tools built on top of these core APIs compete
>
> Please don't move bdist_wininst out of the core, though!
>
> I'd argue that Windows is a special case, as many Windows users don't
> have the ability to build their own extensions,

Hmmmm ... what about devs (me?) trying to build installers for
multiple platforms being in a single OS (let's say Ubuntu ...) ? IMO
in this case where policies are unique for a particular OS-flavour
(deb, rpms, and so on ...) ... there should be a single way to package
your app and to conform to the standards agreed by distros (e.g.
Debian) and maintainers ... isnt it enough std to be in stdlib ? I'd
really like to have those (... at least the most influential systems
... rpm, debs, and maybe two or three more that I'm missing here ...)

Indeed I'd like to know the arguments behind ?deprecating certain
higher-level functionality in it (e.g. bdist_rpm)? BTW ... perhaps
they'r self-explanatory and therefore I should not be asking this at
all ... :P

--
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From thomas at python.org  Fri Mar 27 14:38:27 2009
From: thomas at python.org (Thomas Wouters)
Date: Fri, 27 Mar 2009 14:38:27 +0100
Subject: [Python-Dev] GPython?
In-Reply-To: <79990c6b0903270350g2c2c78eeu5bd9e9f6b78a5bb4@mail.gmail.com>
References: <gqh8pm$vve$1@ger.gmane.org>
	<43aa6ff70903262040r41e99681o5c7abd33f744a114@mail.gmail.com>
	<79990c6b0903270350g2c2c78eeu5bd9e9f6b78a5bb4@mail.gmail.com>
Message-ID: <9e804ac0903270638m19de5038ya141cda4ff0ac6c0@mail.gmail.com>

On Fri, Mar 27, 2009 at 11:50, Paul Moore <p.f.moore at gmail.com> wrote:

> 2009/3/27 Collin Winter <collinw at gmail.com>:
> > In particular, Windows support is one of those things we'll need to
> > address on our end. LLVM's Windows support may be spotty, or there may
> > be other Windows issues we inadvertently introduce. None of the three
> > of us have Windows machines, nor do we particularly want to acquire
> > them :), and Windows support isn't going to be a big priority. If we
> > find that some of our patches have Windows issues, we will certainly
> > fix those before proposing their inclusion in CPython.
>
> On the assumption (sorry, I've done little more than read the press
> releases so far) that you're starting from the CPython base and
> incrementally patching things, you currently have strong Windows
> support. It would be a shame if that got gradually chipped away
> through neglect, until it became a big job to reinstate it.
>
> If the Unladen Swallow team doesn't include any Windows developers,
> you're a bit stuck, I guess, but could you not at least have a Windows
> buildbot which keeps tabs on the current status? Then you might
> encourage interested Windows bystanders to check in occasionally and
> maybe offer fixes.
>
> As things stand, the press releases give me the impression (as a
> Windows user without a lot of time to invest in contributing) that
> this project is irrelevant to me, and I should ignore it until you
> announce "proper" Windows support. By which time, it may have fallen
> completely off my radar. (On a smaller scale, this happened with
> virtualenv - I found to my surprise that it now supported Windows, and
> had for some time without me realising, because when it started it was
> Unix-only and I had not bothered to keep track of it). Maybe again
> it's something that could be clarified in the announcements.


It's not a matter of chipping away support. It's a matter of wishing to not
write our own JIT, but rather leverage other people's work. That currently
means LLVM, but LLVM has a weak Windows story at the moment. Of course, LLVM
has little Windows support because it doesn't have any Windows users :-) The
changes done so far are (mostly) orthogonal to Windows (the actual
performance benefits may depend a little on the platform), but the future
work will not be. On the other hand, getting the Windows story straightened
out is mostly a matter of getting Windows support in LLVM, and not work
specific to Unladen Swallow; hopefully, we can use some of the Windows
knowledge on python-dev or elsewhere in the world for that.

Notice how I said 'currently' and 'at the moment' and 'future work'. LLVM is
also a work in progress,  See, for instance,
http://llvm.org/docs/GettingStartedVS.html . (And if we were to write our
own JIT, we would have the same problem but worse: none of us would be able
to write an effective one for Windows, if at all, and we would have a much
smaller developer pool to work with. And it would take much longer in the
first place.)

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090327/bb75c91c/attachment-0001.htm>

From eric at trueblade.com  Fri Mar 27 14:47:32 2009
From: eric at trueblade.com (Eric Smith)
Date: Fri, 27 Mar 2009 08:47:32 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <24ea26600903270638h27bbacb5nd085db981dbd6bb6@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	<20090326213047.6A8413A4097@sparrow.telecommunity.com>	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>	<79990c6b0903270322r13e11969ja64f732b5bc8698f@mail.gmail.com>	<24ea26600903270629o43227b7ai6bef06c96ddc6018@mail.gmail.com>
	<24ea26600903270638h27bbacb5nd085db981dbd6bb6@mail.gmail.com>
Message-ID: <49CCD8F4.6010700@trueblade.com>

Olemis Lang wrote:
> On Fri, Mar 27, 2009 at 5:22 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>> 2009/3/27 Guido van Rossum <guido at python.org>:
>>> - keep distutils, but start deprecating certain higher-level
>>> functionality in it (e.g. bdist_rpm)
>>> - don't try to provide higher-level functionality in the stdlib, but
>>> instead let third party tools built on top of these core APIs compete
>> Please don't move bdist_wininst out of the core, though!
>>
>> I'd argue that Windows is a special case, as many Windows users don't
>> have the ability to build their own extensions,
> 
> Hmmmm ... what about devs (me?) trying to build installers for
> multiple platforms being in a single OS (let's say Ubuntu ...) ? IMO
> in this case where policies are unique for a particular OS-flavour
> (deb, rpms, and so on ...) ... there should be a single way to package
> your app and to conform to the standards agreed by distros (e.g.
> Debian) and maintainers ... isnt it enough std to be in stdlib ? I'd
> really like to have those (... at least the most influential systems
> ... rpm, debs, and maybe two or three more that I'm missing here ...)

The idea is to make the metadata extensible, so that for example Debian 
specific information can be put in the same config file that contains 
the "normal" metadata. I guess it would even be possible for this 
additional metadata to be in a separate config file. That way if someone 
downstream from me says "I can automatically build a .deb file if you'd 
just include this metadata", I could easily do that. I don't think it's 
reasonable that a package builder could possibly know all of the config 
information. To some extent, this is all currently possible with 
setup.cfg, and I do that myself.

I'm also not convinced there will be a single "bdist_rpm" (or whatever) 
replacement. It's entirely possible that different people would want to 
build different flavors of rpm's from the same metadata. Someone might 
be a real FHS devotee, and someone else might have practical reasons to 
not follow it.

> Indeed I'd like to know the arguments behind ?deprecating certain
> higher-level functionality in it (e.g. bdist_rpm)? BTW ... perhaps
> they'r self-explanatory and therefore I should not be asking this at
> all ... :P

I believe it's largely a refactoring. It's just too complicated and 
difficult to extend the way it is now.

Eric.


From ronaldoussoren at mac.com  Fri Mar 27 15:00:31 2009
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Fri, 27 Mar 2009 09:00:31 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CCCB73.7060706@egenix.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<49CCCB73.7060706@egenix.com>
Message-ID: <1976C9D1-AB74-41A1-AC5C-42CF153CBF9D@mac.com>


On 27 Mar, 2009, at 7:49, M.-A. Lemburg wrote:

> On 2009-03-27 04:19, Guido van Rossum wrote:
>> - keep distutils, but start deprecating certain higher-level
>> functionality in it (e.g. bdist_rpm)
>> - don't try to provide higher-level functionality in the stdlib, but
>> instead let third party tools built on top of these core APIs compete
>
> Should this be read as:
>
> - remove bdist_rpm from the stdlib and let it live on PyPI
>
> ?
>
> Perhaps I just misunderstand the comment.
>
> I think that esp. the bdist_* commands help developers a lot by
> removing the need to know how to build e.g. RPMs or Windows
> installers and let distutils deal with it.
>
> The bdist_* commands don't really provide any higher level
> functionality. They only provide interfaces to certain packaging
> formats commonly used on the various platforms.
>
> Instead of removing such functionality, I think we should add
> more support for standard packaging formats to distutils, e.g.
> bdist_deb, bdist_pkg, etc.

IIRC the reason for wanting to deprecate bdist_rpm (and not adding  
bdist_deb, ...) is that the variour Linux distributions have varying  
policies for how to package Python code and those policies tend to  
vary on another schedule than the Python development schedule. The  
result of this is that the Linux distributors are incapable to use  
bdist_rpm.  It would therefore be better to ensure that Python  
packages / distutils can provide the metadata that's needed to build  
packages and move the actual creation of OS installers outside of the  
core where the tool can be maintained by people that have detailed  
knowlegde about the needs of the packaging system and system policies.

Ronald


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090327/57869720/attachment.bin>

From skip at pobox.com  Fri Mar 27 15:04:34 2009
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 27 Mar 2009 09:04:34 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <gqi0ei$gma$1@ger.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>
	<49CAF030.80707@gmail.com>
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<20090327051935.12555.739678956.divmod.xquotient.6544@weber.divmod.com>
	<gqi0ei$gma$1@ger.gmane.org>
Message-ID: <18892.56562.447003.778067@montanaro.dyndns.org>


    Steve> Careful, Glyph. Nobody likes a smart-ass ;-)

I think he'll be ok.  He escaped the language summit with only minor wounds
yesterday. <wink>

Skip

From mal at egenix.com  Fri Mar 27 15:05:44 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 27 Mar 2009 15:05:44 +0100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <5b8d13220903270558v686a0148u9bb84874dc0bc143@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	<20090326213047.6A8413A4097@sparrow.telecommunity.com>	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>	<49CCCB73.7060706@egenix.com>
	<5b8d13220903270558v686a0148u9bb84874dc0bc143@mail.gmail.com>
Message-ID: <49CCDD38.9090409@egenix.com>

On 2009-03-27 13:58, David Cournapeau wrote:
> On Fri, Mar 27, 2009 at 9:49 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> 
>> I think that esp. the bdist_* commands help developers a lot by
>> removing the need to know how to build e.g. RPMs or Windows
>> installers and let distutils deal with it.
> 
> I think it is a big dangerous to build rpm/deb without knowing how to
> build them, because contrary to windows .exe, rpm/deb install things
> system-wide, and you could easily break something. I don't think you
> can build deb/rpm without knowing quite a lot about them.

Well, at least the bdist_rpm command did a pretty good job: RedHat
used the RPM spec generated by it to ship egenix-mx-base in their
distribution.

>> (*) I've had a go at this a few months ago and then found out
>> that the egg format itself is not documented anywhere. As a result
>> you have to dig deep into setuptools to find out which files
>> are needed and where. That's something that needs to change
>> (Tarek is already working on a PEP for this, AFAIK).
> 
> It is "documented" here:
> 
> http://peak.telecommunity.com/DevCenter/EggFormats

Thanks for the link. I must have missed that in my search for
a format spec.

> But as said in the preambule, people are not supposed to rely on this.
> I for once would be really happy if  I could build eggs without
> setuptools - for example to build eggs from scons, scripts, etc...

Right, the .egg format should be a formal PEP standard.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 27 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2009-03-19: Released mxODBC.Connect 1.0.1      http://python.egenix.com/

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From p.f.moore at gmail.com  Fri Mar 27 15:09:26 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 27 Mar 2009 14:09:26 +0000
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <5b8d13220903270458g3646e8eandca0720db7ce0344@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<79990c6b0903270322r13e11969ja64f732b5bc8698f@mail.gmail.com>
	<878wmrtdb8.fsf@benfinney.id.au>
	<5b8d13220903270458g3646e8eandca0720db7ce0344@mail.gmail.com>
Message-ID: <79990c6b0903270709k35b4912dhfadbb2d6a70ca749@mail.gmail.com>

2009/3/27 David Cournapeau <cournape at gmail.com>:
> Concerning contribution for windows binaries: as the current numpy
> developer in charge of windows binaries and windows support for a
> while, my experience is that the windows situation for contribution is
> very different from the other platforms. The mentality is just
> different. At the risk of an overly broad and unfair generalization,
> my experience is that on windows, people just want things to work,
> complain when they do not, and almost never contribute back to make it
> work, or when they do, they are almost never familiar with how things
> work on other platforms, so they suggest broken fixes.

As a Windows user, I'll actually agree with this :-)

As a bit of self-justification, I'll point out that it's *hard* to get
things (specifically, C extensions, in this context) to work on
Windows. The lack of a C compiler available "as standard" can be a
nuisance (although nowadays, MSVC and mingw are good free options, so
it's *only* a nuisance these days, rather than a showstopper). The
lack of any reasonable standard for external libraries (zlib, jpeg,
whatever) can be a huge issue [1]. So it's often a case of "give me a
working binary, or it's too hard for me to use this module and I'll
find another solution".

Not contributing back is probably a cultural thing. When the norm is
to pay a lot of money for software, you get used to the idea that you
expect not to have to fix the bugs in it yourself :-) I try hard not
to forget that this doesn't apply when I get something for free, but I
don't always remember...

Windows users don't understand Unix, sure. But Unix users don't
understand Windows either. And one person's "broken fix" is another's
"works for me". Would you feel better if I posted a fix and explicitly
said "this works for me, but I don't know anything about your system,
so it may need tweaking to apply generally"? The effect is identical,
it's just how I express it (and FWIW I've seen exactly this type of
scenario where it's a Linux-specific fix not working for Windows (or
Macs, for that matter)). The alternative is not accepting
contributions unless they are perfect, or expecting every contributor
to be willing to spend a lot of time learning the intricacies of
systems they'll never use, to improve their contribution in ways that
make no difference to them, and which they can't even test.

Anyway, this is straying well off-topic, into Linux vs Windows
territory, so I shall say no more!

Paul

[1] As a concrete example: My current PC is newly built, with Python
2.6 and MS Visual Studio 2008 Express and mingw installed. That's all
the development tools I presently have. How much time would I have to
spend to get to a point where I could build numpy for myself, from
this point? Based on previous experience, my instinct is that the
answer is "too much effort to justify it for the limited interest I
have in numpy"...)

From mal at egenix.com  Fri Mar 27 15:14:50 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 27 Mar 2009 15:14:50 +0100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <1976C9D1-AB74-41A1-AC5C-42CF153CBF9D@mac.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	<20090326213047.6A8413A4097@sparrow.telecommunity.com>	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>	<49CCCB73.7060706@egenix.com>
	<1976C9D1-AB74-41A1-AC5C-42CF153CBF9D@mac.com>
Message-ID: <49CCDF5A.3070905@egenix.com>

On 2009-03-27 15:00, Ronald Oussoren wrote:
> 
> On 27 Mar, 2009, at 7:49, M.-A. Lemburg wrote:
> 
>> On 2009-03-27 04:19, Guido van Rossum wrote:
>>> - keep distutils, but start deprecating certain higher-level
>>> functionality in it (e.g. bdist_rpm)
>>> - don't try to provide higher-level functionality in the stdlib, but
>>> instead let third party tools built on top of these core APIs compete
>>
>> Should this be read as:
>>
>> - remove bdist_rpm from the stdlib and let it live on PyPI
>>
>> ?
>>
>> Perhaps I just misunderstand the comment.
>>
>> I think that esp. the bdist_* commands help developers a lot by
>> removing the need to know how to build e.g. RPMs or Windows
>> installers and let distutils deal with it.
>>
>> The bdist_* commands don't really provide any higher level
>> functionality. They only provide interfaces to certain packaging
>> formats commonly used on the various platforms.
>>
>> Instead of removing such functionality, I think we should add
>> more support for standard packaging formats to distutils, e.g.
>> bdist_deb, bdist_pkg, etc.
> 
> IIRC the reason for wanting to deprecate bdist_rpm (and not adding
> bdist_deb, ...) is that the variour Linux distributions have varying
> policies for how to package Python code and those policies tend to vary
> on another schedule than the Python development schedule. The result of
> this is that the Linux distributors are incapable to use bdist_rpm. 

Hmm, I have heard different things - see my reply to Olemis :-)

Besides, the bdist_rpm command as well as the others are not only
for Linux distributors to use, but to help developers ship their
packages to their users *without* the help of some distribution,
ie. by providing an RPM or DEB file for download.

> It would therefore be better to ensure that Python packages / distutils can
> provide the metadata that's needed to build packages and move the actual
> creation of OS installers outside of the core where the tool can be
> maintained by people that have detailed knowlegde about the needs of the
> packaging system and system policies.

Fair enough, though, I'm sure we have enough developer knowledge
on board to maintain bdist_rpm and add a bdist_deb. Perhaps even
an bdist_dmg :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 27 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2009-03-19: Released mxODBC.Connect 1.0.1      http://python.egenix.com/

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From collinw at gmail.com  Fri Mar 27 15:38:00 2009
From: collinw at gmail.com (Collin Winter)
Date: Fri, 27 Mar 2009 09:38:00 -0500
Subject: [Python-Dev] GPython?
In-Reply-To: <79990c6b0903270350g2c2c78eeu5bd9e9f6b78a5bb4@mail.gmail.com>
References: <gqh8pm$vve$1@ger.gmane.org>
	<43aa6ff70903262040r41e99681o5c7abd33f744a114@mail.gmail.com>
	<79990c6b0903270350g2c2c78eeu5bd9e9f6b78a5bb4@mail.gmail.com>
Message-ID: <43aa6ff70903270738qa5cdd43r105fe7eb30a0c3b4@mail.gmail.com>

On Fri, Mar 27, 2009 at 5:50 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> 2009/3/27 Collin Winter <collinw at gmail.com>:
>> In particular, Windows support is one of those things we'll need to
>> address on our end. LLVM's Windows support may be spotty, or there may
>> be other Windows issues we inadvertently introduce. None of the three
>> of us have Windows machines, nor do we particularly want to acquire
>> them :), and Windows support isn't going to be a big priority. If we
>> find that some of our patches have Windows issues, we will certainly
>> fix those before proposing their inclusion in CPython.
>
> On the assumption (sorry, I've done little more than read the press
> releases so far) that you're starting from the CPython base and
> incrementally patching things, you currently have strong Windows
> support. It would be a shame if that got gradually chipped away
> through neglect, until it became a big job to reinstate it.

That's correct, we're starting with CPython 2.6.1.

> If the Unladen Swallow team doesn't include any Windows developers,
> you're a bit stuck, I guess, but could you not at least have a Windows
> buildbot which keeps tabs on the current status? Then you might
> encourage interested Windows bystanders to check in occasionally and
> maybe offer fixes.

We're definitely going to set up buildslaves for Windows and other
platforms (currently we're only running Linux buildslaves). We're
trying to solicit 20% time help from Google Windows developers, but
that experience is relatively rare compared to the vast sea of
Linux-focused engineers (though that's true of the open-source
community in general). Also, it may be that some of the components
we're reusing don't support Windows, or perhaps worse, offer degraded
performance on Windows. We believe we can fix these problems as they
come up -- we certainly don't want Windows issues to prevent patches
from going into mainline -- but it's still a risk that Windows issues
may slow down our development or prevent us from doing something fancy
down the road, and I wanted to be up front about that risk.

I've updated our ProjectPlan in hopes of clarifying this. That section
of the docs was copy/pasted off a slide, and was a bit too terse :)

Collin

From martin at v.loewis.de  Fri Mar 27 16:11:38 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 27 Mar 2009 10:11:38 -0500
Subject: [Python-Dev] version compare function into main lib
In-Reply-To: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>
References: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>
Message-ID: <49CCECAA.7010406@v.loewis.de>

> Correct me if I wrong, but shouldn't Python include function for
> version comparisons?

On the packaging summit yesterday, people agreed that yes, we should
have something like that in the standard library, and it should be more
powerful than what distutils currently offers.

There was no conclusion of how specifically that functionality should
be offered; several people agreed that Python should mandate a standard
format, which it is then able to compare. So you might not be able to
spell it "10.3.40-beta", but perhaps "10.3.40b1" or "10.3.40~beta".

Regards,
Martin


From hrvoje.niksic at avl.com  Fri Mar 27 16:19:10 2009
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Fri, 27 Mar 2009 16:19:10 +0100
Subject: [Python-Dev] suggestion for try/except program flow
In-Reply-To: <32245993.3978785.1238166283254.JavaMail.xicrypt@atgrzls001>
References: <15997572.3965780.1238152899083.JavaMail.xicrypt@atgrzls001>	
	<49CCC72D.8070003@avl.com>
	<32245993.3978785.1238166283254.JavaMail.xicrypt@atgrzls001>
Message-ID: <49CCEE6E.6090509@avl.com>

Mark Donald wrote:
> Thanks for the idea, but I believe it has a different outcome. You'd
> have to copy the generic handler into an except clause to get exactly
> the behaviour I'm looking for, which is worse than nesting the try
> blocks

Then simply change Exception to BaseException.  Since all exceptions 
should derive from BaseException, there should be no difference between 
that and a bare "except:" clause.

From mrts.pydev at gmail.com  Fri Mar 27 16:28:25 2009
From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=)
Date: Fri, 27 Mar 2009 17:28:25 +0200
Subject: [Python-Dev] version compare function into main lib
In-Reply-To: <49CCECAA.7010406@v.loewis.de>
References: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>
	<49CCECAA.7010406@v.loewis.de>
Message-ID: <ad1f81530903270828r16bc6aa2l59b32b329cc1f410@mail.gmail.com>

See http://wiki.python.org/moin/ApplicationInfrastructure , "Version
handling" below for a possible strict version API.

The page is relevant for the general packaging discussion as well, although
it's not fully fleshed out yet.

MS

On Fri, Mar 27, 2009 at 5:11 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> Correct me if I wrong, but shouldn't Python include function for
>> version comparisons?
>>
>
> On the packaging summit yesterday, people agreed that yes, we should
> have something like that in the standard library, and it should be more
> powerful than what distutils currently offers.
>
> There was no conclusion of how specifically that functionality should
> be offered; several people agreed that Python should mandate a standard
> format, which it is then able to compare. So you might not be able to
> spell it "10.3.40-beta", but perhaps "10.3.40b1" or "10.3.40~beta".
>
> Regards,
> Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090327/9dad4e89/attachment.htm>

From p.f.moore at gmail.com  Fri Mar 27 16:51:41 2009
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 27 Mar 2009 15:51:41 +0000
Subject: [Python-Dev] GPython?
In-Reply-To: <9e804ac0903270638m19de5038ya141cda4ff0ac6c0@mail.gmail.com>
References: <gqh8pm$vve$1@ger.gmane.org>
	<43aa6ff70903262040r41e99681o5c7abd33f744a114@mail.gmail.com>
	<79990c6b0903270350g2c2c78eeu5bd9e9f6b78a5bb4@mail.gmail.com>
	<9e804ac0903270638m19de5038ya141cda4ff0ac6c0@mail.gmail.com>
Message-ID: <79990c6b0903270851i3dc2c9adlc9448a253f9d0911@mail.gmail.com>

2009/3/27 Thomas Wouters <thomas at python.org>:
> It's not a matter of chipping away support. It's a matter of wishing to not
> write our own JIT, but rather leverage other people's work. That currently
> means LLVM, but LLVM has a weak Windows story at the moment.

Ah, I see. That's much more encouraging. On the other hand, I'm
definitely not in a position to contribute to LLVM, so I guess I'll
have to wait and see how things go...

Paul.

From solipsis at pitrou.net  Fri Mar 27 16:58:18 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 27 Mar 2009 15:58:18 +0000 (UTC)
Subject: [Python-Dev] "yield" in list comprehensions
Message-ID: <loom.20090327T155509-761@post.gmane.org>

Hello,

Just for the record, I thought I'd point out this slightly exotic and funny
incompatibility between 2.x and py3k. This is due to the fact that list
comprehensions now live in their own frame, so a "yield" expression inside a
list comprehension turns the comprehension itself into a generator (rather than
the enclosing function).

http://bugs.python.org/issue5577

Regards

Antoine.



From eric at trueblade.com  Fri Mar 27 17:01:32 2009
From: eric at trueblade.com (Eric Smith)
Date: Fri, 27 Mar 2009 11:01:32 -0500
Subject: [Python-Dev] version compare function into main lib
In-Reply-To: <49CCECAA.7010406@v.loewis.de>
References: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>
	<49CCECAA.7010406@v.loewis.de>
Message-ID: <49CCF85C.8040902@trueblade.com>

Martin v. L?wis wrote:
>> Correct me if I wrong, but shouldn't Python include function for
>> version comparisons?
> 
> On the packaging summit yesterday, people agreed that yes, we should
> have something like that in the standard library, and it should be more
> powerful than what distutils currently offers.

Yes.

> There was no conclusion of how specifically that functionality should
> be offered; several people agreed that Python should mandate a standard
> format, which it is then able to compare. So you might not be able to
> spell it "10.3.40-beta", but perhaps "10.3.40b1" or "10.3.40~beta".

I got the impression that people are generally happy with what 
setuptools provides for version parsing and comparison.

Does anyone think that's not a good model?

Eric.

From mal at egenix.com  Fri Mar 27 17:08:27 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 27 Mar 2009 17:08:27 +0100
Subject: [Python-Dev] version compare function into main lib
In-Reply-To: <49CCF85C.8040902@trueblade.com>
References: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>	<49CCECAA.7010406@v.loewis.de>
	<49CCF85C.8040902@trueblade.com>
Message-ID: <49CCF9FB.9080408@egenix.com>

On 2009-03-27 17:01, Eric Smith wrote:
> Martin v. L?wis wrote:
>>> Correct me if I wrong, but shouldn't Python include function for
>>> version comparisons?
>>
>> On the packaging summit yesterday, people agreed that yes, we should
>> have something like that in the standard library, and it should be more
>> powerful than what distutils currently offers.
> 
> Yes.
> 
>> There was no conclusion of how specifically that functionality should
>> be offered; several people agreed that Python should mandate a standard
>> format, which it is then able to compare. So you might not be able to
>> spell it "10.3.40-beta", but perhaps "10.3.40b1" or "10.3.40~beta".
> 
> I got the impression that people are generally happy with what
> setuptools provides for version parsing and comparison.
> 
> Does anyone think that's not a good model?

Instead of trying to parse some version string, distutils should
require defining the version as tuple with well-defined entries -
much like what we have in sys.version_info for Python.

The developer can then still use whatever string format s/he wants.

The version compare function would then work on this version tuple
and probably be called cmp() (at least in Python 2.x ;-).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 27 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2009-03-19: Released mxODBC.Connect 1.0.1      http://python.egenix.com/

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From pje at telecommunity.com  Fri Mar 27 17:19:39 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 27 Mar 2009 12:19:39 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CCCB73.7060706@egenix.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<49CCCB73.7060706@egenix.com>
Message-ID: <20090327161716.3A8883A40A7@sparrow.telecommunity.com>

At 01:49 PM 3/27/2009 +0100, M.-A. Lemburg wrote:
>(*) I've had a go at this a few months ago and then found out
>that the egg format itself is not documented anywhere.

It's been documented for just under three years now.  Here's where 
you quoted the email where I announced that documentation, provided 
links to it, and asked you to let me know if there's anything else 
you'd need in it:

http://mail.python.org/pipermail/python-dev/2006-April/064496.html


From mrts.pydev at gmail.com  Fri Mar 27 17:21:12 2009
From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=)
Date: Fri, 27 Mar 2009 18:21:12 +0200
Subject: [Python-Dev] version compare function into main lib
In-Reply-To: <49CCF9FB.9080408@egenix.com>
References: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>
	<49CCECAA.7010406@v.loewis.de> <49CCF85C.8040902@trueblade.com>
	<49CCF9FB.9080408@egenix.com>
Message-ID: <ad1f81530903270921l512d8ca2n7128789f6860fee5@mail.gmail.com>

> Instead of trying to parse some version string, distutils should
> require defining the version as tuple with well-defined entries -
> much like what we have in sys.version_info for Python.
>
> The developer can then still use whatever string format s/he wants.
>
> The version compare function would then work on this version tuple
> and probably be called cmp() (at least in Python 2.x ;-).
>


Except there need to be functions for parsing the tuple from a string and
preferably a canonical string representation to ease that parsing. Hence the
Version class in "Version Handling" referred to above.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090327/fc7fb634/attachment.htm>

From a.badger at gmail.com  Fri Mar 27 17:19:00 2009
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Fri, 27 Mar 2009 11:19:00 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	
	<loom.20090325T121432-793@post.gmane.org>	
	<d2155e360903250812j2d99a49cy71fb98495a19f49f@mail.gmail.com>	
	<1bc395c10903250837x41740648n65d993a27e1a7d30@mail.gmail.com>	
	<49CAF030.80707@gmail.com>	
	<5b8d13220903252119m6a68e7d7ue153aa4bf27bdd61@mail.gmail.com>	
	<94bdd2610903252140o667aed7euefd9d52fe6b6d29d@mail.gmail.com>	
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>	
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
Message-ID: <49CCFC74.3020302@gmail.com>

Guido van Rossum wrote:
> 2009/3/26 Toshio Kuratomi <a.badger at gmail.com>:
>> Guido van Rossum wrote:
>>> On Wed, Mar 25, 2009 at 9:40 PM, Tarek Ziad? <ziade.tarek at gmail.com> wrote:
>>>> I think Distutils (and therefore Setuptools) should provide some APIs
>>>> to play with special files (like resources) and to mark them as being special,
>>>> no matter where they end up in the target system.
>>>>
>>>> So the code inside the package can use these files seamessly no matter
>>>> what the system is
>>>> and no matter where the files have been placed by the packager.
>>>>
>>>> This has been discussed already but not clearly defined.
>>> Yes, this should be done. PEP 302 has some hooks but they are optional
>>> and not available for the default case. A simple wrapper to access a
>>> resource file relative to a given module or package would be easy to
>>> add. It should probably support four APIs:
>>>
>>> - Open as a binary stream
>>> - Open as a text stream
>>> - Get contents as a binary string
>>> - Get contents as a text string
>>>
>> Depending on the definition of a "resource" there's additional
>> information that could be needed.  For instance, if resource includes
>> message catalogs, then being able to get the base directory that the
>> catalogs reside in is needed for passing to gettext.
> 
> Well the whole point is that for certain loaders (e.g. zip files)
> there *is* no base directory. If you do need directories you won't be
> able to use PEP-302 loaders, and you can just use
> os.path.dirname(<some_module>.__file__).
> 
Yep.  Having no base directory isn't sufficient in all cases.

So one way to fix this is to define resources so that these cases fall
outside of that.

Current setuptools works around this by having API in pkg_resources that
unzips when it's necessary to use a filename rather than just retrieving
the data from the file.  So a second option is to have other API methods
  that allow this.

-Toshio

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090327/d338c5c1/attachment-0001.pgp>

From eric at trueblade.com  Fri Mar 27 17:30:35 2009
From: eric at trueblade.com (Eric Smith)
Date: Fri, 27 Mar 2009 11:30:35 -0500
Subject: [Python-Dev] version compare function into main lib
In-Reply-To: <ad1f81530903270921l512d8ca2n7128789f6860fee5@mail.gmail.com>
References: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>	<49CCECAA.7010406@v.loewis.de>
	<49CCF85C.8040902@trueblade.com>	<49CCF9FB.9080408@egenix.com>
	<ad1f81530903270921l512d8ca2n7128789f6860fee5@mail.gmail.com>
Message-ID: <49CCFF2B.50501@trueblade.com>

Mart S?mermaa wrote:
> 
>     Instead of trying to parse some version string, distutils should
>     require defining the version as tuple with well-defined entries -
>     much like what we have in sys.version_info for Python.
> 
>     The developer can then still use whatever string format s/he wants.
> 
>     The version compare function would then work on this version tuple
>     and probably be called cmp() (at least in Python 2.x ;-).
> 
> 
> 
> Except there need to be functions for parsing the tuple from a string 
> and preferably a canonical string representation to ease that parsing. 
> Hence the Version class in "Version Handling" referred to above.

Right. For example, say you need to specify in a config file that your 
package requires version 1.3.4 of some other tool. I think the only 
rational way to do this is in a string, and be able to parse the version 
number (possibly into a tuple) and compare it with other version 
numbers. I don't think you want to directly specify the tuple in such a 
case.

Eric.


From pje at telecommunity.com  Fri Mar 27 17:39:41 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 27 Mar 2009 12:39:41 -0400
Subject: [Python-Dev] version compare function into main lib
In-Reply-To: <49CCF9FB.9080408@egenix.com>
References: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>
	<49CCECAA.7010406@v.loewis.de> <49CCF85C.8040902@trueblade.com>
	<49CCF9FB.9080408@egenix.com>
Message-ID: <20090327163718.8D21E3A40AF@sparrow.telecommunity.com>

At 05:08 PM 3/27/2009 +0100, M.-A. Lemburg wrote:
>On 2009-03-27 17:01, Eric Smith wrote:
> > Martin v. L?wis wrote:
> >>> Correct me if I wrong, but shouldn't Python include function for
> >>> version comparisons?
> >>
> >> On the packaging summit yesterday, people agreed that yes, we should
> >> have something like that in the standard library, and it should be more
> >> powerful than what distutils currently offers.
> >
> > Yes.
> >
> >> There was no conclusion of how specifically that functionality should
> >> be offered; several people agreed that Python should mandate a standard
> >> format, which it is then able to compare. So you might not be able to
> >> spell it "10.3.40-beta", but perhaps "10.3.40b1" or "10.3.40~beta".
> >
> > I got the impression that people are generally happy with what
> > setuptools provides for version parsing and comparison.
> >
> > Does anyone think that's not a good model?
>
>Instead of trying to parse some version string, distutils should
>require defining the version as tuple with well-defined entries -
>much like what we have in sys.version_info for Python.
>
>The developer can then still use whatever string format s/he wants.
>
>The version compare function would then work on this version tuple
>and probably be called cmp() (at least in Python 2.x ;-).

By the way, pkg_resources.parse_version of course returns a tuple 
that can be compared with cmp().


From pje at telecommunity.com  Fri Mar 27 18:04:18 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 27 Mar 2009 13:04:18 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CC8037.1010909@scottdial.com>
References: <49C48D03.7030709@gmail.com>
	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<20090327041138.EEF893A4097@sparrow.telecommunity.com>
	<49CC8037.1010909@scottdial.com>
Message-ID: <20090327170154.AC1753A40A7@sparrow.telecommunity.com>

At 03:28 AM 3/27/2009 -0400, Scott Dial wrote:
>P.J. Eby wrote:
> > One remaining quirk or missing piece: ISTM there needs to be a way to
> > extract the return value without using a yield-from statement.  I mean,
> > you could write a utility function like:
> >
> >    def unyield(geniter):
> >        try:
> >            while 1: geniter.next()
> >        except GeneratorReturn as v:
> >            return v.value
>
>My first thought was to ask why it was not equivalent to say:
>
>     x = yield g
>     x = yield from g
>
>This would seem like a more obvious lack of parallelism to pick on wrt.
>return values.

Because yield-from means you're "inlining" the generator, such that 
sends go into that generator, rather than into the current generator.


>This unyield() operation seems contrived. Never before have you been
>able to write a generator that returns a value, why would these suddenly
>become common practice? The only place a return value seems useful is
>when refactoring a generator and you need to mend having loss of a
>shared scope. What other use is there for a return value?

The use case which these things are being proposed for is to replace 
most of the stack-management code that's currently needed for 
coroutine trampolines.  In such a case, you're likely using 
generators to perform long-running asynchronous operations, or else 
coroutines where two functions are co-operating to produce a result, 
each with its own control flow.

For example, you might have a generator that yields socket objects to 
wait for them to be ready to read or write, then returns a line of 
text read from the socket.  You would unyield this if you wanted to 
write top-level code that was *not* also such a task.  Similarly, you 
might write coroutines where one reads data from a file and sends it 
to a parser, and then the parser sends data back to a main program.

In either case, an unyield would either be the synchronous top-level 
loop of the program, or part of the top-level code.  Either you need 
to get the finished top-level object from your parser at the end of 
its operation, or you are waiting for all your asynchronous I/O tasks 
to complete.


>It would seem unfortunate for it to be considered a runtime error since
>this would prevent sharing a generator amongst "yield from" and
>non-"yield from" use cases.

Has anyone shown a use case for doing so?  I might be biased due to 
previous experience with these things, but I don't see how you write 
a function where both the yielded values *and* the return value are 
useful...  and if you did, you'd still need some sort of unyield operation.

Notice that in both the I/O and coroutine use cases, the point of 
yielding is primarily *to allow other code to execute*, and possibly 
pass a value back IN to the generator.  The values passed *out* by 
the generator are usually either ignored, an indicator of what the 
generator wants to be passed back in, or what sort of event it is 
waiting for before it's to be resumed.

In other words, they're usually not data -- they're just something 
that gets looped over as the task progresses.


>As Greg has said a number of times, we allow functions to return values
>with them silently being ignored all the time.

Sure.  But right now, the return value of a generator function *is 
the generator*.  And you're free to ignore that, sure.

But this is a "second" return value that only goes to a special place 
with special syntax -- without that syntax, you can't access it.

But in the use cases where you'd actually want to make such a 
function return a value to begin with, it's because that value is the 
value you *really* want from the function -- the only reason it's a 
generator is because it needs to be paused and resumed along the way 
to getting that return value.

If you're writing a function that yields values for other than 
control flow reasons, it's probably a bad idea for it to also have a 
"return" value....  because then you'd need an unyield operation to 
get at the data.

And it seems to me that people are saying, "but that's no problem, 
I'll just use yield-from to get the value".  But that doesn't *work*, 
because it turns the function where you use it into another generator!

The generators have to *stop* somewhere, in order for you to *use* 
their return values -- which makes the return feature ONLY relevant 
to co-routine use cases -- i.e., places where you have trampolines or 
a top-level loop to handle the yields...

And conversely, if you *have* such a generator, its real return value 
is the special return value, so you're not going to be able to use it 
outside the coroutine structure...  so "ignoring its return value" 
doesn't make any sense.  You'd have to write a loop over the 
generator, *just to ignore the value*...  which once again is why 
you'd want an unyield operator of some kind.

That's why special return values should be special: you have to 
handle them differently in order to receive that return value... and 
it's monumentally confusing to look at a function with a normal 
'return' that never actually "returns" that value.

A lot of the emails that have been written about this are failing to 
understand the effects of the control-flow proposed by the 
PEP.   IMO, this should be taken as evidence that using a plain 
"return" statement is in fact confusing, *even to Python-Dev 
participants who have read the PEP*.

We would be much better off with something like "yield return X" or 
"return from yield with X", as it would highlight this 
otherwise-obscure and "magical" difference in control flow.


From pje at telecommunity.com  Fri Mar 27 17:07:58 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 27 Mar 2009 12:07:58 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CC57FC.30907@trueblade.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<49CC57FC.30907@trueblade.com>
Message-ID: <20090327170202.4E5D03A40AF@sparrow.telecommunity.com>

At 11:37 PM 3/26/2009 -0500, Eric Smith wrote:
>P.J. Eby wrote:
> > As someone else suggested, moving some of the functionality to PEP 302
> > interfaces would also help.  Most of the code, though, deals with
> > locating/inspecting installed distributions, resolving version
> > requirements, and managing sys.path.  And most of the nastiest
> > complexity comes from trying to support true filename access to
> > resources -- if that were dropped from the stdlib, there'd be no need
> > for egg caches and the like, along with all the complexity entailed.
> >
> > Application environments such as Chandler, Trac, Zope, etc. that want
> > their plugins to live in .egg files wouldn't necessarily be able to use
> > such an API, but the independent pkg_resources wouldn't be
> > disappearing.  (Of course, they could also implement
> > application-specific file extraction, if the stdlib API included the
> > ability to inspect and open zipped resources.)
>
>Could you comment on why they couldn't use such an API?

If a plugin includes C code (.so/.dll), or uses a library that 
operates on filenames rather than bytes in memory (e.g. gettext), 
then the resources would need to be extracted from the 
.egg.  pkg_resources transparently extracts such resources to a cache 
directory when you ask for a resource's filename, rather than asking 
for a stream or string of its contents.

This feature represents a significant chunk of the complexity and 
code size of pkg_resources -- and I was proposing ways to cut down on 
that complexity and code size, for a (limited) stdlib version of the 
functionality.


From Scott.Daniels at Acm.Org  Fri Mar 27 18:09:15 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Fri, 27 Mar 2009 10:09:15 -0700
Subject: [Python-Dev] GPython?
In-Reply-To: <49CCB309.80404@gmail.com>
References: <gqh8pm$vve$1@ger.gmane.org>	<43aa6ff70903262040r41e99681o5c7abd33f744a114@mail.gmail.com>
	<49CCB309.80404@gmail.com>
Message-ID: <gqj112$v52$1@ger.gmane.org>

Nick Coghlan wrote:
> Collin Winter wrote:
>> That would be a bikeshed discussion of such
>> magnitude, you'd have to invent new colors to paint the thing.
> 
> Octarine. Definitely octarine :)

I'm not so sure of the color itself, but its name should definitely
rhyme with "orange."

--Scott David Daniels
Scott.Daniels at Acm.Org


From status at bugs.python.org  Fri Mar 27 18:07:06 2009
From: status at bugs.python.org (Python tracker)
Date: Fri, 27 Mar 2009 18:07:06 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20090327170706.A31A67809D@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (03/20/09 - 03/27/09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2395 open (+33) / 15008 closed (+21) / 17403 total (+54)

Open issues with patches:   845

Average duration of open issues: 661 days.
Median duration of open issues: 400 days.

Open Issues Breakdown
   open  2357 (+33)
pending    38 ( +0)

Issues Created Or Reopened (55)
_______________________________

float compared to decimal is silently incorrect.                 03/21/09
       http://bugs.python.org/issue2531    reopened marketdickinson               
       patch, easy                                                             

multiprocessing won't work with Tkinter (under Linux)            03/20/09
       http://bugs.python.org/issue5527    created  akineko                       
                                                                               

Unable to launch IDLE on Windows                                 03/20/09
CLOSED http://bugs.python.org/issue5528    created  croy                          
                                                                               

Backport sys module docs involving import to 2.7                 03/21/09
       http://bugs.python.org/issue5529    created  brett.cannon                  
                                                                               

datetime class names should obey PEP 8 CapWords convention       03/21/09
CLOSED http://bugs.python.org/issue5530    created  jess.austin                   
       patch                                                                   

unittest: allow failures in tearDown method                      03/21/09
CLOSED http://bugs.python.org/issue5531    created  gfxmonk                       
       patch                                                                   

imap usage in itertools unique_justseen recipe                   03/21/09
CLOSED http://bugs.python.org/issue5532    created  mnewman                       
                                                                               

unittest can't deal with packages                                03/21/09
       http://bugs.python.org/issue5533    created  Indy                          
                                                                               

Decimal __format__ reverses meaning of '<' and '>' alignment spe 03/22/09
CLOSED http://bugs.python.org/issue5534    created  donodonnell                   
                                                                               

json custom encoder not fully functionnal                        03/22/09
CLOSED http://bugs.python.org/issue5535    created  phanser                       
                                                                               

urllib: urlretrieve() does not close file objects on failure     03/22/09
CLOSED http://bugs.python.org/issue5536    created  petr.dolezal                  
                                                                               

LWPCookieJar cannot handle cookies with expirations of 2038 or g 03/22/09
       http://bugs.python.org/issue5537    created  hollec                        
                                                                               

tearDown in unittest should be executed regardless of result in  03/22/09
CLOSED http://bugs.python.org/issue5538    created  yaneurabeya                   
       needs review                                                            

open documentation unclear                                       03/23/09
CLOSED http://bugs.python.org/issue5539    created  LambertDW                     
                                                                               

"file objects" in python 3 tutorial                              03/23/09
       http://bugs.python.org/issue5540    created  LambertDW                     
                                                                               

File's current position inconsistent with 'a+' mode              03/23/09
CLOSED http://bugs.python.org/issue5541    created  ebfe                          
                                                                               

Socket is closed prematurely in httplib, if server sends respons 03/23/09
       http://bugs.python.org/issue5542    created  jasondavies                   
                                                                               

sys.last_type missing                                            03/23/09
CLOSED http://bugs.python.org/issue5543    created  brtzsnr                       
                                                                               

test_fileio fails on windows MSVC Assertion                      03/23/09
       http://bugs.python.org/issue5544    created  andreas.schawo                
                                                                               

multiprocessing: switch to autoconf detection of platform values 03/23/09
       http://bugs.python.org/issue5545    created  jnoller                       
       patch                                                                   

PyDict_SetItemString mentions PyString_FromString which does not 03/23/09
CLOSED http://bugs.python.org/issue5546    created  stutzbach                     
                                                                               

The Py_InitModule functions no longer exist, but remain in the d 03/23/09
CLOSED http://bugs.python.org/issue5547    created  stutzbach                     
                                                                               

In the tutorial, PyMODINIT_FUNC is shown as having a return type 03/23/09
       http://bugs.python.org/issue5548    created  stutzbach                     
                                                                               

PyModule_Create and PyModuleDef are undocumented                 03/23/09
CLOSED http://bugs.python.org/issue5549    created  stutzbach                     
                                                                               

urllib2 use of opener.addheaders                                 03/24/09
       http://bugs.python.org/issue5550    created  cocobear                      
                                                                               

os.path.ismount takes a cross-device symlink for a mountpoint    03/24/09
       http://bugs.python.org/issue5551    created  jae                           
                                                                               

With invalid FD, os.device_encoding() returns None under Linux b 03/24/09
CLOSED http://bugs.python.org/issue5552    created  pitrou                        
                                                                               

Py_LOCAL_INLINE(type) doesn't actually inline except using MSC   03/24/09
       http://bugs.python.org/issue5553    created  stutzbach                     
                                                                               

file.read() doesn't read the whole file                          03/24/09
CLOSED http://bugs.python.org/issue5554    created  Ultrasick                     
                                                                               

optparse                                                         03/24/09
       http://bugs.python.org/issue5555    created  ajs                           
                                                                               

interactive interpreter, source encoding                         03/24/09
       http://bugs.python.org/issue5556    created  jmfauth                       
                                                                               

Byte-code compilation uses excessive memory                      03/24/09
       http://bugs.python.org/issue5557    created  goddard                       
                                                                               

Python 3.0.1 Mac OS X install image ReadMe file is incorrect     03/24/09
       http://bugs.python.org/issue5558    created  xdcdx                         
                                                                               

IDLE Output Window 's goto fails when path has spaces            03/25/09
       http://bugs.python.org/issue5559    created  ccanepa                       
                                                                               

help() no longer reports module docstrings                       03/25/09
CLOSED http://bugs.python.org/issue5560    created  debatem1                      
                                                                               

platform.python_version_tuple returns tuple of ints, should be s 03/25/09
CLOSED http://bugs.python.org/issue5561    created  Larry Hastings                
                                                                               

Locale-based date formatting crashes on non-ASCII data           03/25/09
       http://bugs.python.org/issue5562    created  pitrou                        
                                                                               

Document bdist_msi                                               03/25/09
       http://bugs.python.org/issue5563    created  bethard                       
                                                                               

os.symlink/os.link docs should say old/new, not src/dst          03/26/09
       http://bugs.python.org/issue5564    created  Rhamphoryncus                 
                                                                               

Strange behavior when I logout() with IMAP4_SSL                  03/26/09
       http://bugs.python.org/issue5565    created  toelke                        
                                                                               

Minor error in document of PyLong_AsSsize_t                      03/26/09
       http://bugs.python.org/issue5566    created  bhy                           
                                                                               

Operators in operator module don't work with keyword arguments   03/26/09
       http://bugs.python.org/issue5567    created  leonidas                      
                                                                               

self.writer.closed() extraneous parens in BufferedRWPair of io m 03/26/09
CLOSED http://bugs.python.org/issue5568    created  jimo555                       
                                                                               

Issue in transparency in top level tk window(python) on MAC      03/26/09
CLOSED http://bugs.python.org/issue5569    created  YMohan                        
                                                                               

Bus error when calling .poll() on a closed Connection from multi 03/26/09
       http://bugs.python.org/issue5570    created  amacleod                      
                                                                               

new "TestCase.skip" method causes all tests to skip under trial  03/26/09
CLOSED http://bugs.python.org/issue5571    created  glyph                         
                                                                               

distutils ignores the LIBS configure env var                     03/26/09
       http://bugs.python.org/issue5572    created  collinwinter                  
       patch, patch, needs review                                              

multiprocessing Pipe poll() and recv() semantics.                03/26/09
       http://bugs.python.org/issue5573    created  mallyvai                      
                                                                               

multiprocessing queues.py doesn't include JoinableQueue in its _ 03/26/09
       http://bugs.python.org/issue5574    created  mallyvai                      
                                                                               

Add env vars for controlling building sqlite, hashlib and ssl    03/26/09
       http://bugs.python.org/issue5575    created  collinwinter                  
       patch, patch, needs review                                              

Don't use PyLong_SHIFT with _PyLong_AsScaledDouble()             03/27/09
       http://bugs.python.org/issue5576    created  haypo                         
       patch                                                                   

yield in iterators                                               03/27/09
       http://bugs.python.org/issue5577    created  qwjqwj                        
                                                                               

unqualified exec in class body                                   03/27/09
       http://bugs.python.org/issue5578    created  fijal                         
       patch                                                                   

Display for OrderedDict                                          03/27/09
       http://bugs.python.org/issue5579    created  tweiler                       
                                                                               

Strange error message in Python/getargs.c                        03/27/09
       http://bugs.python.org/issue5580    created  mishok13                      
                                                                               



Issues Now Closed (60)
______________________

asyncore loop lacks timers and work tasks                         415 days
       http://bugs.python.org/issue2006    intgr                         
                                                                               

Traceback changed in 2.6 for unhashable objects                   364 days
       http://bugs.python.org/issue2485    benjamin.peterson             
                                                                               

Hang when calling get() on an empty queue in the queue module     281 days
       http://bugs.python.org/issue3138    tazle                         
                                                                               

improve linecache: reuse tokenize.detect_encoding() and io.open(  103 days
       http://bugs.python.org/issue4016    benjamin.peterson             
       patch                                                                   

float(0.0) singleton                                              169 days
       http://bugs.python.org/issue4024    tjreedy                       
       patch                                                                   

Make multiprocessing compatible with Python 2.4 and 2.5           149 days
       http://bugs.python.org/issue4208    haypo                         
       patch, patch                                                            

Classify language vs. impl-detail tests, step 1                   147 days
       http://bugs.python.org/issue4242    benjamin.peterson             
       patch, patch                                                            

Segfault in stringobject.c                                        125 days
       http://bugs.python.org/issue4358    haypo                         
                                                                               

compile() doesn't ignore the source encoding when a string is pa  104 days
       http://bugs.python.org/issue4626    haypo                         
       patch, needs review                                                     

GC optimization: don't track simple tuples and dicts               96 days
       http://bugs.python.org/issue4688    pitrou                        
       patch                                                                   

PyFile_FromFd() doesn't set the file name                          86 days
       http://bugs.python.org/issue4762    haypo                         
                                                                               

difflib                                                            77 days
       http://bugs.python.org/issue4889    jackdied                      
                                                                               

Windows installer Quad processor issues                            69 days
       http://bugs.python.org/issue4911    loewis                        
                                                                               

tarfile loops forever on broken input                              55 days
       http://bugs.python.org/issue5068    lars.gustaebel                
                                                                               

import tkinter library Visual C++ Concepts:C Run-Time Error R603   58 days
       http://bugs.python.org/issue5090    amaury.forgeotdarc            
                                                                               

Printing Unicode chars from the interpreter in a non-UTF8 termin   55 days
       http://bugs.python.org/issue5110    haypo                         
       patch                                                                   

Space character returns false from isprintable() method            53 days
       http://bugs.python.org/issue5126    benjamin.peterson             
       patch                                                                   

struct.calcsize('cd') returns 16 instead of 9                      51 days
       http://bugs.python.org/issue5145    haypo                         
                                                                               

Add KOI8-RU as a known encoding                                    44 days
       http://bugs.python.org/issue5214    lemburg                       
                                                                               

json needs object_pairs_hook                                       23 days
       http://bugs.python.org/issue5381    cheeaun                       
       patch                                                                   

urllib.response.addinfourl does not support __exit__               22 days
       http://bugs.python.org/issue5418    jhylton                       
                                                                               

test_httpservers on Debian Testing                                 16 days
       http://bugs.python.org/issue5435    facundobatista                
                                                                               

Streamline integer division                                         5 days
       http://bugs.python.org/issue5512    haypo                         
       patch                                                                   

"What's New" should say VERY CLEARLY that the type file	is gone     4 days
       http://bugs.python.org/issue5513    benjamin.peterson             
                                                                               

cPickle produces inconsistent output                                1 days
       http://bugs.python.org/issue5518    loewis                        
                                                                               

refactor test_datetime.py                                           5 days
       http://bugs.python.org/issue5520    jess.austin                   
       patch                                                                   

Unable to launch IDLE on Windows                                    6 days
       http://bugs.python.org/issue5528    amaury.forgeotdarc            
                                                                               

datetime class names should obey PEP 8 CapWords convention          4 days
       http://bugs.python.org/issue5530    gvanrossum                    
       patch                                                                   

unittest: allow failures in tearDown method                         1 days
       http://bugs.python.org/issue5531    brett.cannon                  
       patch                                                                   

imap usage in itertools unique_justseen recipe                      1 days
       http://bugs.python.org/issue5532    benjamin.peterson             
                                                                               

Decimal __format__ reverses meaning of '<' and '>' alignment spe    0 days
       http://bugs.python.org/issue5534    marketdickinson               
                                                                               

json custom encoder not fully functionnal                           1 days
       http://bugs.python.org/issue5535    bob.ippolito                  
                                                                               

urllib: urlretrieve() does not close file objects on failure        0 days
       http://bugs.python.org/issue5536    benjamin.peterson             
                                                                               

tearDown in unittest should be executed regardless of result in     1 days
       http://bugs.python.org/issue5538    yaneurabeya                   
       needs review                                                            

open documentation unclear                                          0 days
       http://bugs.python.org/issue5539    benjamin.peterson             
                                                                               

File's current position inconsistent with 'a+' mode                 1 days
       http://bugs.python.org/issue5541    loewis                        
                                                                               

sys.last_type missing                                               0 days
       http://bugs.python.org/issue5543    benjamin.peterson             
                                                                               

PyDict_SetItemString mentions PyString_FromString which does not    0 days
       http://bugs.python.org/issue5546    benjamin.peterson             
                                                                               

The Py_InitModule functions no longer exist, but remain in the d    0 days
       http://bugs.python.org/issue5547    benjamin.peterson             
                                                                               

PyModule_Create and PyModuleDef are undocumented                    0 days
       http://bugs.python.org/issue5549    stutzbach                     
                                                                               

With invalid FD, os.device_encoding() returns None under Linux b    0 days
       http://bugs.python.org/issue5552    pitrou                        
                                                                               

file.read() doesn't read the whole file                             0 days
       http://bugs.python.org/issue5554    Ultrasick                     
                                                                               

help() no longer reports module docstrings                          1 days
       http://bugs.python.org/issue5560    amaury.forgeotdarc            
                                                                               

platform.python_version_tuple returns tuple of ints, should be s    0 days
       http://bugs.python.org/issue5561    lemburg                       
                                                                               

self.writer.closed() extraneous parens in BufferedRWPair of io m    1 days
       http://bugs.python.org/issue5568    amaury.forgeotdarc            
                                                                               

Issue in transparency in top level tk window(python) on MAC         1 days
       http://bugs.python.org/issue5569    amaury.forgeotdarc            
                                                                               

new "TestCase.skip" method causes all tests to skip under trial     0 days
       http://bugs.python.org/issue5571    benjamin.peterson             
                                                                               

unittest.py patch: add skipped test functionality                1641 days
       http://bugs.python.org/issue1034053 benjamin.peterson             
       patch                                                                   

os.times() is bogus                                              1628 days
       http://bugs.python.org/issue1040026 loewis                        
       patch                                                                   

csv Sniffer returns bad dialect?                                 1481 days
       http://bugs.python.org/issue1157169 skip.montanaro                
                                                                               

Call by object reference sometimes call by value                 1278 days
       http://bugs.python.org/issue1296434 benjamin.peterson             
                                                                               

set.__getstate__ is not overriden                                1254 days
       http://bugs.python.org/issue1326448 rhettinger                    
                                                                               

shutils cannot copy owner                                        1209 days
       http://bugs.python.org/issue1368091 ajaksu2                       
                                                                               

email.Utils.parseaddr() gives arcane result                      1157 days
       http://bugs.python.org/issue1409460 ajaksu2                       
                                                                               

Popened file object close hangs in latest Cygwin update          1155 days
       http://bugs.python.org/issue1413379 sferic                        
                                                                               

CSV Sniffer fails to report mismatch of column counts            1135 days
       http://bugs.python.org/issue1431091 skip.montanaro                
                                                                               

sys.getrefcount should be in gc                                  1031 days
       http://bugs.python.org/issue1494595 benjamin.peterson             
                                                                               

C API to retain GIL during Python Callback                       1031 days
       http://bugs.python.org/issue1497532 benjamin.peterson             
                                                                               

random.choice(setinstance) fails                                  936 days
       http://bugs.python.org/issue1551113 tim_one                       
                                                                               

unittest documentation is incomplete                              740 days
       http://bugs.python.org/issue1681984 haypo                         
                                                                               



Top Issues Most Discussed (10)
______________________________

 16 tearDown in unittest should be executed regardless of result in    1 days
closed  http://bugs.python.org/issue5538   

 15 test_fileio fails on windows MSVC Assertion                        4 days
open    http://bugs.python.org/issue5544   

 10 float compared to decimal is silently incorrect.                   6 days
open    http://bugs.python.org/issue2531   

  8 yield in iterators                                                 0 days
open    http://bugs.python.org/issue5577   

  8 unittest: allow failures in tearDown method                        1 days
closed  http://bugs.python.org/issue5531   

  6 multiprocessing: switch to autoconf detection of platform value    4 days
open    http://bugs.python.org/issue5545   

  6 indentation in IDLE 2.6 different from IDLE 2.5, 2.4 or vim       53 days
open    http://bugs.python.org/issue5129   

  6 compile() doesn't ignore the source encoding when a string is p  104 days
closed  http://bugs.python.org/issue4626   

  6 32-bits unsigned user/group identifier                           109 days
open    http://bugs.python.org/issue4591   

  5 Byte-code compilation uses excessive memory                        3 days
open    http://bugs.python.org/issue5557   




From jared.grubb at gmail.com  Fri Mar 27 18:19:46 2009
From: jared.grubb at gmail.com (Jared Grubb)
Date: Fri, 27 Mar 2009 10:19:46 -0700
Subject: [Python-Dev] Grammar for plus and minus unary ops
Message-ID: <D3053FED-FCB3-47B2-80B2-5F7A779E7943@gmail.com>

I was recently reviewing some Python code for a friend who is a C++  
programmer, and he had code something like this:

def foo():
   try = 0
   while try<MAX:
      ret = bar()
      if ret: break
      ++try

I was a bit surprised that this was syntactically valid, and because  
the timeout condition only occurred in exceptional cases, the error  
has not yet caused any problems.

It appears that the grammar treats the above example as the unary + op  
applied twice:

u_expr ::=
              power | "-" u_expr
               | "+" u_expr | "\~" u_expr

Playing in the interpreter, expressions like "1+++++++++5" and  "1+-+- 
+-+-+-+-5" evaluate to 6.

I'm not a EBNF expert, but it seems that we could modify the grammar  
to be more restrictive so the above code would not be silently valid.  
E.g., "++5" and "1+++5" and "1+-+5" are syntax errors, but still keep  
"1++5", "1+-5", "1-+5" as valid. (Although, '~' throws in a kink...  
should '~-5' be legal? Seems so...)

Jared

From mal at egenix.com  Fri Mar 27 19:16:24 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 27 Mar 2009 19:16:24 +0100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <20090327170202.4E5D03A40AF@sparrow.telecommunity.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>	<49CBCD01.80500@gmail.com>	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	<20090326213047.6A8413A4097@sparrow.telecommunity.com>	<49CC57FC.30907@trueblade.com>
	<20090327170202.4E5D03A40AF@sparrow.telecommunity.com>
Message-ID: <49CD17F8.1020501@egenix.com>

On 2009-03-27 17:07, P.J. Eby wrote:
> At 11:37 PM 3/26/2009 -0500, Eric Smith wrote:
>> P.J. Eby wrote:
>> > As someone else suggested, moving some of the functionality to PEP 302
>> > interfaces would also help.  Most of the code, though, deals with
>> > locating/inspecting installed distributions, resolving version
>> > requirements, and managing sys.path.  And most of the nastiest
>> > complexity comes from trying to support true filename access to
>> > resources -- if that were dropped from the stdlib, there'd be no need
>> > for egg caches and the like, along with all the complexity entailed.
>> >
>> > Application environments such as Chandler, Trac, Zope, etc. that want
>> > their plugins to live in .egg files wouldn't necessarily be able to use
>> > such an API, but the independent pkg_resources wouldn't be
>> > disappearing.  (Of course, they could also implement
>> > application-specific file extraction, if the stdlib API included the
>> > ability to inspect and open zipped resources.)
>>
>> Could you comment on why they couldn't use such an API?
> 
> If a plugin includes C code (.so/.dll), or uses a library that operates
> on filenames rather than bytes in memory (e.g. gettext), then the
> resources would need to be extracted from the .egg.  pkg_resources
> transparently extracts such resources to a cache directory when you ask
> for a resource's filename, rather than asking for a stream or string of
> its contents.
> 
> This feature represents a significant chunk of the complexity and code
> size of pkg_resources -- and I was proposing ways to cut down on that
> complexity and code size, for a (limited) stdlib version of the
> functionality.

This functionality is one of the more annoying setuptools
"features". It causes each and every user of e.g. Trac to have
their own little version of the same piece of software in their
home dir cache.

The solution to this is simple: don't use ZIP files for installed
packages, instead unzip them into normal directories on sys.path.

This makes all these problems go away and allows users to access
embedded documentation, configuration, etc.

Zip files are great for shipping packages to the end-user, but
there's no need to keep them zipped once they get there.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 27 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2009-03-19: Released mxODBC.Connect 1.0.1      http://python.egenix.com/

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From eric at trueblade.com  Fri Mar 27 19:21:33 2009
From: eric at trueblade.com (Eric Smith)
Date: Fri, 27 Mar 2009 13:21:33 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CD17F8.1020501@egenix.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>	<49CBCD01.80500@gmail.com>	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	<20090326213047.6A8413A4097@sparrow.telecommunity.com>	<49CC57FC.30907@trueblade.com>
	<20090327170202.4E5D03A40AF@sparrow.telecommunity.com>
	<49CD17F8.1020501@egenix.com>
Message-ID: <49CD192D.1090402@trueblade.com>

M.-A. Lemburg wrote:
> On 2009-03-27 17:07, P.J. Eby wrote:
>> At 11:37 PM 3/26/2009 -0500, Eric Smith wrote:
>>> P.J. Eby wrote:
>>>> As someone else suggested, moving some of the functionality to PEP 302
>>>> interfaces would also help.  Most of the code, though, deals with
>>>> locating/inspecting installed distributions, resolving version
>>>> requirements, and managing sys.path.  And most of the nastiest
>>>> complexity comes from trying to support true filename access to
>>>> resources -- if that were dropped from the stdlib, there'd be no need
>>>> for egg caches and the like, along with all the complexity entailed.
>>>>
>>>> Application environments such as Chandler, Trac, Zope, etc. that want
>>>> their plugins to live in .egg files wouldn't necessarily be able to use
>>>> such an API, but the independent pkg_resources wouldn't be
>>>> disappearing.  (Of course, they could also implement
>>>> application-specific file extraction, if the stdlib API included the
>>>> ability to inspect and open zipped resources.)
>>> Could you comment on why they couldn't use such an API?
>> If a plugin includes C code (.so/.dll), or uses a library that operates
>> on filenames rather than bytes in memory (e.g. gettext), then the
>> resources would need to be extracted from the .egg.  pkg_resources
>> transparently extracts such resources to a cache directory when you ask
>> for a resource's filename, rather than asking for a stream or string of
>> its contents.
>>
>> This feature represents a significant chunk of the complexity and code
>> size of pkg_resources -- and I was proposing ways to cut down on that
>> complexity and code size, for a (limited) stdlib version of the
>> functionality.
> 
> This functionality is one of the more annoying setuptools
> "features". It causes each and every user of e.g. Trac to have
> their own little version of the same piece of software in their
> home dir cache.
> 
> The solution to this is simple: don't use ZIP files for installed
> packages, instead unzip them into normal directories on sys.path.
> 
> This makes all these problems go away and allows users to access
> embedded documentation, configuration, etc.
> 
> Zip files are great for shipping packages to the end-user, but
> there's no need to keep them zipped once they get there.
> 

I also think the feature should go. If you want functionality that's so 
difficult to provide when you install as a zip file, the answer is not 
to make things more complex, but to not install as zip files.


From olemis at gmail.com  Fri Mar 27 19:41:30 2009
From: olemis at gmail.com (Olemis Lang)
Date: Fri, 27 Mar 2009 13:41:30 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CD192D.1090402@trueblade.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<49CC57FC.30907@trueblade.com>
	<20090327170202.4E5D03A40AF@sparrow.telecommunity.com>
	<49CD17F8.1020501@egenix.com> <49CD192D.1090402@trueblade.com>
Message-ID: <24ea26600903271141h278fc5bfu7ed5f568bf470073@mail.gmail.com>

On Fri, Mar 27, 2009 at 1:21 PM, Eric Smith <eric at trueblade.com> wrote:
> M.-A. Lemburg wrote:
>> On 2009-03-27 17:07, P.J. Eby wrote:
>>> At 11:37 PM 3/26/2009 -0500, Eric Smith wrote:
>>>>
>>>> P.J. Eby wrote:
>>>>>
>>>>> As someone else suggested, moving some of the functionality to PEP 302
>>>>> interfaces would also help. ?Most of the code, though, deals with
>>>>> locating/inspecting installed distributions, resolving version
>>>>> requirements, and managing sys.path. ?And most of the nastiest
>>>>> complexity comes from trying to support true filename access to
>>>>> resources -- if that were dropped from the stdlib, there'd be no need
>>>>> for egg caches and the like, along with all the complexity entailed.
>>>>>
>>>>> Application environments such as Chandler, Trac, Zope, etc. that want
>>>>> their plugins to live in .egg files wouldn't necessarily be able to use
>>>>> such an API, but the independent pkg_resources wouldn't be
>>>>> disappearing. ?(Of course, they could also implement
>>>>> application-specific file extraction, if the stdlib API included the
>>>>> ability to inspect and open zipped resources.)
>>>>
>>>> Could you comment on why they couldn't use such an API?
>>>
>>> If a plugin includes C code (.so/.dll), or uses a library that operates
>>> on filenames rather than bytes in memory (e.g. gettext), then the
>>> resources would need to be extracted from the .egg. ?pkg_resources
>>> transparently extracts such resources to a cache directory when you ask
>>> for a resource's filename, rather than asking for a stream or string of
>>> its contents.
>>>
>>> This feature represents a significant chunk of the complexity and code
>>> size of pkg_resources --
>>
>> The solution to this is simple: don't use ZIP files for installed
>> packages, instead unzip them into normal directories on sys.path.
>>
>> Zip files are great for shipping packages to the end-user, but
>> there's no need to keep them zipped once they get there.
>>
>
> I also think the feature should go. If you want functionality that's so
> difficult to provide when you install as a zip file, the answer is not to
> make things more complex, but to not install as zip files.
>

What about environments like Google App Engine ? I mean, AFAIK using
ZIP files is the *official* solution to meet some quotas &
requirements ... CMIIW anyway ...

I mean, for example: what if someone writes an app containing
resources like message catalogs for i18n, uploads it to GAE using ZIP
files and still wants to use the MO files (i.e. resources) for
translation (i.e. for anything else ...) ... Hmmmm ?

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From guido at python.org  Fri Mar 27 20:09:20 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Mar 2009 14:09:20 -0500
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CC69D3.4030306@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<49CC69D3.4030306@canterbury.ac.nz>
Message-ID: <ca471dc20903271209t1af35fd9re95f475eb5e81b24@mail.gmail.com>

On Fri, Mar 27, 2009 at 12:53 AM, Greg Ewing
<greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>
>> That +0 could turn into a +1 if there was a way to flag this as an
>> error (at runtime), at least if the return is actually executed:
>>
>> def g():
>> ? ?yield 42
>> ? ?return 43
>>
>> for x in g():
>> ? ?print x ? ?# probably expected to print 42 and then 43
>>
>> Perhaps the exception used in this case could be a different exception
>> than StopIteration?
>
> Would checking that the value is None be sufficient? Or do you
> want to distinguish between 'return' and 'return None'? That
> would feel rather strange to me.

Well currently there *is* a difference, because "return None" is a
(phase 2) SyntaxError in a generator, while "return" isn't. But I
don't really care if you cannot distinguish these two cases.

> I'm inclined to regard this as an unnecessary complication.
> People are already trained to think of 'return' and 'yield'
> as quite different things. I don't know why they would suddenly
> start using 'return' when they mean 'yield'.

Because coroutines are mind-bendingly hard, and it's easy to confuse
yield (pass a value back while keeping the stack frame) and return
(pass a value back dropping the stack frame). The syntactic
prohibition on "return <expr>" in generators currently helps people to
be consistent. I'd like to keep this small crutch around so that
people who are writing vanilla (== meant for iteration) generators are
kept from accidentally introducing hard-to-debug problems.

Perhaps the crux is that *if* you accidentally use "return <value>" in
a vanilla generator expecting the value to show up somewhere, you are
probably enough of a newbie that debugging this will be quite hard.
I'd like not to have such a newbie trap lying around.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mal at egenix.com  Fri Mar 27 20:12:05 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 27 Mar 2009 20:12:05 +0100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <20090327161716.3A8883A40A7@sparrow.telecommunity.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	<20090326213047.6A8413A4097@sparrow.telecommunity.com>	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>	<49CCCB73.7060706@egenix.com>
	<20090327161716.3A8883A40A7@sparrow.telecommunity.com>
Message-ID: <49CD2505.8020001@egenix.com>

On 2009-03-27 17:19, P.J. Eby wrote:
> At 01:49 PM 3/27/2009 +0100, M.-A. Lemburg wrote:
>> (*) I've had a go at this a few months ago and then found out
>> that the egg format itself is not documented anywhere.
> 
> It's been documented for just under three years now.  Here's where you
> quoted the email where I announced that documentation, provided links to
> it, and asked you to let me know if there's anything else you'd need in it:
> 
> http://mail.python.org/pipermail/python-dev/2006-April/064496.html

Thanks for reminding me. I must have forgotten about that
wiki page and instead looked on the setuptools page.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 27 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2009-03-19: Released mxODBC.Connect 1.0.1      http://python.egenix.com/

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From guido at python.org  Fri Mar 27 20:23:06 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Mar 2009 14:23:06 -0500
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CC6CF4.5080603@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com> <49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<20090327041138.EEF893A4097@sparrow.telecommunity.com>
	<49CC6CF4.5080603@canterbury.ac.nz>
Message-ID: <ca471dc20903271223i542038ebrdcac6d93dabea738@mail.gmail.com>

On Fri, Mar 27, 2009 at 1:06 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> P.J. Eby wrote:
>
>> Could we at least have some syntax like 'return from yield with 43', to
>> distinguish it from a regular return, clarify that it's returning a value to
>> a yield-from statement, and emphasize that you need a yield-from to call it?
>
> You don't, though -- yield-from just happens to be a particularly
> convenient way.
>
> I suppose what you really mean is that you can't just use an
> ordinary call. But generators already have that property,
> whether they return values or not, and they're already
> syntactically marked as such by containing 'yield'. I don't
> see that we need a second syntactic marker.

Because for vanilla generators a return is always a mistake.

>> If it doesn't have some sort of variant syntax, the error message for the
>> return exception is going to need to be rather verbose
>
> If we're going to treat this as an error at all, I imagine
> it would say something like "Return value from generator not
> used." RTM to sort out the details.

That would be fine with me.

>> One remaining quirk or missing piece: ISTM there needs to be a way to
>> extract the return value without using a yield-from statement.
>
> I did suggest a for-loop variant for doing this, but Guido warned
> me not to complicate the PEP any further, so I haven't. A followup
> PEP for it might be in order.

I don't think this is a requirement, though I expect there will be a
way since the error will result into some kind of exception, which
means you'll be able to catch it by explicitly invoking __next__().

(BTW, at the language summit we reached the conclusion that proposed
language changes need to be aimed at Py3k first (3.1 in this case) and
backported to 2.x -- 2.7 in this case. You might want to tweak the
proposal to apply to 3.x by default.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Fri Mar 27 20:24:47 2009
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 27 Mar 2009 14:24:47 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CD17F8.1020501@egenix.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<49CC57FC.30907@trueblade.com>
	<20090327170202.4E5D03A40AF@sparrow.telecommunity.com>
	<49CD17F8.1020501@egenix.com>
Message-ID: <18893.10239.228276.666597@montanaro.dyndns.org>


    mal> Zip files are great for shipping packages to the end-user, but
    mal> there's no need to keep them zipped once they get there.

I thought one of the arguments for zip files was a performance increase
(reduced stat(2) calls, etc).  I may misremember though.

Skip

From eric at trueblade.com  Fri Mar 27 20:27:05 2009
From: eric at trueblade.com (Eric Smith)
Date: Fri, 27 Mar 2009 14:27:05 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <24ea26600903271141h278fc5bfu7ed5f568bf470073@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	<20090326213047.6A8413A4097@sparrow.telecommunity.com>	<49CC57FC.30907@trueblade.com>	<20090327170202.4E5D03A40AF@sparrow.telecommunity.com>	<49CD17F8.1020501@egenix.com>
	<49CD192D.1090402@trueblade.com>
	<24ea26600903271141h278fc5bfu7ed5f568bf470073@mail.gmail.com>
Message-ID: <49CD2889.5050109@trueblade.com>

Olemis Lang wrote:
>> I also think the feature should go. If you want functionality that's so
>> difficult to provide when you install as a zip file, the answer is not to
>> make things more complex, but to not install as zip files.
>>
> 
> What about environments like Google App Engine ? I mean, AFAIK using
> ZIP files is the *official* solution to meet some quotas &
> requirements ... CMIIW anyway ...
> 
> I mean, for example: what if someone writes an app containing
> resources like message catalogs for i18n, uploads it to GAE using ZIP
> files and still wants to use the MO files (i.e. resources) for
> translation (i.e. for anything else ...) ... Hmmmm ?

I mentioned yesterday (in the language summit) that we really need to 
get all the requirements together before we start any work. I fear that 
there are many hidden requirements (or ones that I'm personally not 
aware of, at least). I don't know gettext well enough give an answer yet.



From unknown_kev_cat at hotmail.com  Fri Mar 27 20:15:21 2009
From: unknown_kev_cat at hotmail.com (Joe Smith)
Date: Fri, 27 Mar 2009 15:15:21 -0400
Subject: [Python-Dev] Grammar for plus and minus unary ops
References: <D3053FED-FCB3-47B2-80B2-5F7A779E7943@gmail.com>
Message-ID: <gqj9fk$sqv$1@ger.gmane.org>


Jared Grubb wrote:
> I'm not a EBNF expert, but it seems that we could modify the grammar  to 
> be more restrictive so the above code would not be silently valid.  E.g., 
> "++5" and "1+++5" and "1+-+5" are syntax errors, but still keep  "1++5", 
> "1+-5", "1-+5" as valid. (Although, '~' throws in a kink...  should '~-5' 
> be legal? Seems so...)

So you want something like
u_expr :: =
          power | "-" xyzzy_expr | "+" xyzzy_expr | "\~" u_expr
xyzzy_expr :: =
          power | "\~" u_expr

Such that:
5   # valid u_expr
+5  # valid u_expr
-5  # valid u_expr
~5  # valid u_expr
~~5 # valid u_expr
~+5 # valid u_expr
+~5 # valid u_expr
~-5 # valid u_expr
-~5 # valid u_expr
+~-5# valid u_expr

++5 # not valid u_expr
+-5 # not valid u_expr
-+5 # not valid u_expr
--5 # not valid u_expr

While, I'm not a python developer, (just a python user) that sounds 
reasonable to me, as long as this does not silently change the meaning of 
any expression, but only noisily breaks programs, and that the broken 
constructs are not used frequently.

Can anybody come up with any expressions that would silently change in 
meaning if the above were applied?

Obviously a sane name would need to be chosen to replace xyzzy_expr. 



From fdrake at acm.org  Fri Mar 27 20:29:48 2009
From: fdrake at acm.org (Fred Drake)
Date: Fri, 27 Mar 2009 15:29:48 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <18893.10239.228276.666597@montanaro.dyndns.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<49CC57FC.30907@trueblade.com>
	<20090327170202.4E5D03A40AF@sparrow.telecommunity.com>
	<49CD17F8.1020501@egenix.com>
	<18893.10239.228276.666597@montanaro.dyndns.org>
Message-ID: <A5CA88C2-0D8D-4318-A273-34EFEC849C74@acm.org>

On Mar 27, 2009, at 3:24 PM, skip at pobox.com wrote:
> I thought one of the arguments for zip files was a performance  
> increase
> (reduced stat(2) calls, etc).  I may misremember though.


You're memory is working fine, but I don't think the way eggs are used  
accomplishes that.

The measurements that supported that performance assertion were made  
by putting the standard library in a single ZIP, and using that.

I don't have measurements handy, but I recall being told by someone  
who did that using a *lot* of separate ZIP files wasn't as effective.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>


From olemis at gmail.com  Fri Mar 27 20:41:49 2009
From: olemis at gmail.com (Olemis Lang)
Date: Fri, 27 Mar 2009 14:41:49 -0500
Subject: [Python-Dev]  "setuptools has divided the Python community"
In-Reply-To: <24ea26600903271240t6aea5a77gb6eae8dfe2c22b54@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<49CC57FC.30907@trueblade.com>
	<20090327170202.4E5D03A40AF@sparrow.telecommunity.com>
	<49CD17F8.1020501@egenix.com> <49CD192D.1090402@trueblade.com>
	<24ea26600903271141h278fc5bfu7ed5f568bf470073@mail.gmail.com>
	<49CD2889.5050109@trueblade.com>
	<24ea26600903271240t6aea5a77gb6eae8dfe2c22b54@mail.gmail.com>
Message-ID: <24ea26600903271241u28cb759t7a8d5a802c185875@mail.gmail.com>

On Fri, Mar 27, 2009 at 2:27 PM, Eric Smith <eric at trueblade.com> wrote:
> Olemis Lang wrote:
>>>
>>> I also think the feature should go. If you want functionality that's so
>>> difficult to provide when you install as a zip file, the answer is not to
>>> make things more complex, but to not install as zip files.
>>>
>>
>> What about environments like Google App Engine ? I mean, AFAIK using
>> ZIP files is the *official* solution to meet some quotas &
>> requirements ... CMIIW anyway ...
>>
>> I mean, for example: what if someone writes an app containing
>> resources like message catalogs for i18n, uploads it to GAE using ZIP
>> files and still wants to use the MO files (i.e. resources) for
>> translation (i.e. for anything else ...) ... Hmmmm ?
>
> I mentioned yesterday (in the language summit) that we really need to get
> all the requirements together before we start any work. I fear that there
> are many hidden requirements (or ones that I'm personally not aware of, at
> least). I don't know gettext well enough give an answer yet.
>
>

Perhaps it is not necessary ... i18n was only a motivation to
establish a concrete (... *very real* & *no more choices* ...) use
case. The same could be said like this :

?What if someone writes an app containing many (... *A LOT OF* ...)
resource files, uploads it to GAE using ZIP files, and still wants to
use them for anything he/she wants ... Hmmmm ??

;o)

--
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From guido at python.org  Fri Mar 27 20:44:00 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Mar 2009 14:44:00 -0500
Subject: [Python-Dev] Grammar for plus and minus unary ops
In-Reply-To: <gqj9fk$sqv$1@ger.gmane.org>
References: <D3053FED-FCB3-47B2-80B2-5F7A779E7943@gmail.com>
	<gqj9fk$sqv$1@ger.gmane.org>
Message-ID: <ca471dc20903271244p75e4603eydfa0845fc7cff6c6@mail.gmail.com>

Please take this to python-ideas.

On Fri, Mar 27, 2009 at 2:15 PM, Joe Smith <unknown_kev_cat at hotmail.com> wrote:
>
> Jared Grubb wrote:
>>
>> I'm not a EBNF expert, but it seems that we could modify the grammar ?to
>> be more restrictive so the above code would not be silently valid. ?E.g.,
>> "++5" and "1+++5" and "1+-+5" are syntax errors, but still keep ?"1++5",
>> "1+-5", "1-+5" as valid. (Although, '~' throws in a kink... ?should '~-5' be
>> legal? Seems so...)
>
> So you want something like
> u_expr :: =
> ? ? ? ? power | "-" xyzzy_expr | "+" xyzzy_expr | "\~" u_expr
> xyzzy_expr :: =
> ? ? ? ? power | "\~" u_expr
>
> Such that:
> 5 ? # valid u_expr
> +5 ?# valid u_expr
> -5 ?# valid u_expr
> ~5 ?# valid u_expr
> ~~5 # valid u_expr
> ~+5 # valid u_expr
> +~5 # valid u_expr
> ~-5 # valid u_expr
> -~5 # valid u_expr
> +~-5# valid u_expr
>
> ++5 # not valid u_expr
> +-5 # not valid u_expr
> -+5 # not valid u_expr
> --5 # not valid u_expr
>
> While, I'm not a python developer, (just a python user) that sounds
> reasonable to me, as long as this does not silently change the meaning of
> any expression, but only noisily breaks programs, and that the broken
> constructs are not used frequently.
>
> Can anybody come up with any expressions that would silently change in
> meaning if the above were applied?
>
> Obviously a sane name would need to be chosen to replace xyzzy_expr.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Fri Mar 27 20:49:51 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 27 Mar 2009 14:49:51 -0500
Subject: [Python-Dev] version compare function into main lib
In-Reply-To: <49CCF85C.8040902@trueblade.com>
References: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>
	<49CCECAA.7010406@v.loewis.de> <49CCF85C.8040902@trueblade.com>
Message-ID: <49CD2DDF.3010704@v.loewis.de>


> I got the impression that people are generally happy with what 
> setuptools provides for version parsing and comparison.
> 
> Does anyone think that's not a good model?

Procedurally, I think it's a very bad approach. I don't mind
the setuptools implementation being used as a basis (assuming
it gets contributed), but *independently* I think a specfication
is needed what version strings it actually understands. Such
specification must precede the actual implementation (in distutils).

Regards,
Martin

From eric at trueblade.com  Fri Mar 27 20:52:45 2009
From: eric at trueblade.com (Eric Smith)
Date: Fri, 27 Mar 2009 14:52:45 -0500
Subject: [Python-Dev] version compare function into main lib
In-Reply-To: <49CD2DDF.3010704@v.loewis.de>
References: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>	<49CCECAA.7010406@v.loewis.de>
	<49CCF85C.8040902@trueblade.com> <49CD2DDF.3010704@v.loewis.de>
Message-ID: <49CD2E8D.5070509@trueblade.com>

Martin v. L?wis wrote:
> 
>> I got the impression that people are generally happy with what 
>> setuptools provides for version parsing and comparison.
>>
>> Does anyone think that's not a good model?
> 
> Procedurally, I think it's a very bad approach. I don't mind
> the setuptools implementation being used as a basis (assuming
> it gets contributed), but *independently* I think a specfication
> is needed what version strings it actually understands. Such
> specification must precede the actual implementation (in distutils).

Agreed. Specifications first, for all of this work.

From guido at python.org  Fri Mar 27 20:56:00 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Mar 2009 14:56:00 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CCCE7A.8090903@trueblade.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<49CCCB73.7060706@egenix.com> <49CCCE7A.8090903@trueblade.com>
Message-ID: <ca471dc20903271256q212deee1j179a23a89b55d596@mail.gmail.com>

On Fri, Mar 27, 2009 at 8:02 AM, Eric Smith <eric at trueblade.com> wrote:
> M.-A. Lemburg wrote:
>>
>> On 2009-03-27 04:19, Guido van Rossum wrote:
>>>
>>> - keep distutils, but start deprecating certain higher-level
>>> functionality in it (e.g. bdist_rpm)
>>> - don't try to provide higher-level functionality in the stdlib, but
>>> instead let third party tools built on top of these core APIs compete
>>
>> Should this be read as:
>>
>> - remove bdist_rpm from the stdlib and let it live on PyPI
>>
>> ?
>
> As one of the people who proposed this, I think it means: move bdist_rpm,
> bdist_msi, etc. out of distutils, but provide some of them with the standard
> Python installation. I'm certain that as part of the refactoring and
> simplification of distutils we'll gradually move the existing bdist_*
> commands into separate, stand-alone "things" (scripts, callable modules, or
> something). We'll need to do this if only for testing, so we may as well
> make them work.

One of the motivations for deprecating this (and for using this
specific example) was that Matthias Klose, the Python packager for
Debian, said he never uses bdist_rpm.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fdrake at acm.org  Fri Mar 27 20:59:42 2009
From: fdrake at acm.org (Fred Drake)
Date: Fri, 27 Mar 2009 15:59:42 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <ca471dc20903271256q212deee1j179a23a89b55d596@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<49CCCB73.7060706@egenix.com> <49CCCE7A.8090903@trueblade.com>
	<ca471dc20903271256q212deee1j179a23a89b55d596@mail.gmail.com>
Message-ID: <C21D1DFA-A88E-45DD-A8B5-7999637138C5@acm.org>

On Mar 27, 2009, at 3:56 PM, Guido van Rossum wrote:
> One of the motivations for deprecating this (and for using this
> specific example) was that Matthias Klose, the Python packager for
> Debian, said he never uses bdist_rpm.

Given that Debian doesn't use RPMs, isn't that expected?

I'm actually in favor of removing the bdist_* from the standard  
library, and allowing 3rd-party tools to implement whatever they need  
for the distros.  But I don't think what you're presenting there  
supports it.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>


From ziade.tarek at gmail.com  Fri Mar 27 21:06:21 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 27 Mar 2009 15:06:21 -0500
Subject: [Python-Dev] My summit notes (packaging)
Message-ID: <94bdd2610903271306i9e627a9ubc3cfa68805a15f2@mail.gmail.com>

Here's a wrapup of my notes for the Distutils part (with Jim's help).
The PyPI part will come later.

I have presented the various problems developers have with packaging
today, wether they are using Distutils, Setuptools or any other third
party tools out there.

Here's the short-list:

- Distutils code is still undertested, with over 100 issues to
  take care of in the tracker.

- Setuptools is the de-facto extension of Distutils. It brings
  a lot of good features, but some features are controversial
  amongst the community.

- We have now two tools that install a package and look for its
  dependencies at PyPI and install them: pip and easy_install.
  The latter is included in setuptools. They both aim at the
  same goal besides a few differences, and they both rely
  on a new metadata introduced by setuptools, wich is.
  "install_requires". This new metadata extends the metadata.
  described in PEP 314 but is slightly different from.
  what is descibred in the Draft PEP 345  ("Requires").
..
  PEP 345 introduces "Requires" and "Provides" wich are
  are implemented in Distutils and PyP, but are not
  widely used. 40 out of +4000 if I remember correctly. Martin will
  correct me here if I am wrong.

- pip, easy_install and distutils use different formats for installed
  metadata. For instance, Distutils installs a PKG-INFO file whereas
  setuptools creates a directory with the PKG-INFO file in it.

- There's no way to uninstall a package. The only way to do it
  is to remove manually the directory located in your site-packages
  and remove the line located in the .pth file that might point to it.

- There's no standard way to use resource files, for instance
  to be able to read or write a resource file that might be installed
  in a specific place on the target system by the system packager.
  Most of the time people are just working with files located.
  in the package directory.

Ian Bicking and Jim Fulton gave short presentations of virtualenv
and zc.buildout. These tools will let you build an isolated environment,
where you can install some packages of certain versions, that play well
together to run an application, without having to put anything into your
site-packages. Many agreed that adding packages into the system's
site-packages for an application to run is not a good idea.

Also, many agreed that these third party tools are doing their.
job and that we shouldn't try to include some of the features they.
provide in any way in the standard library. Such as supporting multiple.
versions for the same package for example.

Instead, the idea is to try to reduce the scope of Distutils and to
provide some better API and conventions for people to access to the package
metadata (wether it's installed or not) because people have different
needs and purposes using these data.

*What's next ?*

Guido provided a high level plan for the work to be done in packaging.
Let me quote him:

    - standardize more metadata, especially including dependencies,.
    and APIs for accessing the metadata and dependencies.

    - there should be an API to get metadata for a package without
    actually executing any of the package's installation script.

    - this will go into the stdlib for Python 2.7 and 3.1

    - it will be provided as separately downloadable backports for prior
    versions (maybe using a bootstrap not unlike ez_install)

    - keep distutils, but start deprecating certain higher-level
    functionality in it (e.g. bdist_rpm)

    - don't try to provide higher-level functionality in the stdlib, but
    instead let third party tools built on top of these core APIs compete

So we are currently working this week to flesh out this plan, and we
will send a mail to Distutils-SIG with the ouput asap.

Cheers
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From pje at telecommunity.com  Fri Mar 27 21:14:43 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 27 Mar 2009 16:14:43 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CD2505.8020001@egenix.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<49CCCB73.7060706@egenix.com>
	<20090327161716.3A8883A40A7@sparrow.telecommunity.com>
	<49CD2505.8020001@egenix.com>
Message-ID: <20090327201219.16B7D3A40A7@sparrow.telecommunity.com>

At 08:12 PM 3/27/2009 +0100, M.-A. Lemburg wrote:
>On 2009-03-27 17:19, P.J. Eby wrote:
> > At 01:49 PM 3/27/2009 +0100, M.-A. Lemburg wrote:
> >> (*) I've had a go at this a few months ago and then found out
> >> that the egg format itself is not documented anywhere.
> >
> > It's been documented for just under three years now.  Here's where you
> > quoted the email where I announced that documentation, provided links to
> > it, and asked you to let me know if there's anything else you'd need in it:
> >
> > http://mail.python.org/pipermail/python-dev/2006-April/064496.html
>
>Thanks for reminding me. I must have forgotten about that
>wiki page and instead looked on the setuptools page.

For future reference, you can also Google "Python egg formats" or 
"Python egg file format" or other such phrases, and it always comes 
up as the first or second hit.  Ironically, that's how I found it the 
other day myself, as I didn't remember right off what the wiki page 
name was.  ;-)


From olemis at gmail.com  Fri Mar 27 21:26:44 2009
From: olemis at gmail.com (Olemis Lang)
Date: Fri, 27 Mar 2009 15:26:44 -0500
Subject: [Python-Dev]  "setuptools has divided the Python community"
In-Reply-To: <24ea26600903271312j644d5db9la2e825303b45dc2a@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<49CCCB73.7060706@egenix.com> <49CCCE7A.8090903@trueblade.com>
	<ca471dc20903271256q212deee1j179a23a89b55d596@mail.gmail.com>
	<C21D1DFA-A88E-45DD-A8B5-7999637138C5@acm.org>
	<24ea26600903271312j644d5db9la2e825303b45dc2a@mail.gmail.com>
Message-ID: <24ea26600903271326k2ff264d6t9072ea55caa8fb49@mail.gmail.com>

On Fri, Mar 27, 2009 at 2:59 PM, Fred Drake <fdrake at acm.org> wrote:
> On Mar 27, 2009, at 3:56 PM, Guido van Rossum wrote:
>>
>> One of the motivations for deprecating this (and for using this
>> specific example) was that Matthias Klose, the Python packager for
>> Debian, said he never uses bdist_rpm.
>
> Given that Debian doesn't use RPMs,

Only if using alien ... but ...

> isn't that expected?
>

... yes I assume that the best way for building debs is not building
an RPM first so ...

> I'm actually in favor of removing the bdist_* from the standard library, and
> allowing 3rd-party tools to implement whatever they need for the distros.
> ?But I don't think what you're presenting there supports it.
>

I agree ...


BTW ... bdist_rpm is also there in Windows, and viceversa,
bdist_wininst is also the in FC | RH ... and noione needs that, except
devs willing to build RPMs in order to provide their own installers
... but most of the use cases for distutils bdist_* cmds come from
similar situations ... IMHO ... and the OS of users of a pkg doesnt
match the OS of the devs. The later may even have different OSs
installed ... ;)

I mean, who is gonna use bdist_* if not devs or distro maintainers or
somebody trying to build an installable artifact (... RPM, DEB, MSI,
...) for users having their own OS-flavors ?

--
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From glyph at divmod.com  Fri Mar 27 21:49:53 2009
From: glyph at divmod.com (glyph at divmod.com)
Date: Fri, 27 Mar 2009 20:49:53 -0000
Subject: [Python-Dev] splitting out bdist_* (was: interminable 'setuptools'
	thread)
Message-ID: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>


On 07:59 pm, fdrake at acm.org wrote:
>I'm actually in favor of removing the bdist_* from the standard 
>library, and allowing 3rd-party tools to implement whatever they need 
>for the distros.  But I don't think what you're presenting there 
>supports it.

I do think that it's relevant that the respective operating system 
packagers don't find bdist_rpm, bdist_deb, et. al. useful.  It's not 
very useful to have a bdist_deb that nobody actually builds debs with. 
This has been a problem for me, personally, since debian has made 
various ad-hoc change to Twisted or Twisted-based packages to break our 
plugin system, since the distutils metadata has been insufficient for 
their purposes.  If the deb-generating stuff were in a separate project 
with a faster release cycle that were easier to contribute packages to, 
perhaps debian packagers could be convinced to contribute their build- 
hacks there (and bdist_deb could invoke debhelper, or vice-versa).

It would be great if someone could volunteer to maintain this stuff 
independently, put it in a Launchpad project or something.  IMHO it 
would be better for the community at large if this were spun as 
"increasing the release speed" of the bdist_* family, rather than 
"removing", which seems to me like it would generate another teacup- 
tempest on the blogowebs.  Of course I'm not volunteering, but I will be 
best friends forever with whoever does this PR/maintenance :).

Given that py2exe and py2app (which includes "bdist_mpkg") are both 
based on distutils, it seems like we're on the way to independent 
maintenance anyway.  Perhaps bdist_wininst/_msi could be donated to the 
py2exe project if they would be willing to maintain it, and the new 
project for _deb and _rpm could be called "py2packman" or something.

From ziade.tarek at gmail.com  Fri Mar 27 21:36:49 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 27 Mar 2009 15:36:49 -0500
Subject: [Python-Dev] My summit notes (packaging)
In-Reply-To: <20090327202429.223D73A40A7@sparrow.telecommunity.com>
References: <94bdd2610903271306i9e627a9ubc3cfa68805a15f2@mail.gmail.com>
	<20090327202429.223D73A40A7@sparrow.telecommunity.com>
Message-ID: <94bdd2610903271336u4d959822l8acb53e78792d883@mail.gmail.com>

2009/3/27 P.J. Eby <pje at telecommunity.com>:
> Also, it's quite likely that platform-specific dependencies may exist as
> well. ?It might be possible to accommodate these things with a sufficiently
> flexible format, but currently, the only way to handle them with
> distutils/setuptools is in the setup script.
>

Yes, we are working on these particular issues when some metadata
require some code to run.
But most of the time, a package is able to express its dependencies in
a static file.

Regards
Tarek

From theller at ctypes.org  Fri Mar 27 21:42:31 2009
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 27 Mar 2009 21:42:31 +0100
Subject: [Python-Dev] splitting out bdist_* (was: interminable
	'setuptools' thread)
In-Reply-To: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
References: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
Message-ID: <gqjdnl$mju$1@ger.gmane.org>

glyph at divmod.com schrieb:
> On 07:59 pm, fdrake at acm.org wrote:
>>I'm actually in favor of removing the bdist_* from the standard 
>>library, and allowing 3rd-party tools to implement whatever they need 
>>for the distros.  But I don't think what you're presenting there 
>>supports it.
> 
> I do think that it's relevant that the respective operating system 
> packagers don't find bdist_rpm, bdist_deb, et. al. useful.  It's not 
> very useful to have a bdist_deb that nobody actually builds debs with. 
> This has been a problem for me, personally, since debian has made 
> various ad-hoc change to Twisted or Twisted-based packages to break our 
> plugin system, since the distutils metadata has been insufficient for 
> their purposes.  If the deb-generating stuff were in a separate project 
> with a faster release cycle that were easier to contribute packages to, 
> perhaps debian packagers could be convinced to contribute their build- 
> hacks there (and bdist_deb could invoke debhelper, or vice-versa).
> 
> It would be great if someone could volunteer to maintain this stuff 
> independently, put it in a Launchpad project or something.  IMHO it 
> would be better for the community at large if this were spun as 
> "increasing the release speed" of the bdist_* family, rather than 
> "removing", which seems to me like it would generate another teacup- 
> tempest on the blogowebs.  Of course I'm not volunteering, but I will be 
> best friends forever with whoever does this PR/maintenance :).
> 
> Given that py2exe and py2app (which includes "bdist_mpkg") are both 
> based on distutils, it seems like we're on the way to independent 
> maintenance anyway.  Perhaps bdist_wininst/_msi could be donated to the 
> py2exe project if they would be willing to maintain it, and the new 
> project for _deb and _rpm could be called "py2packman" or something.

Well, py2exe is Windows only.  And I know that people used bdist_wininst
to create windows installers on Linux.

-- 
Thanks,
Thomas


From tjreedy at udel.edu  Fri Mar 27 21:44:43 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 27 Mar 2009 16:44:43 -0400
Subject: [Python-Dev] Bug#521275: Acknowledgement (colored prompt
 conflicts with cursor positioning)
In-Reply-To: <handler.521275.B.123806739521183.ack@bugs.debian.org>
References: <200903261236.21657.eckhardt@satorlaser.com>
	<handler.521275.B.123806739521183.ack@bugs.debian.org>
Message-ID: <gqjdrp$u1e$1@ger.gmane.org>

I forwarded this to owner at bugs.debian.org (and the actual submitter) 
suggesting that this was misaddressed.

Debian Bug Tracking System wrote:
> Thank you for filing a new Bug report with Debian.
> 
> This is an automatically generated reply to let you know your message
> has been received.
> 
> Your message is being forwarded to the package maintainers and other
> interested parties for their attention; they will reply in due course.
> 
> Your message has been sent to the package maintainer(s):
>  Matthias Klose <doko at debian.org>
> 
> If you wish to submit further information on this problem, please
> send it to 521275 at bugs.debian.org, as before.
> 
> Please do not send mail to owner at bugs.debian.org unless you wish
> to report a problem with the Bug-tracking system.
> 
> 


From mal at egenix.com  Fri Mar 27 21:48:24 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 27 Mar 2009 21:48:24 +0100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <ca471dc20903271256q212deee1j179a23a89b55d596@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	<20090326213047.6A8413A4097@sparrow.telecommunity.com>	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>	<49CCCB73.7060706@egenix.com>
	<49CCCE7A.8090903@trueblade.com>
	<ca471dc20903271256q212deee1j179a23a89b55d596@mail.gmail.com>
Message-ID: <49CD3B98.7060004@egenix.com>

On 2009-03-27 20:56, Guido van Rossum wrote:
> On Fri, Mar 27, 2009 at 8:02 AM, Eric Smith <eric at trueblade.com> wrote:
>> M.-A. Lemburg wrote:
>>> On 2009-03-27 04:19, Guido van Rossum wrote:
>>>> - keep distutils, but start deprecating certain higher-level
>>>> functionality in it (e.g. bdist_rpm)
>>>> - don't try to provide higher-level functionality in the stdlib, but
>>>> instead let third party tools built on top of these core APIs compete
>>> Should this be read as:
>>>
>>> - remove bdist_rpm from the stdlib and let it live on PyPI
>>>
>>> ?
>> As one of the people who proposed this, I think it means: move bdist_rpm,
>> bdist_msi, etc. out of distutils, but provide some of them with the standard
>> Python installation. I'm certain that as part of the refactoring and
>> simplification of distutils we'll gradually move the existing bdist_*
>> commands into separate, stand-alone "things" (scripts, callable modules, or
>> something). We'll need to do this if only for testing, so we may as well
>> make them work.
> 
> One of the motivations for deprecating this (and for using this
> specific example) was that Matthias Klose, the Python packager for
> Debian, said he never uses bdist_rpm.

Why would a Debian developer want to use bdist_rpm ? Perhaps you
meant a bdist_deb command, but that's not part of the stdlib
distutils.

More importantly:

Why is the non-use of a command by a single Python developer enough
motivation to remove a useful feature of distutils that's been in
use by many others for years ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 27 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2009-03-19: Released mxODBC.Connect 1.0.1      http://python.egenix.com/

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ziade.tarek at gmail.com  Fri Mar 27 21:55:34 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Fri, 27 Mar 2009 15:55:34 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CD3B98.7060004@egenix.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<49CCCB73.7060706@egenix.com> <49CCCE7A.8090903@trueblade.com>
	<ca471dc20903271256q212deee1j179a23a89b55d596@mail.gmail.com>
	<49CD3B98.7060004@egenix.com>
Message-ID: <94bdd2610903271355p19719c92l3a56414d754bf637@mail.gmail.com>

On Fri, Mar 27, 2009 at 3:48 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> More importantly:
>
> Why is the non-use of a command by a single Python developer enough
> motivation to remove a useful feature of distutils that's been in
> use by many others for years ?

>From the discussions I had with RPM packagers, bdist_rpm is hard to use to
comply with all the different RPM-based systems out there,

Fedora, Red Hat, etc..

I think that each OS community should maintain its own tool, that complies
to the OS standard (wich has its own evolution cycle)

Of course this will be possible as long as Distutils let the system
packager find/change
the metadata in an easy way.

I think this is the same rationale for debian packages. Right now people tend
to use external tools like stdeb and they are OK with it (but still
gets problems
extracting stuff out of Python packages at this point)

Regards
Tarek

-- 
Tarek Ziad? | Association AfPy | www.afpy.org
Blog FR | http://programmation-python.org
Blog EN | http://tarekziade.wordpress.com/

From mal at egenix.com  Fri Mar 27 22:01:47 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 27 Mar 2009 22:01:47 +0100
Subject: [Python-Dev] splitting out bdist_* (was: interminable
 'setuptools' thread)
In-Reply-To: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
References: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
Message-ID: <49CD3EBB.9070409@egenix.com>

On 2009-03-27 21:49, glyph at divmod.com wrote:
> 
> On 07:59 pm, fdrake at acm.org wrote:
>> I'm actually in favor of removing the bdist_* from the standard
>> library, and allowing 3rd-party tools to implement whatever they need
>> for the distros.  But I don't think what you're presenting there
>> supports it.
> 
> I do think that it's relevant that the respective operating system
> packagers don't find bdist_rpm, bdist_deb, et. al. useful.  It's not
> very useful to have a bdist_deb that nobody actually builds debs with.
> This has been a problem for me, personally, since debian has made
> various ad-hoc change to Twisted or Twisted-based packages to break our
> plugin system, since the distutils metadata has been insufficient for
> their purposes.  If the deb-generating stuff were in a separate project
> with a faster release cycle that were easier to contribute packages to,
> perhaps debian packagers could be convinced to contribute their build-
> hacks there (and bdist_deb could invoke debhelper, or vice-versa).
> 
> It would be great if someone could volunteer to maintain this stuff
> independently, put it in a Launchpad project or something.  IMHO it
> would be better for the community at large if this were spun as
> "increasing the release speed" of the bdist_* family, rather than
> "removing", which seems to me like it would generate another teacup-
> tempest on the blogowebs.  Of course I'm not volunteering, but I will be
> best friends forever with whoever does this PR/maintenance :).
> 
> Given that py2exe and py2app (which includes "bdist_mpkg") are both
> based on distutils, it seems like we're on the way to independent
> maintenance anyway.  Perhaps bdist_wininst/_msi could be donated to the
> py2exe project if they would be willing to maintain it, and the new
> project for _deb and _rpm could be called "py2packman" or something.

Do you really think that splitting up the distutils package is
going to create a better user experience ?

What would benefit the bdist_* commands is externalized maintenance,
ie. have more frequent releases on PyPI, but still ship the
most up-to-date versions with core distutils in each new Python
release.

BTW: py2exe and py2app solve a different set of problems than distutils
is trying to solve. They are about packaging complete applications,
not individual packages, so I don't see how they relate to the
bdist_* commands. But perhaps I'm missing some context.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 27 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2009-03-19: Released mxODBC.Connect 1.0.1      http://python.egenix.com/

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From barry at python.org  Fri Mar 27 22:15:20 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 27 Mar 2009 16:15:20 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <24ea26600903270551w36295e9dld8c6eb2343261c14@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<3580D947-0674-41D4-BC90-A83C6541095D@python.org>
	<24ea26600903261307x1042a7bfn61d9ce8db02c16c7@mail.gmail.com>
	<094EF425-5929-48F6-8024-E61D28D5B0EC@python.org>
	<24ea26600903270551w36295e9dld8c6eb2343261c14@mail.gmail.com>
Message-ID: <74BD8119-FD6C-4D41-AF0E-4BCDE5409043@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 27, 2009, at 7:51 AM, Olemis Lang wrote:

> from pkg_resources import *
>
> for fnm in sorted(resource_listdir('mailman.database', 'sql'), \
>                   my_own_cmp ): # Only if needed ... ;)

Thanks, it was pkg_resource.resource_listdir() that I was missing.
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSc1B6HEjvBPtnXfVAQI39wP/VZ4m7x8hd6HzI4VFetFwe/zgLxIJO5Wt
adpQEv/hou+JVgRxSaa0mNAzZSVzDw8pDa2aZ0RXeM/fEaSnhX2IUh+28gNvhyMZ
m+57LF5GZMxJsV2fYNj6ELFKeC/JXCQYB43Fxs7AShzlP/1LRnrN9LntQzFUp6hV
/GddDd/vOhw=
=FN5H
-----END PGP SIGNATURE-----

From mal at egenix.com  Fri Mar 27 22:22:50 2009
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 27 Mar 2009 22:22:50 +0100
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <18893.10239.228276.666597@montanaro.dyndns.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>	<49CBCD01.80500@gmail.com>	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	<20090326213047.6A8413A4097@sparrow.telecommunity.com>	<49CC57FC.30907@trueblade.com>	<20090327170202.4E5D03A40AF@sparrow.telecommunity.com>	<49CD17F8.1020501@egenix.com>
	<18893.10239.228276.666597@montanaro.dyndns.org>
Message-ID: <49CD43AA.1080805@egenix.com>

On 2009-03-27 20:24, skip at pobox.com wrote:
>     mal> Zip files are great for shipping packages to the end-user, but
>     mal> there's no need to keep them zipped once they get there.
> 
> I thought one of the arguments for zip files was a performance increase
> (reduced stat(2) calls, etc).  I may misremember though.

True and like Fred already mentioned that's the main reason why we
have ZIP file package imports.

Putting the stdlib into a ZIP file does make our favorite interpreter
start up faster. E.g. py2exe makes use of this feature.

However, using eggs (which are ZIP files) directly on the sys.path
causes these to get scanned on every startup - regardless of whether
you use any of their content. Very much unlike standard Python package
directories that only get scanned if they are referenced.

Due to the nature of eggs (many small packages), you usually end up
having a whole Easter basket full of them in your sys.path.

Unless you want to maintain a separate Python installation per task,
your overall Python startup time will increase noticeably for every
single script you run with it.

Perhaps someone should start working on a tool called "FryingPan" to
create "Omelettes", ie. all eggs squashed into a single ZIP file... ;-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Mar 27 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2009-03-19: Released mxODBC.Connect 1.0.1      http://python.egenix.com/

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From martin at v.loewis.de  Fri Mar 27 22:23:03 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 27 Mar 2009 16:23:03 -0500
Subject: [Python-Dev] splitting out bdist_* (was: interminable
 'setuptools' thread)
In-Reply-To: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
References: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
Message-ID: <49CD43B7.3050904@v.loewis.de>

> I do think that it's relevant that the respective operating system 
> packagers don't find bdist_rpm, bdist_deb, et. al. useful.  It's not 
> very useful to have a bdist_deb that nobody actually builds debs with. 

I think that conclusion is invalid: just because the distributions don't
use it doesn't mean that nobody uses it. As a data point, there are 16
packages on PyPI that release RPMs (I haven't checked how
they actually built them, though).

> This has been a problem for me, personally, since debian has made 
> various ad-hoc change to Twisted or Twisted-based packages to break our 
> plugin system, since the distutils metadata has been insufficient for 
> their purposes.  If the deb-generating stuff were in a separate project 
> with a faster release cycle that were easier to contribute packages to, 
> perhaps debian packagers could be convinced to contribute their build- 
> hacks there (and bdist_deb could invoke debhelper, or vice-versa).

I don't think this would happen. For .deb, you can't simply have
"deb-generating stuff" - you have to acually manually package each
file. The only way to get that out of the hands of the Debian maintainer
would be if the package author provided the necessary data, which
in turn requires that the "deb generating stuff" is readily available
to the package author.

In fact, .deb is a proof that it does *not* help to have the package
commands outside distutils. For .deb, the command actually *is* outside
distutils (there is no bdist_deb in distutils) - and it hasn't helped.

> It would be great if someone could volunteer to maintain this stuff 
> independently, put it in a Launchpad project or something.

Perhaps. However, for none of the bdist commands, anybody actually
volunteered to maintain them outside of distutils. I would not want
to see any of them removed until I know whom to blame when they
die after being removed :-)

> Given that py2exe and py2app (which includes "bdist_mpkg") are both 
> based on distutils, it seems like we're on the way to independent 
> maintenance anyway.  Perhaps bdist_wininst/_msi could be donated to the 
> py2exe project if they would be willing to maintain it, and the new 
> project for _deb and _rpm could be called "py2packman" or something.

Perhaps. I'm skeptical that they want it. Also notice that bdist_wininst
works fine on Linux, and I'm skeptical that Linux users would have easy
access to it if it became part of py2exe.

Regards,
Martin


From barry at python.org  Fri Mar 27 22:27:52 2009
From: barry at python.org (Barry Warsaw)
Date: Fri, 27 Mar 2009 16:27:52 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CD2889.5050109@trueblade.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	<20090326213047.6A8413A4097@sparrow.telecommunity.com>	<49CC57FC.30907@trueblade.com>	<20090327170202.4E5D03A40AF@sparrow.telecommunity.com>	<49CD17F8.1020501@egenix.com>
	<49CD192D.1090402@trueblade.com>
	<24ea26600903271141h278fc5bfu7ed5f568bf470073@mail.gmail.com>
	<49CD2889.5050109@trueblade.com>
Message-ID: <0B6ECAA5-0BA9-4D72-81A6-A8EF27FFBC76@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 27, 2009, at 2:27 PM, Eric Smith wrote:

> Olemis Lang wrote:
>>> I also think the feature should go. If you want functionality  
>>> that's so
>>> difficult to provide when you install as a zip file, the answer is  
>>> not to
>>> make things more complex, but to not install as zip files.
>>>
>> What about environments like Google App Engine ? I mean, AFAIK using
>> ZIP files is the *official* solution to meet some quotas &
>> requirements ... CMIIW anyway ...
>> I mean, for example: what if someone writes an app containing
>> resources like message catalogs for i18n, uploads it to GAE using ZIP
>> files and still wants to use the MO files (i.e. resources) for
>> translation (i.e. for anything else ...) ... Hmmmm ?
>
> I mentioned yesterday (in the language summit) that we really need  
> to get all the requirements together before we start any work. I  
> fear that there are many hidden requirements (or ones that I'm  
> personally not aware of, at least). I don't know gettext well enough  
> give an answer yet.

The class-based API for gettext takes streams, so resource_stream()  
would work just fine.  I think i18n plugins for Python do not  
necessarily need to use the classic gettext API.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSc1E2XEjvBPtnXfVAQKc5QP+NOgx0Q/yZ47d1HKR2M/TP4gwT08hr1K2
rTZ+YsGVQ6B+pO36uBi3dwOWNzVbPd3VMBCPNE/TBiGRHklqNiI4AxjRyougdmcw
6nhZomS0GxlE//r27nKx/Wl+qS/b8eQ6UWB/jpGRGL1+K/TuI4Zij0yUYBL7PUgn
7JZxeLkueOY=
=MI9F
-----END PGP SIGNATURE-----

From fdrake at acm.org  Fri Mar 27 22:31:11 2009
From: fdrake at acm.org (Fred Drake)
Date: Fri, 27 Mar 2009 17:31:11 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CD43AA.1080805@egenix.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<49CC57FC.30907@trueblade.com>
	<20090327170202.4E5D03A40AF@sparrow.telecommunity.com>
	<49CD17F8.1020501@egenix.com>
	<18893.10239.228276.666597@montanaro.dyndns.org>
	<49CD43AA.1080805@egenix.com>
Message-ID: <952D0C14-6188-4ACA-96A0-986E79EB4CF6@acm.org>

On Mar 27, 2009, at 5:22 PM, M.-A. Lemburg wrote:
> Perhaps someone should start working on a tool called "FryingPan" to
> create "Omelettes", ie. all eggs squashed into a single ZIP  
> file... ;-)


I've certainly suggested such a tool in various conversations, but it  
usually comes down to not wanting to build a separate ZIP for  
(potentially) every script.

In practice, I suspect it's a lot better than this.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>


From skip at pobox.com  Fri Mar 27 22:32:41 2009
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 27 Mar 2009 16:32:41 -0500 (CDT)
Subject: [Python-Dev] Partial 2to3?
Message-ID: <20090327213242.00422EB016D@montanaro.dyndns.org>

Following up on yesterday's conversation about 2to3 and 3to2, I wonder if
it's easily possible to run 2to3 with a specific small subset of its fixers?
For example, people not wanting to make the 2->3 leap yet might still be
interersted in the exception handling changes ("except Foo as exc")?  

Thx,

-- 
Skip Montanaro - skip at pobox.com - http://www.smontanaro.net/

From collinw at gmail.com  Fri Mar 27 22:36:20 2009
From: collinw at gmail.com (Collin Winter)
Date: Fri, 27 Mar 2009 16:36:20 -0500
Subject: [Python-Dev] Partial 2to3?
In-Reply-To: <20090327213242.00422EB016D@montanaro.dyndns.org>
References: <20090327213242.00422EB016D@montanaro.dyndns.org>
Message-ID: <43aa6ff70903271436h15400426h309051ef0d4e2458@mail.gmail.com>

2009/3/27  <skip at pobox.com>:
> Following up on yesterday's conversation about 2to3 and 3to2, I wonder if
> it's easily possible to run 2to3 with a specific small subset of its fixers?
> For example, people not wanting to make the 2->3 leap yet might still be
> interersted in the exception handling changes ("except Foo as exc")?

Sure, that's easily possible: run 2to3 -f
some_fixer,other_fixer,this_fixer,that_fixer. You can get a full list
of fixers using the --list-fixes option.

Collin Winter

From guido at python.org  Fri Mar 27 22:37:53 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Mar 2009 16:37:53 -0500
Subject: [Python-Dev] Partial 2to3?
In-Reply-To: <20090327213242.00422EB016D@montanaro.dyndns.org>
References: <20090327213242.00422EB016D@montanaro.dyndns.org>
Message-ID: <ca471dc20903271437u2079cbfdq57d129b8fa85f3d7@mail.gmail.com>

Yes, you can easily specify the set of fixers to use with a
command-line flag. Each specific transform (e.g. "except a, b" ->
"except a as b") can be turned on or off that way.

2009/3/27  <skip at pobox.com>:
> Following up on yesterday's conversation about 2to3 and 3to2, I wonder if
> it's easily possible to run 2to3 with a specific small subset of its fixers?
> For example, people not wanting to make the 2->3 leap yet might still be
> interersted in the exception handling changes ("except Foo as exc")?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Fri Mar 27 22:50:18 2009
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 27 Mar 2009 16:50:18 -0500
Subject: [Python-Dev] Version strings
Message-ID: <49CD4A1A.6050008@v.loewis.de>

I just collected the version strings of versions that
got released to PyPI, and put up the list on

http://www.dcl.hpi.uni-potsdam.de/home/loewis/versions

Most of them probably fit into any schema we come up with,
but there are certainly some unconventional ones:

1 to 3
1.*
Verrsion 1
working proof of concept

Regards,
Martin

From eric at trueblade.com  Fri Mar 27 22:51:11 2009
From: eric at trueblade.com (Eric Smith)
Date: Fri, 27 Mar 2009 16:51:11 -0500
Subject: [Python-Dev] splitting out bdist_*
In-Reply-To: <49CD43B7.3050904@v.loewis.de>
References: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
	<49CD43B7.3050904@v.loewis.de>
Message-ID: <49CD4A4F.30900@trueblade.com>

Martin v. L?wis wrote:
>> I do think that it's relevant that the respective operating system 
>> packagers don't find bdist_rpm, bdist_deb, et. al. useful.  It's not 
>> very useful to have a bdist_deb that nobody actually builds debs with. 
> 
> I think that conclusion is invalid: just because the distributions don't
> use it doesn't mean that nobody uses it. As a data point, there are 16
> packages on PyPI that release RPMs (I haven't checked how
> they actually built them, though).

And I personally use bdist_rpm for my work, which I distribute to a farm 
of servers under my control. So no doubt it's used.

> In fact, .deb is a proof that it does *not* help to have the package
> commands outside distutils. For .deb, the command actually *is* outside
> distutils (there is no bdist_deb in distutils) - and it hasn't helped.

It proves that it doesn't help given the current state of affairs. I 
suspect that if all of this additional information needed to build a 
.deb (for example) could be included as metadata in the python package 
(using the word "package" loosely), that it would be. It would make the 
ultimate packager's life easier, and it's no real burden for the 
original author.


From bignose+hates-spam at benfinney.id.au  Fri Mar 27 23:37:11 2009
From: bignose+hates-spam at benfinney.id.au (Ben Finney)
Date: Sat, 28 Mar 2009 09:37:11 +1100
Subject: [Python-Dev] version compare function into main lib
References: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>
	<49CCECAA.7010406@v.loewis.de> <49CCF85C.8040902@trueblade.com>
	<49CD2DDF.3010704@v.loewis.de>
Message-ID: <87zlf6sico.fsf@benfinney.id.au>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> I don't mind the setuptools implementation being used as a basis
> (assuming it gets contributed), but *independently* I think a
> specfication is needed what version strings it actually understands.
> Such specification must precede the actual implementation (in
> distutils).

Yes, please. The comparison of version strings needs to be easily done
by non-Python programs (e.g. tools for packaging Python
distributions), so a specification that can be implemented in other
languages or environments is a must.

-- 
 \            ?I was in the grocery store. I saw a sign that said ?pet |
  `\      supplies?. So I did. Then I went outside and saw a sign that |
_o__)                             said ?compact cars?.? ?Steven Wright |
Ben Finney


From eric at trueblade.com  Fri Mar 27 23:28:38 2009
From: eric at trueblade.com (Eric Smith)
Date: Fri, 27 Mar 2009 17:28:38 -0500
Subject: [Python-Dev] Version strings
In-Reply-To: <49CD4A1A.6050008@v.loewis.de>
References: <49CD4A1A.6050008@v.loewis.de>
Message-ID: <49CD5316.70608@trueblade.com>

Martin v. L?wis wrote:
> I just collected the version strings of versions that
> got released to PyPI, and put up the list on
> 
> http://www.dcl.hpi.uni-potsdam.de/home/loewis/versions

That's excellent, thank you.

The following chunk of text is present. I don't really care, except that 
it might mean a bug in your extraction routine. Or maybe it's just a 
very wacky version string!

----------------
Sysv_ipc gives Python programs access to System V semaphores, shared memory
  and message queues. Most (all?) Unixes (including OS X) support System 
V IPC.
  Windows+Cygwin 1.7 might also work.

  Sample code is included.

  This extension is released under the GPL.

  You might also be interested in the similar POSIX IPC module at:
  http://semanchuk.com/philip/posix_ipc/
  T-0.3.4 (BitTornado)
  Trunk
----------------

From skippy.hammond at gmail.com  Sat Mar 28 00:10:13 2009
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Sat, 28 Mar 2009 10:10:13 +1100
Subject: [Python-Dev] splitting out bdist_* (was: interminable
 'setuptools' thread)
In-Reply-To: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
References: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
Message-ID: <49CD5CD5.30205@gmail.com>

On 28/03/2009 7:49 AM, glyph at divmod.com wrote:
> Perhaps bdist_wininst/_msi could be donated to the
> py2exe project if they would be willing to maintain it, and the new
> project for _deb and _rpm could be called "py2packman" or something.

As mentioned, it isn't really a natural fit - but regardless, py2exe is 
struggling to maintain itself at the moment...

Cheers,

Mark

From solipsis at pitrou.net  Sat Mar 28 00:22:19 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 27 Mar 2009 23:22:19 +0000 (UTC)
Subject: [Python-Dev]
	=?utf-8?q?splitting_out_bdist=5F*_=28was=3A_intermin?=
	=?utf-8?q?able_=27setuptools=27_thread=29?=
References: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
	<49CD5CD5.30205@gmail.com>
Message-ID: <loom.20090327T232148-586@post.gmane.org>

Mark Hammond <skippy.hammond <at> gmail.com> writes:
> 
> As mentioned, it isn't really a natural fit - but regardless, py2exe is 
> struggling to maintain itself at the moment...

It is the first auto-maintained package I have ever heard of :-)

Regards

Antoine.



From tseaver at palladion.com  Sat Mar 28 01:18:02 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 27 Mar 2009 20:18:02 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <18893.10239.228276.666597@montanaro.dyndns.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>	<49CBCD01.80500@gmail.com>	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	<20090326213047.6A8413A4097@sparrow.telecommunity.com>	<49CC57FC.30907@trueblade.com>	<20090327170202.4E5D03A40AF@sparrow.telecommunity.com>	<49CD17F8.1020501@egenix.com>
	<18893.10239.228276.666597@montanaro.dyndns.org>
Message-ID: <gqjqbr$slc$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

skip at pobox.com wrote:
>     mal> Zip files are great for shipping packages to the end-user, but
>     mal> there's no need to keep them zipped once they get there.
> 
> I thought one of the arguments for zip files was a performance increase
> (reduced stat(2) calls, etc).  I may misremember though.

Empirical evidence suggests that the win for zipfiles is only for a
*very* small set of them (basically, the whole app and all libraries in
a single zipfile).  The two lines cross pretty fast after N == 1.



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJzWy6+gerLs4ltQ4RAjYUAKCc/yiv5noitSwWavsH3voK9veYnwCfe9qn
16tnRWGZjMazB2fP5G3Q2N4=
=LUFx
-----END PGP SIGNATURE-----


From greg.ewing at canterbury.ac.nz  Sat Mar 28 01:53:56 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 28 Mar 2009 12:53:56 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903271209t1af35fd9re95f475eb5e81b24@mail.gmail.com>
References: <49C48D03.7030709@gmail.com>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<49CC69D3.4030306@canterbury.ac.nz>
	<ca471dc20903271209t1af35fd9re95f475eb5e81b24@mail.gmail.com>
Message-ID: <49CD7524.30403@canterbury.ac.nz>

Guido van Rossum wrote:

> Perhaps the crux is that *if* you accidentally use "return <value>" in
> a vanilla generator expecting the value to show up somewhere, you are
> probably enough of a newbie that debugging this will be quite hard.
> I'd like not to have such a newbie trap lying around.

Okay, so would you be happy if the for-loop were to
raise an exception if it gets a StopIteration whose
value is not None?

-- 
Greg

From benjamin at python.org  Sat Mar 28 02:09:44 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 27 Mar 2009 20:09:44 -0500
Subject: [Python-Dev] Partial 2to3?
In-Reply-To: <43aa6ff70903271436h15400426h309051ef0d4e2458@mail.gmail.com>
References: <20090327213242.00422EB016D@montanaro.dyndns.org>
	<43aa6ff70903271436h15400426h309051ef0d4e2458@mail.gmail.com>
Message-ID: <1afaf6160903271809h1b1d224di907b5cd40f840c4f@mail.gmail.com>

2009/3/27 Collin Winter <collinw at gmail.com>:
> 2009/3/27 ?<skip at pobox.com>:
>> Following up on yesterday's conversation about 2to3 and 3to2, I wonder if
>> it's easily possible to run 2to3 with a specific small subset of its fixers?
>> For example, people not wanting to make the 2->3 leap yet might still be
>> interersted in the exception handling changes ("except Foo as exc")?
>
> Sure, that's easily possible: run 2to3 -f
> some_fixer,other_fixer,this_fixer,that_fixer. You can get a full list
> of fixers using the --list-fixes option.

In addition you can use the -x option to not run a set of fixers.



-- 
Regards,
Benjamin

From pje at telecommunity.com  Sat Mar 28 02:22:16 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 27 Mar 2009 21:22:16 -0400
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <49CD43AA.1080805@egenix.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<49CC57FC.30907@trueblade.com>
	<20090327170202.4E5D03A40AF@sparrow.telecommunity.com>
	<49CD17F8.1020501@egenix.com>
	<18893.10239.228276.666597@montanaro.dyndns.org>
	<49CD43AA.1080805@egenix.com>
Message-ID: <20090328011953.A585B3A4097@sparrow.telecommunity.com>

At 10:22 PM 3/27/2009 +0100, M.-A. Lemburg wrote:
>Perhaps someone should start working on a tool called "FryingPan" to
>create "Omelettes", ie. all eggs squashed into a single ZIP file... ;-)

They're called baskets actually.  ;-)  There's no tool to do it, but 
pkg_resources does support multiple eggs being contained in the same zipfile.


From pje at telecommunity.com  Sat Mar 28 02:25:29 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 27 Mar 2009 21:25:29 -0400
Subject: [Python-Dev] My summit notes (packaging)
Message-ID: <20090328012305.1B7043A4097@sparrow.telecommunity.com>

At 03:06 PM 3/27/2009 -0500, Tarek Ziad? wrote:
>  They both aim at the
>   same goal besides a few differences, and they both rely
>   on a new metadata introduced by setuptools, wich is.
>   "install_requires". This new metadata extends the metadata.
>   described in PEP 314 but is slightly different from.
>   what is descibred in the Draft PEP 345  ("Requires").
>..
>   PEP 345 introduces "Requires" and "Provides" wich are
>   are implemented in Distutils and PyP, but are not
>   widely used. 40 out of +4000 if I remember correctly. Martin will
>   correct me here if I am wrong.

FYI, The reason setuptools uses a different way of specifying 
requirements is that the PEP-proposed way could not be used without 
some kind of indexed repository of packages -- and PyPI did not index 
"provides" at the time.  Also, the PEP-proposed versioning scheme was 
not compatible with the versioning schemes actually used in the field 
at the time.

These conditions could be considered to have changed now, or be 
changeable given enough will and volunteer effort.  Since setuptools 
was only a 1.5-person effort back in the day (i.e., me plus 
occasional contribs from Ian Bicking, Bob Ippolito, and Jim Fulton), 
and backward compatibility was a critical requirement to get 
adoption, using RPM-style provides/requires was not an option at that time.


>     - there should be an API to get metadata for a package without
>     actually executing any of the package's installation script.

Please note that where dependencies are concerned, practical 
considerations require that, at least for currently-released versions 
of Python, a package's dependencies may well be Python-version 
specific.  A library that targets Python 2.4 and 2.5 and uses 
wsgiref, sqlite, ctypes, or ElementTree, for example, may have 
different dependencies depending on the version it is being installed in.

Of course, if future Python versions declare metadata for what they 
include, this could be resolved.  But for existing versions it's a 
bit of a sticky issue.

Also, it's quite likely that platform-specific dependencies may exist 
as well.  It might be possible to accommodate these things with a 
sufficiently flexible format, but currently, the only way to handle 
them with distutils/setuptools is in the setup script.


From kevin at bud.ca  Sat Mar 28 02:42:39 2009
From: kevin at bud.ca (Kevin Teague)
Date: Fri, 27 Mar 2009 18:42:39 -0700
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <20090328011953.A585B3A4097@sparrow.telecommunity.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903260941j6c9113f1l360b17ab103e8904@mail.gmail.com>
	<49CBCD01.80500@gmail.com>
	<ca471dc20903261154t4a48e8dcw1254f3137bfa1913@mail.gmail.com>
	<10F2D511-D09D-4200-A587-CE35F05E0F77@python.org>
	<ca471dc20903261231x74d32f39wc06013f666b9baa7@mail.gmail.com>
	<487B554B-6FDE-4AF6-8700-E44749A5676E@python.org>
	<24ea26600903261242p2ebed8dbp8c53d119453d809a@mail.gmail.com>
	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>
	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<49CC57FC.30907@trueblade.com>
	<20090327170202.4E5D03A40AF@sparrow.telecommunity.com>
	<49CD17F8.1020501@egenix.com>
	<18893.10239.228276.666597@montanaro.dyndns.org>
	<49CD43AA.1080805@egenix.com>
	<20090328011953.A585B3A4097@sparrow.telecommunity.com>
Message-ID: <4B8805E2-7EB1-4829-A132-DEE52074C644@bud.ca>


On Mar 27, 2009, at 6:22 PM, P.J. Eby wrote:

> At 10:22 PM 3/27/2009 +0100, M.-A. Lemburg wrote:
>> Perhaps someone should start working on a tool called "FryingPan" to
>> create "Omelettes", ie. all eggs squashed into a single ZIP  
>> file... ;-)
>
> They're called baskets actually.  ;-)  There's no tool to do it, but  
> pkg_resources does support multiple eggs being contained in the same  
> zipfile.
>

And omelettes are when you take a collection of unzipped eggs and  
symlink into them to create a "flat" view of the files those eggs  
provide.

(or at least that's the term used in Plone land after they switched to  
eggs and wrote the collective.recipe.omelette recipe for Buildout to  
make creating these symlinks easy)

http://pypi.python.org/pypi/collective.recipe.omelette

There is also a recipe for use with Buildout for gathering a  
collection of eggs and putting them all in a basket called  
z3c.recipe.eggbasket. Although that's a little different than having a  
basket as a zip file with multiple eggs in an installed location,  
since the z3c.recipe.eggbasket is for sending a bunch of eggs over the  
internet all in one go, with the idea that that file is uncompressed  
once the eggs are installed ... because you need to take the eggs out  
of the basket before you bake an omelette  :P

http://pypi.python.org/pypi/z3c.recipe.eggbasket


From pje at telecommunity.com  Sat Mar 28 02:45:45 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Fri, 27 Mar 2009 21:45:45 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CD7524.30403@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<49CC69D3.4030306@canterbury.ac.nz>
	<ca471dc20903271209t1af35fd9re95f475eb5e81b24@mail.gmail.com>
	<49CD7524.30403@canterbury.ac.nz>
Message-ID: <20090328014323.37D323A4097@sparrow.telecommunity.com>

At 12:53 PM 3/28/2009 +1200, Greg Ewing wrote:
>Guido van Rossum wrote:
>
>>Perhaps the crux is that *if* you accidentally use "return <value>" in
>>a vanilla generator expecting the value to show up somewhere, you are
>>probably enough of a newbie that debugging this will be quite hard.
>>I'd like not to have such a newbie trap lying around.
>
>Okay, so would you be happy if the for-loop were to
>raise an exception if it gets a StopIteration whose
>value is not None?

Wouldn't it have to be more than just for-loops?  What about list(), 
map(), ...?

It seems a lot simpler to just make it use a different exception, as 
nothing else has to change for that to work correctly; the new 
construct can just catch it, and everywhere else it's an error.


From guido at python.org  Sat Mar 28 03:21:09 2009
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Mar 2009 21:21:09 -0500
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090328014323.37D323A4097@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<49CC69D3.4030306@canterbury.ac.nz>
	<ca471dc20903271209t1af35fd9re95f475eb5e81b24@mail.gmail.com>
	<49CD7524.30403@canterbury.ac.nz>
	<20090328014323.37D323A4097@sparrow.telecommunity.com>
Message-ID: <ca471dc20903271921k1774ec16sba8d2d3ca18d50a@mail.gmail.com>

On Fri, Mar 27, 2009 at 8:45 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 12:53 PM 3/28/2009 +1200, Greg Ewing wrote:
>> Guido van Rossum wrote:
>>> Perhaps the crux is that *if* you accidentally use "return <value>" in
>>> a vanilla generator expecting the value to show up somewhere, you are
>>> probably enough of a newbie that debugging this will be quite hard.
>>> I'd like not to have such a newbie trap lying around.
>>
>> Okay, so would you be happy if the for-loop were to
>> raise an exception if it gets a StopIteration whose
>> value is not None?
>
> Wouldn't it have to be more than just for-loops? ?What about list(), map(),
> ...?
>
> It seems a lot simpler to just make it use a different exception, as nothing
> else has to change for that to work correctly; the new construct can just
> catch it, and everywhere else it's an error.

Exactly. Making StopIteration() with a non-None value invalid is not
backwards compatible.

The new exception could either be a designated (built-in) subclass of
StopIteration, or not; I haven't thought enough about which choice
would be more useful or easier. I think in either case a check in
PyIter_Next() would cover most cases -- though there are probable a
few other places in the interpreter where tp_next is called that
should also have this check added.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From kevin at bud.ca  Sat Mar 28 03:54:40 2009
From: kevin at bud.ca (Kevin Teague)
Date: Fri, 27 Mar 2009 19:54:40 -0700
Subject: [Python-Dev] My summit notes (packaging)
In-Reply-To: <20090328012305.1B7043A4097@sparrow.telecommunity.com>
References: <20090328012305.1B7043A4097@sparrow.telecommunity.com>
Message-ID: <E9C33337-7C4F-4C82-9F78-DC8ADC026E1F@bud.ca>


On Mar 27, 2009, at 6:25 PM, P.J. Eby wrote:

> At 03:06 PM 3/27/2009 -0500, Tarek Ziad? wrote:
>> They both aim at the
>>  same goal besides a few differences, and they both rely
>>  on a new metadata introduced by setuptools, wich is.
>>  "install_requires". This new metadata extends the metadata.
>>  described in PEP 314 but is slightly different from.
>>  what is descibred in the Draft PEP 345  ("Requires").
>> ..
>>  PEP 345 introduces "Requires" and "Provides" wich are
>>  are implemented in Distutils and PyP, but are not
>>  widely used. 40 out of +4000 if I remember correctly. Martin will
>>  correct me here if I am wrong.
>
> FYI, The reason setuptools uses a different way of specifying  
> requirements is that the PEP-proposed way could not be used without  
> some kind of indexed repository of packages -- and PyPI did not  
> index "provides" at the time.  Also, the PEP-proposed versioning  
> scheme was not compatible with the versioning schemes actually used  
> in the field at the time.
>
> These conditions could be considered to have changed now, or be  
> changeable given enough will and volunteer effort.  Since setuptools  
> was only a 1.5-person effort back in the day (i.e., me plus  
> occasional contribs from Ian Bicking, Bob Ippolito, and Jim Fulton),  
> and backward compatibility was a critical requirement to get  
> adoption, using RPM-style provides/requires was not an option at  
> that time.
>


Tarek, was there any further discussion on "Requires" vs  
"install_requires" or any decisions made on what to do about this?

(I've got a +1 ready for including 'install_requires' in the standard  
package metadata and marking 'Requires' as deprecated if that was put  
forth :P)

 From what I understand of PEP 345's "Requires" and "Provides" fields  
is that it's not possible to use them unambigously? For example, if  
I'm working on a Python project and want to depend upon a project that  
provides a 'utils' module, there could be several packages on PyPI  
that declare they provide a 'utils' module. Which project would get  
picked then?

It's also a lot more succinct to just spell out the project name in  
the "install_requires" field than to have to list all of the modules  
and packages you expect to use in the project you are depending upon.  
For example, this is pretty reasonable:

install_requires: Django

And this is a disaster:

Requires: django
Requires: django.conf
Requires: django.contrib
Requires: django.contrib.admin
Requires: django.contrib.admindocs
Requires: django.contrib.auth
Requires: django.contrib.comments
Requires: django.contrib.contenttypes
Requires: django.contrib.csrf
Requires: django.contrib.databrowse
Requires: django.contrib.flatpages
Requires: django.contrib.formtools
Requires: django.contrib.gis
Requires: django.contrib.humanize
Requires: django.contrib.localflavor
Requires: django.contrib.markup
Requires: django.contrib.redirects
Requires: django.contrib.sessions
Requires: django.contrib.sitemaps
Requires: django.contrib.sites
Requires: django.contrib.syndication
Requires: django.contrib.webdesign
Requires: django.core
Requires: django.db
Requires: django.dispatch
Requires: django.forms
Requires: django.http
Requires: django.middleware
Requires: django.shortcuts
Requires: django.template
Requires: django.templatetags
Requires: django.test
Requires: django.utils
Requires: django.views

Not to mention that if you were upgrading to a newer version of Django  
you'd could no longer just say '1.0' or '1.1' in 'install_requires'  
but would instead have to fiddle with the Requires list to match newly  
introduced modules and packages. I guess one is supposed to just list  
"Requires: django" since that would resolve to the Django project  
which, and it's implied that it would then provide everything else? Or  
perhaps one is supposed to comb through their code and just list the  
particular bits of Django that they're importing and using? There are  
other times when a project may provide several top-level packages, and  
again it's a lot easier to just ask for the project name rather than  
list all of the top-level packages (usually this type of project is  
kept on an private package server where it's reasonable to occupy  
common top-level names such as 'utils' without worrying too much about  
naming conflicts). Additionally, consuming the "Requires" field is  
always more work, since you need to do some kind of look-up to go from  
module/package name to project name before you can fetch a  
distribution for that project.


From michele.simionato at gmail.com  Sat Mar 28 04:26:46 2009
From: michele.simionato at gmail.com (Michele Simionato)
Date: Sat, 28 Mar 2009 04:26:46 +0100
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <4222a8490903270533k104a3a2cv78b66396c1796104@mail.gmail.com>
References: <49C48D03.7030709@gmail.com> <49C966A0.6090401@canterbury.ac.nz>
	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>
	<49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<49CC5D6D.7060108@canterbury.ac.nz>
	<loom.20090327T105353-433@post.gmane.org>
	<4222a8490903270533k104a3a2cv78b66396c1796104@mail.gmail.com>
Message-ID: <4edc17eb0903272026r57e5e285n986ca3693293d6e4@mail.gmail.com>

On Fri, Mar 27, 2009 at 1:33 PM, Jesse Noller <jnoller at gmail.com> wrote:
> Antoine Pitrou:
>> As a matter of fact, the people whom this PEP is supposed to benefit haven't
>> expressed a lot of enthusiasm right now. That's why it looks so academic.
> That's because most of us who might like this have been patently
> avoiding this thread.

I have been avoiding this thread too - even if I have implemented my
own trampoline as
everybody else here - because I had nothing to say that was not said
already here.
But just to add a data point, let me say that I agree with Eby.
I am 0+ on the syntax, but please keep the hidden logic simple and
absolutely do NOT add confusion
between yield and return. Use yield Return(value) or raise SomeException(value),
as you like. The important thing for me is to have a trampoline in the
standard library, not
the syntax.

               Michele Simionato

From stephen at xemacs.org  Sat Mar 28 07:17:40 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 28 Mar 2009 15:17:40 +0900
Subject: [Python-Dev] splitting out bdist_*
In-Reply-To: <49CD4A4F.30900@trueblade.com>
References: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
	<49CD43B7.3050904@v.loewis.de> <49CD4A4F.30900@trueblade.com>
Message-ID: <878wmqxjaz.fsf@xemacs.org>

Eric Smith writes:

 > And I personally use bdist_rpm for my work, which I distribute to a farm 
 > of servers under my control. So no doubt it's used.

Sure, but use for internal distribution is very different from to
problem its being asked to solve now, isn't it?  IIUC, you're
basically using RPM as an installer for a standalone application,
where you set policy at both ends, packaging and installation.  This
may be a group of modules which may have internal interdependencies,
but very likely the environment doesn't change much.  Pretty much
anything will do in that kind of situation, and I don't think it would
matter to you if there are zero, one, or twelve such tools in stdlib,
as long as there's one you like in PyPI somewhere.

 > [That .deb tools are necessarily maintained outside of bdist]
 > proves that [external maintenance] doesn't help given the current
 > state of affairs. I suspect that if all of this additional
 > information needed to build a .deb (for example) could be included
 > as metadata in the python package (using the word "package"
 > loosely), that it would be. It would make the ultimate packager's
 > life easier, and it's no real burden for the original author.

I'm not sure what you mean by "it would be".  Are you referring to
what I believe to be true, that because Python and Python-based apps
need to integrate with the rest of the OS, it's quite burdensome for
module authors to include the necessary information, which is likely
to vary from packaging tool to packaging tool, and according to policy
even among packagers using the same tool?  Or do you mean that it
would be useful, so it would be nice if such information were
included, and that as you see it the required effort would be small?


From greg.ewing at canterbury.ac.nz  Sat Mar 28 07:36:42 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 28 Mar 2009 18:36:42 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903271921k1774ec16sba8d2d3ca18d50a@mail.gmail.com>
References: <49C48D03.7030709@gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<49CC69D3.4030306@canterbury.ac.nz>
	<ca471dc20903271209t1af35fd9re95f475eb5e81b24@mail.gmail.com>
	<49CD7524.30403@canterbury.ac.nz>
	<20090328014323.37D323A4097@sparrow.telecommunity.com>
	<ca471dc20903271921k1774ec16sba8d2d3ca18d50a@mail.gmail.com>
Message-ID: <49CDC57A.1050202@canterbury.ac.nz>

Guido van Rossum wrote:

> The new exception could either be a designated (built-in) subclass of
> StopIteration, or not;

I think it would have to not be; otherwise any existing
code that catches StopIteration would catch the new
exception as well without complaint.

Using a different exception raises another question.
Would you check whether the return value is None and
raise an ordinary StopIteration in that case? Or would
return with a value always raise the new exception?

If the latter, then 'return' and 'return None' would
no longer be equivalent in all cases, which would be
rather strange.

> I think in either case a check in
> PyIter_Next() would cover most cases

If that's acceptable, then the check might as well
be for None as the StopIteration value, and there's
no need for a new exception.

-- 
Greg

From cournape at gmail.com  Sat Mar 28 08:42:53 2009
From: cournape at gmail.com (David Cournapeau)
Date: Sat, 28 Mar 2009 16:42:53 +0900
Subject: [Python-Dev] splitting out bdist_*
In-Reply-To: <878wmqxjaz.fsf@xemacs.org>
References: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
	<49CD43B7.3050904@v.loewis.de> <49CD4A4F.30900@trueblade.com>
	<878wmqxjaz.fsf@xemacs.org>
Message-ID: <5b8d13220903280042v80394e8xf2b55515f2c533ac@mail.gmail.com>

2009/3/28 Stephen J. Turnbull <stephen at xemacs.org>:

>
> Sure, but use for internal distribution is very different from to
> problem its being asked to solve now, isn't it? ?IIUC, you're
> basically using RPM as an installer for a standalone application,
> where you set policy at both ends, packaging and installation. ?This
> may be a group of modules which may have internal interdependencies,
> but very likely the environment doesn't change much. ?Pretty much
> anything will do in that kind of situation, and I don't think it would
> matter to you if there are zero, one, or twelve such tools in stdlib,
> as long as there's one you like in PyPI somewhere.

I myself would never use such a tool, unless sanctioned by my OS
vendor, because I would not trust it not to break my system. But I
think bdist_rpm and bdist_deb solve a real deficiency: no
uninstallation feature. Thinking of it, that's exactly why I like
bdist_wininst so much when I am on windows (and because the
consequences of a bad installer from bdist_wininst seem minimal on
windows, seems everything is one directory).

David

From ncoghlan at gmail.com  Sat Mar 28 10:34:12 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 28 Mar 2009 19:34:12 +1000
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <4edc17eb0903272026r57e5e285n986ca3693293d6e4@mail.gmail.com>
References: <49C48D03.7030709@gmail.com>
	<49C966A0.6090401@canterbury.ac.nz>	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>	<49C9C947.3080608@canterbury.ac.nz>	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>	<loom.20090326T102716-650@post.gmane.org>	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>	<49CC5D6D.7060108@canterbury.ac.nz>	<loom.20090327T105353-433@post.gmane.org>	<4222a8490903270533k104a3a2cv78b66396c1796104@mail.gmail.com>
	<4edc17eb0903272026r57e5e285n986ca3693293d6e4@mail.gmail.com>
Message-ID: <49CDEF14.6030302@gmail.com>

Michele Simionato wrote:
> On Fri, Mar 27, 2009 at 1:33 PM, Jesse Noller <jnoller at gmail.com> wrote:
>> Antoine Pitrou:
>>> As a matter of fact, the people whom this PEP is supposed to benefit haven't
>>> expressed a lot of enthusiasm right now. That's why it looks so academic.
>> That's because most of us who might like this have been patently
>> avoiding this thread.
> 
> I have been avoiding this thread too - even if I have implemented my
> own trampoline as
> everybody else here - because I had nothing to say that was not said
> already here.
> But just to add a data point, let me say that I agree with Eby.
> I am 0+ on the syntax, but please keep the hidden logic simple and
> absolutely do NOT add confusion
> between yield and return. Use yield Return(value) or raise SomeException(value),
> as you like.

I still think raise is out due to the fact that it would trigger
subsequent except clauses. Guido has (sensibly) ruled out raising
StopIteration and complaining if it has value in old code, since there
is too much code which cases StopIteration *without* performing such a
check.

If those two points are accepted as valid, then that leaves the two
options as being:

1. Add a new GeneratorReturn exception that will escape from existing
code that only traps StopIteration. The only real downside of this is
that either "return" and "return None" will mean different things in
generators (unlike functions) or else "return None" will need to be
special cased to raise StopIteration in the calling code rather than
raising GeneratorReturn(None). The latter approach is probably
preferable if this option is chosen - any code for dealing with
"generators as coroutines" is going to have to deal with the possibility
of bare returns and falling off the end of the function anyway, so the
special case really wouldn't be that special.

2. In addition to the "yield from" syntax for delegating to a
subgenerator, also add new syntax for returning values from
subgenerators so that the basic "return X" can continue to trigger
SyntaxError.

Since option 2 would most likely lead to a bikeshed discussion of epic
proportions, I'm currently a fan of option 1 ;)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From mrts.pydev at gmail.com  Sat Mar 28 11:06:09 2009
From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=)
Date: Sat, 28 Mar 2009 12:06:09 +0200
Subject: [Python-Dev] version compare function into main lib
In-Reply-To: <87zlf6sico.fsf@benfinney.id.au>
References: <d34314100903270042q60f5213bv57349cd1ab1d3190@mail.gmail.com>
	<49CCECAA.7010406@v.loewis.de> <49CCF85C.8040902@trueblade.com>
	<49CD2DDF.3010704@v.loewis.de> <87zlf6sico.fsf@benfinney.id.au>
Message-ID: <ad1f81530903280306k61f0c48y6c0c6c37a122de7c@mail.gmail.com>

On Sat, Mar 28, 2009 at 12:37 AM, Ben Finney <
bignose+hates-spam at benfinney.id.au <bignose%2Bhates-spam at benfinney.id.au>>wrote:

> "Martin v. L?wis" <martin at v.loewis.de> writes:
>
> > I don't mind the setuptools implementation being used as a basis
> > (assuming it gets contributed), but *independently* I think a
> > specfication is needed what version strings it actually understands.
> > Such specification must precede the actual implementation (in
> > distutils).
>
> Yes, please. The comparison of version strings needs to be easily done
> by non-Python programs (e.g. tools for packaging Python
> distributions), so a specification that can be implemented in other
> languages or environments is a must.


There's a specification in
http://wiki.python.org/moin/ApplicationInfrastructure , see "Version API"
below (at least, it's a start).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090328/c10a1139/attachment-0001.htm>

From guido at python.org  Sat Mar 28 12:47:30 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 28 Mar 2009 06:47:30 -0500
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CDC57A.1050202@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<49CC69D3.4030306@canterbury.ac.nz>
	<ca471dc20903271209t1af35fd9re95f475eb5e81b24@mail.gmail.com>
	<49CD7524.30403@canterbury.ac.nz>
	<20090328014323.37D323A4097@sparrow.telecommunity.com>
	<ca471dc20903271921k1774ec16sba8d2d3ca18d50a@mail.gmail.com>
	<49CDC57A.1050202@canterbury.ac.nz>
Message-ID: <ca471dc20903280447o563848b4hdc74b2ac06c9d130@mail.gmail.com>

On Sat, Mar 28, 2009 at 1:36 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>
>> The new exception could either be a designated (built-in) subclass of
>> StopIteration, or not;
>
> I think it would have to not be; otherwise any existing
> code that catches StopIteration would catch the new
> exception as well without complaint.

OK.

> Using a different exception raises another question.
> Would you check whether the return value is None and
> raise an ordinary StopIteration in that case? Or would
> return with a value always raise the new exception?
I'm not sure it matters much, but let's sat the latter.

> If the latter, then 'return' and 'return None' would
> no longer be equivalent in all cases, which would be
> rather strange.

They already aren't in generators, 'return' is allowed but 'return None' isn't.

>> I think in either case a check in
>> PyIter_Next() would cover most cases
>
> If that's acceptable, then the check might as well
> be for None as the StopIteration value, and there's
> no need for a new exception.

I don't understand this.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Mar 28 12:52:52 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 28 Mar 2009 06:52:52 -0500
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CDEF14.6030302@gmail.com>
References: <49C48D03.7030709@gmail.com> <49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<49CC5D6D.7060108@canterbury.ac.nz>
	<loom.20090327T105353-433@post.gmane.org>
	<4222a8490903270533k104a3a2cv78b66396c1796104@mail.gmail.com>
	<4edc17eb0903272026r57e5e285n986ca3693293d6e4@mail.gmail.com>
	<49CDEF14.6030302@gmail.com>
Message-ID: <ca471dc20903280452p5e41367m233611adefabeddb@mail.gmail.com>

On Sat, Mar 28, 2009 at 4:34 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> I still think raise is out due to the fact that it would trigger
> subsequent except clauses. Guido has (sensibly) ruled out raising
> StopIteration and complaining if it has value in old code, since there
> is too much code which cases StopIteration *without* performing such a
> check.
>
> If those two points are accepted as valid, then that leaves the two
> options as being:
>
> 1. Add a new GeneratorReturn exception that will escape from existing
> code that only traps StopIteration. The only real downside of this is
> that either "return" and "return None" will mean different things in
> generators (unlike functions) or else "return None" will need to be
> special cased to raise StopIteration in the calling code rather than
> raising GeneratorReturn(None). The latter approach is probably
> preferable if this option is chosen - any code for dealing with
> "generators as coroutines" is going to have to deal with the possibility
> of bare returns and falling off the end of the function anyway, so the
> special case really wouldn't be that special.

It seems so indeed.

> 2. In addition to the "yield from" syntax for delegating to a
> subgenerator, also add new syntax for returning values from
> subgenerators so that the basic "return X" can continue to trigger
> SyntaxError.
>
> Since option 2 would most likely lead to a bikeshed discussion of epic
> proportions, I'm currently a fan of option 1 ;)

Me too. It also seems option 2 doesn't help us decide what it should
do: I still think that raising StopIteration(value) would be
misleading to vanilla users of the generators.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From eric at trueblade.com  Sat Mar 28 14:33:26 2009
From: eric at trueblade.com (Eric Smith)
Date: Sat, 28 Mar 2009 08:33:26 -0500
Subject: [Python-Dev] splitting out bdist_*
In-Reply-To: <878wmqxjaz.fsf@xemacs.org>
References: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>	<49CD43B7.3050904@v.loewis.de>
	<49CD4A4F.30900@trueblade.com> <878wmqxjaz.fsf@xemacs.org>
Message-ID: <49CE2726.3050307@trueblade.com>

Stephen J. Turnbull wrote:
> Eric Smith writes:
> 
>  > And I personally use bdist_rpm for my work, which I distribute to a farm 
>  > of servers under my control. So no doubt it's used.
> 
> Sure, but use for internal distribution is very different from to
> problem its being asked to solve now, isn't it?  IIUC, you're
> basically using RPM as an installer for a standalone application,
> where you set policy at both ends, packaging and installation.  This
> may be a group of modules which may have internal interdependencies,
> but very likely the environment doesn't change much.  Pretty much
> anything will do in that kind of situation, and I don't think it would
> matter to you if there are zero, one, or twelve such tools in stdlib,
> as long as there's one you like in PyPI somewhere.

I was just pointing out that bdist_rpm has users, and it's not likely to 
be abandoned. It's certainly true that different users have different 
use cases for it.

It's this lack of understanding of all the use cases that most concerns 
me about this overall effort. How can we know if we succeeded if we 
don't all agree on the state we're trying to get to?

To be concrete, I think distutils should support (among other things):
- entry points for plugins
- entry points as used for producing console and windowed "scripts"
- dependency declarations for other python packages
- dependency declarations for non-python packages

But maybe these goals aren't shared by others, and don't fall into 
anyone else's "make distutils better" concept.

PJE pointed out "A library that targets Python 2.4 and 2.5 and uses 
wsgiref, sqlite, ctypes, or ElementTree, for example, may have different 
dependencies depending on the version it is being installed in." Is that 
something we want to support?

>  > [That .deb tools are necessarily maintained outside of bdist]
>  > proves that [external maintenance] doesn't help given the current
>  > state of affairs. I suspect that if all of this additional
>  > information needed to build a .deb (for example) could be included
>  > as metadata in the python package (using the word "package"
>  > loosely), that it would be. It would make the ultimate packager's
>  > life easier, and it's no real burden for the original author.
> 
> I'm not sure what you mean by "it would be".  Are you referring to
> what I believe to be true, that because Python and Python-based apps
> need to integrate with the rest of the OS, it's quite burdensome for
> module authors to include the necessary information, which is likely
> to vary from packaging tool to packaging tool, and according to policy
> even among packagers using the same tool?  Or do you mean that it
> would be useful, so it would be nice if such information were
> included, and that as you see it the required effort would be small?

I don't see how they differ. It's definitely true that packagers using 
the same tool might want to produce different package layouts and no 
doubt other differences. I don't see why it wouldn't be easy to have 
these differences driven by different policies acting on the same 
metadata, or small amounts of custom (per-packager) metadata.

From pje at telecommunity.com  Sat Mar 28 14:39:02 2009
From: pje at telecommunity.com (P.J. Eby)
Date: Sat, 28 Mar 2009 09:39:02 -0400
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903280452p5e41367m233611adefabeddb@mail.gmail.com
 >
References: <49C48D03.7030709@gmail.com> <49C9C947.3080608@canterbury.ac.nz>
	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>
	<loom.20090326T102716-650@post.gmane.org>
	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>
	<49CC5D6D.7060108@canterbury.ac.nz>
	<loom.20090327T105353-433@post.gmane.org>
	<4222a8490903270533k104a3a2cv78b66396c1796104@mail.gmail.com>
	<4edc17eb0903272026r57e5e285n986ca3693293d6e4@mail.gmail.com>
	<49CDEF14.6030302@gmail.com>
	<ca471dc20903280452p5e41367m233611adefabeddb@mail.gmail.com>
Message-ID: <20090328133638.193553A4097@sparrow.telecommunity.com>

At 06:52 AM 3/28/2009 -0500, Guido van Rossum wrote:
> > 2. In addition to the "yield from" syntax for delegating to a
> > subgenerator, also add new syntax for returning values from
> > subgenerators so that the basic "return X" can continue to trigger
> > SyntaxError.
> >
> > Since option 2 would most likely lead to a bikeshed discussion of epic
> > proportions, I'm currently a fan of option 1 ;)
>
>Me too. It also seems option 2 doesn't help us decide what it should
>do: I still think that raising StopIteration(value) would be
>misleading to vanilla users of the generators.

Agreed.  (I still think new syntax is a plus, though, as it helps to 
clearly signal both the intent to make the generator a "subtask" and 
the need to call it with yield-from.  Readability counts.)


From ziade.tarek at gmail.com  Sat Mar 28 15:24:37 2009
From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 28 Mar 2009 09:24:37 -0500
Subject: [Python-Dev] My summit notes (packaging)
In-Reply-To: <E9C33337-7C4F-4C82-9F78-DC8ADC026E1F@bud.ca>
References: <20090328012305.1B7043A4097@sparrow.telecommunity.com>
	<E9C33337-7C4F-4C82-9F78-DC8ADC026E1F@bud.ca>
Message-ID: <94bdd2610903280724l43910a9am807113e7b19c1ca8@mail.gmail.com>

On Fri, Mar 27, 2009 at 9:54 PM, Kevin Teague <kevin at bud.ca> wrote:

>
>
> Tarek, was there any further discussion on "Requires" vs "install_requires"
> or any decisions made on what to do about this?
>
> (I've got a +1 ready for including 'install_requires' in the standard
> package metadata and marking 'Requires' as deprecated if that was put forth
> :P)

Yes that is what we were thinking of doing. (deprectating Requires and
Provides and and install_requires)

We are working on it. We'll try to organize our work in the wiki in
the comng days, so people can participate.

Regards
Tarek

From stephen at xemacs.org  Sat Mar 28 16:10:15 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 29 Mar 2009 00:10:15 +0900
Subject: [Python-Dev] splitting out bdist_*
In-Reply-To: <49CE2726.3050307@trueblade.com>
References: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
	<49CD43B7.3050904@v.loewis.de> <49CD4A4F.30900@trueblade.com>
	<878wmqxjaz.fsf@xemacs.org> <49CE2726.3050307@trueblade.com>
Message-ID: <87tz5dwunc.fsf@xemacs.org>

Eric Smith writes:

 > I was just pointing out that bdist_rpm has users, and it's not likely to 
 > be abandoned.

OK, I see.  I don't think there's a reason to remove useful
functionality from the stdlib, unless it's clearly superseded by a
similar module.

 > I don't see how they differ. It's definitely true that packagers using 
 > the same tool might want to produce different package layouts and no 
 > doubt other differences. I don't see why it wouldn't be easy to have 
 > these differences driven by different policies acting on the same 
 > metadata, or small amounts of custom (per-packager) metadata.

My experience in XEmacs has been that Debian, Fedora Core, Gentoo,
SuSE, and Mandriva have rather different expressions of things like
dependencies, and they tend to have different ideas of how forceful
they should be with any given supporting package (when the package
system supports different strengths of dependency).

Debian, for example, supports "predepends" (you can't even install the
dependent unless the prerequisite is already installed), "depends"
(installing the dependent will also install the prerequisite unless
the admin is quite forceful about saying no), "recommends" (it's easy
to say no), and "suggests" (you only get a message saying "Things go
better with Coke" and a suggestion that you may want to install Coke
because you installed Things).  In other systems there's either a
dependency, or there isn't.  Gentoo has "use flags" which are about as
flexible as Debian dependencies, but their taste in recommendations is
quite different.

I really don't see how that kind of thing can be easily supported by a
Python module maintainer, unless they're also the downstream packager.


From andrew at acooke.org  Sat Mar 28 15:41:54 2009
From: andrew at acooke.org (andrew cooke)
Date: Sat, 28 Mar 2009 10:41:54 -0400 (CLT)
Subject: [Python-Dev] Unladen-Swallow: A faster python
In-Reply-To: <49CE2E2E.8000801@gmail.com>
References: <ab2c8071-1d3f-4fcb-9669-a52114997aa9@h28g2000yqd.googlegroups.com>
	<lhgrs4pqklhe43e00jm8g0vhfcm7j2cnfp@4ax.com>
	<bec05a9536fe5b1d769c5374ae05739b.squirrel@localhost>
	<49CE2E2E.8000801@gmail.com>
Message-ID: <20f41fcdc7ab12f58b19487ef847569c.squirrel@acooke.dyndns.org>

Mark Hammond wrote:
> On 28/03/2009 9:50 PM, andrew cooke wrote:
>> Tim Roberts wrote:
>>> [...]  IronPython has certainly shown that Python can be successfully
>>> implemented in a JIT compiled VM in a performant way, but it has issues
>>> running C extension modules.
>>>
>>> I'll be curious to see where this project goes.
>>
>> given the comments on python-dev i wonder if this is the first
>> indication
>> that python is going to split into separate implementations for windows
>> and unix (via .net and llvm, respectively)?
>
> What comments are they?  There is no indication that unladen-swallow  is
> fundamentally broken for Windows, just temporarily broken due to the
> lack of windows developers/contributors...

the comments you are referring to - that windows is not a priority, that
they currently have no testing on windows machines, etc.  i quote, for
example: "None of the three of us have Windows machines, nor do we
particularly want to acquire them :), and Windows support isn't going to
be a big priority."

> Saying-no-to-fud ly,

which part of "i wonder" don't you understand?  i'm not saying it is true,
i'm just discussing the possibility.  i am getting a little tired of
people here acting so defensively...  i'm discussing a programming
language, not the size of your dick.

andrew



From barry at python.org  Sat Mar 28 17:08:30 2009
From: barry at python.org (Barry Warsaw)
Date: Sat, 28 Mar 2009 11:08:30 -0500
Subject: [Python-Dev] splitting out bdist_*
In-Reply-To: <87tz5dwunc.fsf@xemacs.org>
References: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
	<49CD43B7.3050904@v.loewis.de> <49CD4A4F.30900@trueblade.com>
	<878wmqxjaz.fsf@xemacs.org> <49CE2726.3050307@trueblade.com>
	<87tz5dwunc.fsf@xemacs.org>
Message-ID: <62A3FE1E-6908-4FBC-9F25-2D36AEDB93E5@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 28, 2009, at 10:10 AM, Stephen J. Turnbull wrote:

> I really don't see how that kind of thing can be easily supported by a
> Python module maintainer, unless they're also the downstream packager.

They simply can't.  As a package developer, I'd be sort of okay with  
integrating patches that help downstreams, but I wouldn't be happy  
about it.  I can't test it, and there might be conflicts between the  
demands of various downstreams.  Much more appealing is for me to  
describe what's in my package with rich enough metadata that  
downstreams don't need anything else to overlay their build rules  
according to their own needs.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSc5Lf3EjvBPtnXfVAQJoJQQAjJ4BeTk/ovhPvfJLMOSM+mcB7wz4XaX8
X9YlCnmQzPyNtbPdaaaLtkE86Sk6AC3fGO5hVjrSTANmzI02ztvNw4Lm1+HurTC5
6lghbQ1yEudZ3EgVdFu91jYBHrNPkgQtQA/oaB2/paER/8LeGqBppZiitm9plfHB
0LLHkLtylJ8=
=qT/n
-----END PGP SIGNATURE-----

From cournape at gmail.com  Sat Mar 28 17:19:41 2009
From: cournape at gmail.com (David Cournapeau)
Date: Sun, 29 Mar 2009 01:19:41 +0900
Subject: [Python-Dev] splitting out bdist_*
In-Reply-To: <87tz5dwunc.fsf@xemacs.org>
References: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
	<49CD43B7.3050904@v.loewis.de> <49CD4A4F.30900@trueblade.com>
	<878wmqxjaz.fsf@xemacs.org> <49CE2726.3050307@trueblade.com>
	<87tz5dwunc.fsf@xemacs.org>
Message-ID: <5b8d13220903280919m4c09afbfn8209572106a7cb57@mail.gmail.com>

2009/3/29 Stephen J. Turnbull <stephen at xemacs.org>:

> I really don't see how that kind of thing can be easily supported by a
> Python module maintainer, unless they're also the downstream packager.

Almost none. But in my understanding, that's not what most linux
packagers vendors ask about - they will handle the dependencies
themselves anyway, because naming conventions and the like are
different.

What is a pain right now with distutils for packagers is:
 - how to control which files are installed where
 - how to control the build (compilation flags, etc...).

Packagers generally "like" autotools packages because they can be
customized along each distribution convention. Autotools do not really
handle dependencies either, but they can be customized for vastly
different kind of deployement (one dir for everything ala gobolinux,
along the FHS for most major distributions, etc...) - and the upstream
developer doesn't need to care much about it.

cheers,

David

From jim at zope.com  Sat Mar 28 19:14:22 2009
From: jim at zope.com (Jim Fulton)
Date: Sat, 28 Mar 2009 14:14:22 -0400
Subject: [Python-Dev] My summit notes (packaging)
In-Reply-To: <20090328012305.1B7043A4097@sparrow.telecommunity.com>
References: <20090328012305.1B7043A4097@sparrow.telecommunity.com>
Message-ID: <BD5BF4C6-6503-4157-A6DB-8A72B2094F48@zope.com>


On Mar 27, 2009, at 9:25 PM, P.J. Eby wrote:

> At 03:06 PM 3/27/2009 -0500, Tarek Ziad? wrote:
>> They both aim at the
>>  same goal besides a few differences, and they both rely
>>  on a new metadata introduced by setuptools, wich is.
>>  "install_requires". This new metadata extends the metadata.
>>  described in PEP 314 but is slightly different from.
>>  what is descibred in the Draft PEP 345  ("Requires").
>> ..
>>  PEP 345 introduces "Requires" and "Provides" wich are
>>  are implemented in Distutils and PyP, but are not
>>  widely used. 40 out of +4000 if I remember correctly. Martin will
>>  correct me here if I am wrong.
>
> FYI, The reason setuptools uses a different way of specifying  
> requirements is that the PEP-proposed way could not be used without  
> some kind of indexed repository of packages -- and PyPI did not  
> index "provides" at the time.  Also, the PEP-proposed versioning  
> scheme was not compatible with the versioning schemes actually used  
> in the field at the time.


There's a deeper issue IMO.  As Kevin pointed out, distutil's Requires  
data works at the module and package level, rather than at the project  
level.  I can see some value in this, but I think working at the  
project level is a lot simpler and more practically useful.

Jim

--
Jim Fulton
Zope Corporation



From fdrake at acm.org  Sat Mar 28 20:04:58 2009
From: fdrake at acm.org (Fred Drake)
Date: Sat, 28 Mar 2009 15:04:58 -0400
Subject: [Python-Dev] splitting out bdist_*
In-Reply-To: <49CE2726.3050307@trueblade.com>
References: <20090327204953.12555.1384799699.divmod.xquotient.6636@weber.divmod.com>
	<49CD43B7.3050904@v.loewis.de> <49CD4A4F.30900@trueblade.com>
	<878wmqxjaz.fsf@xemacs.org> <49CE2726.3050307@trueblade.com>
Message-ID: <B7D9E610-72C6-4036-B678-CC2CDDFDB8C8@acm.org>

On Mar 28, 2009, at 9:33 AM, Eric Smith wrote:
> To be concrete, I think distutils should support (among other things):
> - entry points for plugins
> - entry points as used for producing console and windowed "scripts"

This strikes me as a nice-to-have, but:

1. I don't think they're two distinct features.
2. I'm not convinced these should go in distutils.

I'd rather see an API to get resources from a package, and conventions  
can be developed among tool developers on how to store that  
information in a named resource.

> - dependency declarations for other python packages

This is the most important requirement here, I think.

As part of this, I'd want to be able to say things like "PIL, with  
JPEG and PNG support compiled in."

> - dependency declarations for non-python packages

This would be very nice to have, but I suspect this is actually much  
more difficult than Python package dependencies, especially with any  
pretense at cross-platform expressions of dependencies.

> PJE pointed out "A library that targets Python 2.4 and 2.5 and uses  
> wsgiref, sqlite, ctypes, or ElementTree, for example, may have  
> different dependencies depending on the version it is being  
> installed in." Is that something we want to support?

Even simple cases present issues with regard to this.  For example, I  
work on a project that relies on the uuid module, so we declare a  
dependency on Ka-Ping Ye's uuid module (since we're using Python  
2.4).  How should we write that in a version-agnostic way if we want  
to use the standard library version of that module with newer Pythons?


   -Fred

-- 
Fred Drake   <fdrake at acm.org>


From seefeld at sympatico.ca  Sat Mar 28 20:57:43 2009
From: seefeld at sympatico.ca (Stefan Seefeld)
Date: Sat, 28 Mar 2009 15:57:43 -0400
Subject: [Python-Dev] mentoring Roundup work
Message-ID: <49CE8137.2050903@sympatico.ca>

Hello,

I have just applied to be considered as GSoC mentor with the PSF, 
notably work on Roundup. (My ID is 'stefan')

I'm a long-term Roundup user and contributor. My recent contributions 
include the XMLRPC interface, as well as many bug fixes and 
enhancements. I also spearheaded the migration of python.org from the 
sf.net bug tracker(s) to bugs.python.org.

Please let me know if there are any mailing lists or other channels I 
should connect to to follow-up.

Thanks,
       Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...


From martin at v.loewis.de  Sat Mar 28 21:48:49 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Mar 2009 15:48:49 -0500
Subject: [Python-Dev] My summit notes (packaging)
In-Reply-To: <94bdd2610903271306i9e627a9ubc3cfa68805a15f2@mail.gmail.com>
References: <94bdd2610903271306i9e627a9ubc3cfa68805a15f2@mail.gmail.com>
Message-ID: <49CE8D31.4040504@v.loewis.de>


>   PEP 345 introduces "Requires" and "Provides" wich are
>   are implemented in Distutils and PyP, but are not
>   widely used. 40 out of +4000 if I remember correctly. Martin will
>   correct me here if I am wrong.

Here are the actual numbers of packages that had a certain specifier
in atleast one of their releases:
- requires: 334
- provides: 84
- obsoletes: 18
(total number of packages in my local database snapshot: 6101)

Regards,
Martin


From greg.ewing at canterbury.ac.nz  Sat Mar 28 22:37:43 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 29 Mar 2009 09:37:43 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903280447o563848b4hdc74b2ac06c9d130@mail.gmail.com>
References: <49C48D03.7030709@gmail.com>
	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>
	<20090326211646.3392B3A4097@sparrow.telecommunity.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<49CC69D3.4030306@canterbury.ac.nz>
	<ca471dc20903271209t1af35fd9re95f475eb5e81b24@mail.gmail.com>
	<49CD7524.30403@canterbury.ac.nz>
	<20090328014323.37D323A4097@sparrow.telecommunity.com>
	<ca471dc20903271921k1774ec16sba8d2d3ca18d50a@mail.gmail.com>
	<49CDC57A.1050202@canterbury.ac.nz>
	<ca471dc20903280447o563848b4hdc74b2ac06c9d130@mail.gmail.com>
Message-ID: <49CE98A7.7050303@canterbury.ac.nz>

Guido van Rossum wrote:

>>> I think in either case a check in
>>> PyIter_Next() would cover most cases
>
>>If that's acceptable, then the check might as well
>>be for None as the StopIteration value, and there's
>>no need for a new exception.
> 
> I don't understand this.

Maybe I misunderstood what you were saying. What
check were you suggesting to perform in PyIter_Next?

-- 
Greg

From skippy.hammond at gmail.com  Sat Mar 28 22:40:54 2009
From: skippy.hammond at gmail.com (Mark Hammond)
Date: Sun, 29 Mar 2009 08:40:54 +1100
Subject: [Python-Dev] Unladen-Swallow: A faster python
In-Reply-To: <20f41fcdc7ab12f58b19487ef847569c.squirrel@acooke.dyndns.org>
References: <ab2c8071-1d3f-4fcb-9669-a52114997aa9@h28g2000yqd.googlegroups.com>	<lhgrs4pqklhe43e00jm8g0vhfcm7j2cnfp@4ax.com>	<bec05a9536fe5b1d769c5374ae05739b.squirrel@localhost>	<49CE2E2E.8000801@gmail.com>
	<20f41fcdc7ab12f58b19487ef847569c.squirrel@acooke.dyndns.org>
Message-ID: <49CE9966.5030803@gmail.com>

On 29/03/2009 1:41 AM, andrew cooke wrote:
> Mark Hammond wrote:
>> On 28/03/2009 9:50 PM, andrew cooke wrote:
>>> Tim Roberts wrote:
>>>> [...]  IronPython has certainly shown that Python can be successfully
>>>> implemented in a JIT compiled VM in a performant way, but it has issues
>>>> running C extension modules.
>>>>
>>>> I'll be curious to see where this project goes.
>>> given the comments on python-dev i wonder if this is the first
>>> indication
>>> that python is going to split into separate implementations for windows
>>> and unix (via .net and llvm, respectively)?

>> What comments are they?  There is no indication that unladen-swallow  is
>> fundamentally broken for Windows, just temporarily broken due to the
>> lack of windows developers/contributors...
>
> the comments you are referring to - that windows is not a priority, that
> they currently have no testing on windows machines, etc.  i quote, for
> example: "None of the three of us have Windows machines, nor do we
> particularly want to acquire them :), and Windows support isn't going to
> be a big priority."
>
>> Saying-no-to-fud ly,
>
> which part of "i wonder" don't you understand?  i'm not saying it is true,
> i'm just discussing the possibility.

Fair enough - although its reasonable for people to look for some 
indication of what you are saying to have some basis in reality, surely?

 > i am getting a little tired of people here acting so defensively...

Which part of my reply did you think was defensive?  I wasn't suggesting 
such a split would necessarily be a bad idea, just that there was no 
evidence of it.  IMO discussing such a split would create a huge amount 
of FUD, generating a huge amount of wasted energy if it wasn't true.

 > i'm discussing a programming language, not the size of your dick.

Wow, talk about jumping to conclusions :)  Is there something you feel 
the need to get off your chest?

Mark

From andrew at acooke.org  Sat Mar 28 22:59:40 2009
From: andrew at acooke.org (andrew cooke)
Date: Sat, 28 Mar 2009 17:59:40 -0400 (CLT)
Subject: [Python-Dev] Unladen-Swallow: A faster python
In-Reply-To: <49CE9966.5030803@gmail.com>
References: <ab2c8071-1d3f-4fcb-9669-a52114997aa9@h28g2000yqd.googlegroups.com>
	<lhgrs4pqklhe43e00jm8g0vhfcm7j2cnfp@4ax.com>
	<bec05a9536fe5b1d769c5374ae05739b.squirrel@localhost>
	<49CE2E2E.8000801@gmail.com>
	<20f41fcdc7ab12f58b19487ef847569c.squirrel@acooke.dyndns.org>
	<49CE9966.5030803@gmail.com>
Message-ID: <54e5e7e34e47a9ee1fdbb2f8b9c1634f.squirrel@acooke.dyndns.org>

Mark Hammond wrote:
[...I wrote]
>  > i'm discussing a programming language, not the size of your dick.
>
> Wow, talk about jumping to conclusions :)  Is there something you feel
> the need to get off your chest?

i'm not sure how this has ended up in python-dev; i was responding in
python and if you read that group my comments may have made a little more
sense (there were some hysterics in a separate thread accusing me of
saying python was "dying" because i was concerned about how the discussion
groups had evolved).

anyway, that had nothing to do with you and i am sorry i responded like that.

andrew



From guido at python.org  Sat Mar 28 23:08:43 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 28 Mar 2009 17:08:43 -0500
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CE98A7.7050303@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<49CC69D3.4030306@canterbury.ac.nz>
	<ca471dc20903271209t1af35fd9re95f475eb5e81b24@mail.gmail.com>
	<49CD7524.30403@canterbury.ac.nz>
	<20090328014323.37D323A4097@sparrow.telecommunity.com>
	<ca471dc20903271921k1774ec16sba8d2d3ca18d50a@mail.gmail.com>
	<49CDC57A.1050202@canterbury.ac.nz>
	<ca471dc20903280447o563848b4hdc74b2ac06c9d130@mail.gmail.com>
	<49CE98A7.7050303@canterbury.ac.nz>
Message-ID: <ca471dc20903281508p7d048028ve244b2b6b14deb42@mail.gmail.com>

On Sat, Mar 28, 2009 at 4:37 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>
>>>> I think in either case a check in
>>>> PyIter_Next() would cover most cases
>>
>>> If that's acceptable, then the check might as well
>>> be for None as the StopIteration value, and there's
>>> no need for a new exception.
>>
>> I don't understand this.
>
> Maybe I misunderstood what you were saying. What
> check were you suggesting to perform in PyIter_Next?

I now realize what you were saying. You said effectively "the check
added to PyIter_Next() might as well check whether the value attribute
of the StopIteration is not None", but due to PyCon tiredness last
night my brain's English parser didn't come up with any meaningful
parse of what you wrote.

But it's been answered already -- we can't change the meaning of
StopIteration() with a value unequal to None, so it has to be a
separate exception, and it should not derive from StopIteration.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Sat Mar 28 23:40:44 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 28 Mar 2009 15:40:44 -0700
Subject: [Python-Dev] Unladen-Swallow: A faster python
In-Reply-To: <54e5e7e34e47a9ee1fdbb2f8b9c1634f.squirrel@acooke.dyndns.org>
References: <ab2c8071-1d3f-4fcb-9669-a52114997aa9@h28g2000yqd.googlegroups.com>
	<lhgrs4pqklhe43e00jm8g0vhfcm7j2cnfp@4ax.com>
	<bec05a9536fe5b1d769c5374ae05739b.squirrel@localhost>
	<49CE2E2E.8000801@gmail.com>
	<20f41fcdc7ab12f58b19487ef847569c.squirrel@acooke.dyndns.org>
	<49CE9966.5030803@gmail.com>
	<54e5e7e34e47a9ee1fdbb2f8b9c1634f.squirrel@acooke.dyndns.org>
Message-ID: <20090328224044.GA26247@panix.com>

On Sat, Mar 28, 2009, andrew cooke wrote:
> Mark Hammond wrote:
> [...I wrote]
>>>
>>> i'm discussing a programming language, not the size of your dick.
>>
>> Wow, talk about jumping to conclusions :)  Is there something you feel
>> the need to get off your chest?
> 
> i'm not sure how this has ended up in python-dev; i was responding in
> python and if you read that group my comments may have made a little more
> sense (there were some hysterics in a separate thread accusing me of
> saying python was "dying" because i was concerned about how the discussion
> groups had evolved).

There certainly was no such accusation.  You said that c.l.py was in
"decline" (your own word), and I made reference to the ancient "Usenet
is dead, news at 11" meme.

http://groups.google.com/group/comp.lang.python/msg/0b3fbfcdb92ae0e3

Mark's question seems pertinent.  ;-)
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"At Resolver we've found it useful to short-circuit any doubt and just
refer to comments in code as 'lies'. :-)"
--Michael Foord paraphrases Christian Muirhead on python-dev, 2009-3-22

From greg.ewing at canterbury.ac.nz  Sun Mar 29 01:02:07 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 29 Mar 2009 12:02:07 +1200
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <ca471dc20903281508p7d048028ve244b2b6b14deb42@mail.gmail.com>
References: <49C48D03.7030709@gmail.com>
	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>
	<49CC69D3.4030306@canterbury.ac.nz>
	<ca471dc20903271209t1af35fd9re95f475eb5e81b24@mail.gmail.com>
	<49CD7524.30403@canterbury.ac.nz>
	<20090328014323.37D323A4097@sparrow.telecommunity.com>
	<ca471dc20903271921k1774ec16sba8d2d3ca18d50a@mail.gmail.com>
	<49CDC57A.1050202@canterbury.ac.nz>
	<ca471dc20903280447o563848b4hdc74b2ac06c9d130@mail.gmail.com>
	<49CE98A7.7050303@canterbury.ac.nz>
	<ca471dc20903281508p7d048028ve244b2b6b14deb42@mail.gmail.com>
Message-ID: <49CEBA7F.2020705@canterbury.ac.nz>

Guido van Rossum wrote:

> But it's been answered already -- we can't change the meaning of
> StopIteration() with a value unequal to None, so it has to be a
> separate exception, and it should not derive from StopIteration.

How about having StopIteration be a subclass of the
new exception? Then things that want to get generator
return values only have one exception to catch, and
things that only know about StopIteration will fail
to catch the new exception.

-- 
Greg


From regebro at gmail.com  Sun Mar 29 01:01:21 2009
From: regebro at gmail.com (Lennart Regebro)
Date: Sat, 28 Mar 2009 19:01:21 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <loom.20090325T121432-793@post.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
Message-ID: <319e029f0903281701p37f51cb8jac6abe2f6344d5d1@mail.gmail.com>

2009/3/25 Antoine Pitrou <solipsis at pitrou.net>:
> I'm not a Windows user, but I suppose it boils down to whether people are
> comfortable with the command-line or not (which even many Windows /developers/
> aren't). Since having GUIs for everything is part of the Windows philosophy,
> it's a fair expectation that Python libraries come with graphical Windows
> installers.

The people who use pythonlibraries are programmers. It can be expected
that they are comfortable with the command line. If not, somebody can
write a GUI for easy_install/pip interfacing to pypi.

Applications are expected to have a graphical installer, so a
gui-version of zc.buildout then? (I'm only half serious).

-- 
Lennart Regebro: Pythonista, Barista, Notsotrista.
http://regebro.wordpress.com/
+33 661 58 14 64

From solipsis at pitrou.net  Sun Mar 29 01:16:59 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 29 Mar 2009 00:16:59 +0000 (UTC)
Subject: [Python-Dev] "setuptools has divided the Python community"
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<319e029f0903281701p37f51cb8jac6abe2f6344d5d1@mail.gmail.com>
Message-ID: <loom.20090329T001600-988@post.gmane.org>

Lennart Regebro <regebro <at> gmail.com> writes:
> 
> The people who use pythonlibraries are programmers. It can be expected
> that they are comfortable with the command line.

You probably haven't met lots of Windows (so-called) programmers...

Regards

Antoine.



From ncoghlan at gmail.com  Sun Mar 29 03:14:49 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 29 Mar 2009 11:14:49 +1000
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CEBA7F.2020705@canterbury.ac.nz>
References: <49C48D03.7030709@gmail.com>	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>	<49CC69D3.4030306@canterbury.ac.nz>	<ca471dc20903271209t1af35fd9re95f475eb5e81b24@mail.gmail.com>	<49CD7524.30403@canterbury.ac.nz>	<20090328014323.37D323A4097@sparrow.telecommunity.com>	<ca471dc20903271921k1774ec16sba8d2d3ca18d50a@mail.gmail.com>	<49CDC57A.1050202@canterbury.ac.nz>	<ca471dc20903280447o563848b4hdc74b2ac06c9d130@mail.gmail.com>	<49CE98A7.7050303@canterbury.ac.nz>	<ca471dc20903281508p7d048028ve244b2b6b14deb42@mail.gmail.com>
	<49CEBA7F.2020705@canterbury.ac.nz>
Message-ID: <49CECB89.1020900@gmail.com>

Greg Ewing wrote:
> Guido van Rossum wrote:
> 
>> But it's been answered already -- we can't change the meaning of
>> StopIteration() with a value unequal to None, so it has to be a
>> separate exception, and it should not derive from StopIteration.
> 
> How about having StopIteration be a subclass of the
> new exception? Then things that want to get generator
> return values only have one exception to catch, and
> things that only know about StopIteration will fail
> to catch the new exception.

I actually like that, because it means that "coroutine-y" code could
pretty much ignore the existence of StopIteration (despite it existing
first).

It would also mean that it wouldn't matter if "return" and "return None"
both raised StopIteration, since they would still be intercepted by
GeneratorReturn exception handlers.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From jimjjewett at gmail.com  Sun Mar 29 03:17:47 2009
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sat, 28 Mar 2009 21:17:47 -0400
Subject: [Python-Dev] package resources [was: "setuptools has divided the
	Python community"]
Message-ID: <fb6fbf560903281817i352058a0t8470fa17b305ce5d@mail.gmail.com>

At 11:27 PM 3/26/2009 +0000, Paul Moore wrote:
>> What I'd really like is essentially some form of "virtual filesystem"
>> access to stuff addressed relative to a Python package name,

P.J. Eby responded:
> Note that relative to a *Python package name* isn't quite as useful,
> due to namespace packages.  To be unambiguous as to the targeted
> resource, one needs to be able to reference a specific project, and
> that requires you to go off the name of a module *within* a
> package.  For example, 'zope.somemodule' rather than just 'zope'.

I would expect it to be *most* important then.  If I know for sure
that an entire package is all together in a single directory, I can
just use that directory.  If I want all xxx files used by zope, then
... I *do* want information on the duplicates, and the multiple
locations.

-jJ

From rdmurray at bitdance.com  Sun Mar 29 03:53:58 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Sat, 28 Mar 2009 21:53:58 -0400 (EDT)
Subject: [Python-Dev] Are buffer overflows in modules deleted in py3k
	ignorable?
Message-ID: <Pine.LNX.4.64.0903282146390.26362@kimball.webabinitio.net>

I'm reviewing http://bugs.python.org/issue2591, which is marked as
'security' because it is a potential buffer overflow.  almodule.c has
been dropped in py3k, so my impulse is to close the bug as "won't fix".
But I thought I should check in to find out what the policy is before
doing that since it is a 'security' bug.

--
R. David Murray     http://www.bitdance.com

From guido at python.org  Sun Mar 29 05:01:09 2009
From: guido at python.org (Guido van Rossum)
Date: Sat, 28 Mar 2009 22:01:09 -0500
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49CECB89.1020900@gmail.com>
References: <49C48D03.7030709@gmail.com> <49CD7524.30403@canterbury.ac.nz>
	<20090328014323.37D323A4097@sparrow.telecommunity.com>
	<ca471dc20903271921k1774ec16sba8d2d3ca18d50a@mail.gmail.com>
	<49CDC57A.1050202@canterbury.ac.nz>
	<ca471dc20903280447o563848b4hdc74b2ac06c9d130@mail.gmail.com>
	<49CE98A7.7050303@canterbury.ac.nz>
	<ca471dc20903281508p7d048028ve244b2b6b14deb42@mail.gmail.com>
	<49CEBA7F.2020705@canterbury.ac.nz> <49CECB89.1020900@gmail.com>
Message-ID: <ca471dc20903282001k22ab956fp42a13cda45157292@mail.gmail.com>

On Sat, Mar 28, 2009 at 8:14 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Greg Ewing wrote:
>> Guido van Rossum wrote:
>>> But it's been answered already -- we can't change the meaning of
>>> StopIteration() with a value unequal to None, so it has to be a
>>> separate exception, and it should not derive from StopIteration.
>>
>> How about having StopIteration be a subclass of the
>> new exception? Then things that want to get generator
>> return values only have one exception to catch, and
>> things that only know about StopIteration will fail
>> to catch the new exception.
>
> I actually like that, because it means that "coroutine-y" code could
> pretty much ignore the existence of StopIteration (despite it existing
> first).

Okay.

> It would also mean that it wouldn't matter if "return" and "return None"
> both raised StopIteration, since they would still be intercepted by
> GeneratorReturn exception handlers.

Well I already didn't care about that. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mhagger at alum.mit.edu  Sun Mar 29 06:48:57 2009
From: mhagger at alum.mit.edu (Michael Haggerty)
Date: Sun, 29 Mar 2009 06:48:57 +0200
Subject: [Python-Dev] suggestion for try/except program flow
In-Reply-To: <e5b661850903270157yb43d076jdfd0fdd724902bc4@mail.gmail.com>
References: <e5b661850903270157yb43d076jdfd0fdd724902bc4@mail.gmail.com>
Message-ID: <49CEFDB9.7060509@alum.mit.edu>

Mark Donald wrote:
> I frequently have this situation:
> 
> try:
>     try:
>         raise Thing
>     except Thing, e:
>         # handle Thing exceptions
>         raise
> except:
>     # handle all exceptions, including Thing

This seems like an unusual pattern.  Are you sure you can't use

try:
    raise Thing
except Thing, e:
    # handle Thing exceptions
    raise
finally:
    # handle *all situations*, including Thing

Obviously, the finally: block is also invoked in the case that no
exceptions are triggered, but often this is what you want anyway...

Michael

From list at qtrac.plus.com  Sun Mar 29 09:21:44 2009
From: list at qtrac.plus.com (Mark Summerfield)
Date: Sun, 29 Mar 2009 08:21:44 +0100
Subject: [Python-Dev] Non-Core project: IDLE
In-Reply-To: <ac2200130903241911l30c096bbjefef87c7c30a9edf@mail.gmail.com>
References: <bad82a80903181356i4b6bb40ar716ab8cbe6196ff3@mail.gmail.com>
	<200903240726.34577.list@qtrac.plus.com>
	<ac2200130903241911l30c096bbjefef87c7c30a9edf@mail.gmail.com>
Message-ID: <200903290821.44695.list@qtrac.plus.com>

On 2009-03-25, Guilherme Polo wrote:
> On Tue, Mar 24, 2009 at 4:26 AM, Mark Summerfield <list at qtrac.plus.com> 
wrote:
> > On 2009-03-23, Guilherme Polo wrote:
> >> On Mon, Mar 23, 2009 at 5:39 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> >> > Guilherme Polo wrote:
> >> >> On Wed, Mar 18, 2009 at 8:53 PM, Terry Reedy <tjreedy at udel.edu> 
wrote:
> >> >>> IDLE needs lots of attention -- more than any one experienced person
> >> >>> is likely to have
> >> >>
> >> >> I'm willing to step up as a student for this but I still have to
> >> >> write a good proposal for it.
> >> >> My actual concern is about mentor availability, is someone around
> >> >> interested on being an IDLE mentor ?
> >> >
> >> > If I could, I would, and would have said so.  But I have only read
> >> > about tk and have not actually used it.  If I did decide to dive into
> >> > it, you'd be mentoring me ;-).  What I can and would do is give ideas
> >> > for changes, read and comment on a proposal, and user test patched
> >> > versions.
> >>
> >> That is very nice Terry. Do you have some specific ideas that you want
> >> to share publicly (or in private) about IDLE ? Your expectations about
> >> what should be addressed first, or areas that should be improved.. you
> >> know, anything.
> >
> > I have one suggestion that I think might be widely appreciated:
> >
> > Add somewhere in the configuration dialog when users can enter a block
> > of Python code to be executed at startup and whenever Restart Shell is
> > executed.
> >
> > Use case: for people who use IDLE for calculations/experiments they
> > might like to always have certain module imported. For me personally, it
> > would be:
> >
> >    import os
> >    import re
> >    import sys
> >    from math import *
> >
> > but of course the whole point is that people can write any code they
> > like. (Some people might want to do various from __future__ imports in
> > Python 2.6 to get various Python 3 features for example.)
> >
> > I know that you can use the -c option, but that only works at startup,
> > not every time you Restart Shell.
>
> Looks like a good suggestion to me, Mark.
> I would recommend adding it as a feature request on the typical place
> (bugs.python.org) because although I could just go and do it, I
> believe you are aware that new features in IDLE are subject to
> approval or disapproval by other members involved with IDLE. Hope you
> understand my position.
[snip]

Added as issue 5594.


-- 
Mark Summerfield, Qtrac Ltd, www.qtrac.eu
    C++, Python, Qt, PyQt - training and consultancy
        "Programming in Python 3" - ISBN 0137129297


From martin at v.loewis.de  Sun Mar 29 14:07:53 2009
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Mar 2009 07:07:53 -0500
Subject: [Python-Dev] bdist_linux (was: setuptools has divided the
 Python community)
In-Reply-To: <94bdd2610903271355p19719c92l3a56414d754bf637@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<24ea26600903261243h1d159792i963697a70b7ee2a8@mail.gmail.com>	<78407EF1-6240-42A0-BF49-E14E7EE4DC8A@python.org>	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	<20090326213047.6A8413A4097@sparrow.telecommunity.com>	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>	<49CCCB73.7060706@egenix.com>
	<49CCCE7A.8090903@trueblade.com>	<ca471dc20903271256q212deee1j179a23a89b55d596@mail.gmail.com>	<49CD3B98.7060004@egenix.com>
	<94bdd2610903271355p19719c92l3a56414d754bf637@mail.gmail.com>
Message-ID: <49CF6499.2080900@v.loewis.de>

> I think that each OS community should maintain its own tool, that complies
> to the OS standard (wich has its own evolution cycle)
> 
> Of course this will be possible as long as Distutils let the system
> packager find/change the metadata in an easy way.

In the specific case of RPMs, I still think that a distutils command
is the right solution. It may be that bdist_rpm is a bit too general,
and that bdist_fedora and bdist_suse might be more useful.

It all comes down to whether the .spec file is written manually or not.
*If* it is written manually, there is IMO no need to have the packager's
metadata readily available. Whoever writes the spec file can also look
at setup.py. OTOH, if the spec file is automatically generated, I can't
see why a bdist_ command couldn't do that - and a bdist_ command can
easily get at all the package (meta) data it needs.

So in this case, I think separate access to the meta-data
isn't needed/doesn't help.

For bdist_deb, things might be different, as the packager will certainly
want to maintain parts of the debian/ directory manually; other parts
would be generated. However, I still believe that a bdist_ command would
be appropriate (e.g. bdist_dpkg). As I understand Matthias Klose, the
tricky part here is that the packaging sometimes wants to reorganize
the files in a different layout, and for that, it needs to know what
files have what function (documentation, regular package, test suite,
etc). If that file classification isn't provided by the package author,
then there would be still a manually-maintained step to reorganize the
files.

The same is potentially true for RPM: if the files need to be layout
differently than the package author intended, automatic generation
of the spec file will also fail.

> I think this is the same rationale for debian packages. Right now people tend
> to use external tools like stdeb and they are OK with it (but still
> gets problems extracting stuff out of Python packages at this point)

Explicit is better than implicit: What is your list of problems with stdeb?

Regards,
Martin


From cournape at gmail.com  Sun Mar 29 15:09:21 2009
From: cournape at gmail.com (David Cournapeau)
Date: Sun, 29 Mar 2009 22:09:21 +0900
Subject: [Python-Dev] bdist_linux (was: setuptools has divided the
	Python community)
In-Reply-To: <49CF6499.2080900@v.loewis.de>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<49CCCB73.7060706@egenix.com> <49CCCE7A.8090903@trueblade.com>
	<ca471dc20903271256q212deee1j179a23a89b55d596@mail.gmail.com>
	<49CD3B98.7060004@egenix.com>
	<94bdd2610903271355p19719c92l3a56414d754bf637@mail.gmail.com>
	<49CF6499.2080900@v.loewis.de>
Message-ID: <5b8d13220903290609p6d9e2acejeccbd8ec260a29d4@mail.gmail.com>

2009/3/29 "Martin v. L?wis" <martin at v.loewis.de>:
>> I think that each OS community should maintain its own tool, that complies
>> to the OS standard (wich has its own evolution cycle)
>>
>> Of course this will be possible as long as Distutils let the system
>> packager find/change the metadata in an easy way.
>
> In the specific case of RPMs, I still think that a distutils command
> is the right solution. It may be that bdist_rpm is a bit too general,
> and that bdist_fedora and bdist_suse might be more useful.
>
> It all comes down to whether the .spec file is written manually or not.
> *If* it is written manually, there is IMO no need to have the packager's
> metadata readily available. Whoever writes the spec file can also look
> at setup.py. OTOH, if the spec file is automatically generated, I can't
> see why a bdist_ command couldn't do that - and a bdist_ command can
> easily get at all the package (meta) data it needs.
>
> So in this case, I think separate access to the meta-data
> isn't needed/doesn't help.
>
> For bdist_deb, things might be different, as the packager will certainly
> want to maintain parts of the debian/ directory manually; other parts
> would be generated. However, I still believe that a bdist_ command would
> be appropriate (e.g. bdist_dpkg). As I understand Matthias Klose, the
> tricky part here is that the packaging sometimes wants to reorganize
> the files in a different layout, and for that, it needs to know what
> files have what function (documentation, regular package, test suite,
> etc). If that file classification isn't provided by the package author,
> then there would be still a manually-maintained step to reorganize the
> files.

Maybe I don't understand what is meant by metadata, but I don't
understand why we can't provide the same metadata as autotools, so
that distutils could be customized from the command line to put data
where they belong (according to each OS convention). So that making a
FHS compliant package would be as simple as

python setup.py distutils --datadir=bla --htmldir=foo ....

I spent some time looking at cabal this afternoon ("haskell
distutils"), and from my current understanding, that's exactly what
they are doing:

http://www.haskell.org/cabal/release/cabal-latest/doc/users-guide/authors.html#pkg-descr

This way, if some metadata are not provided by upstream, the
downstream packager can fix it, and send patches upstream so that
other packagers benefit from it.

(FWIW, from the reading of cabal documentation, it looks like cabal
provides everything I would like distutils to provide: static
metadata, good documentation, sane handling of options, etc... Maybe
that's something worth looking into as inspiration for
improving/fixing distutils)

cheers,

David

From stephen at xemacs.org  Sun Mar 29 15:37:46 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 29 Mar 2009 22:37:46 +0900
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <loom.20090329T001600-988@post.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<49C88503.2030902@v.loewis.de> <49C886EF.80203@v.loewis.de>
	<49C8C9B3.3070403@holdenweb.com> <49C939BA.8040206@v.loewis.de>
	<49C95E9D.8070604@gmail.com> <49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>
	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>
	<loom.20090325T121432-793@post.gmane.org>
	<319e029f0903281701p37f51cb8jac6abe2f6344d5d1@mail.gmail.com>
	<loom.20090329T001600-988@post.gmane.org>
Message-ID: <87prg0witx.fsf@xemacs.org>

Antoine Pitrou writes:
 > Lennart Regebro <regebro <at> gmail.com> writes:
 > > 
 > > The people who use pythonlibraries are programmers. It can be expected
 > > that they are comfortable with the command line.
 > 
 > You probably haven't met lots of Windows (so-called) programmers...

Hey, the "(so-called)" should be avoided.  You'll lose your audience.

And it has very little to do with the issue at hand.  It's certainly
true that a lot of code is produced by people working in the Windows
environment who are somewhere between uncomfortable and totally at a
loss when faced with a shell prompt.  Whether they're "real"
programmers or "so-called" programmers, we believe that Python will
help them produce the best programs they're capable of, no?  Better
programs are a good thing, yes?  So they should (FSVO "should" not
necessarily implying Python-Dev should do something about it) have a
programming environment that enables their style of work.

From greg at krypto.org  Sun Mar 29 15:35:31 2009
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 29 Mar 2009 08:35:31 -0500
Subject: [Python-Dev] Are buffer overflows in modules deleted in py3k
	ignorable?
In-Reply-To: <Pine.LNX.4.64.0903282146390.26362@kimball.webabinitio.net>
References: <Pine.LNX.4.64.0903282146390.26362@kimball.webabinitio.net>
Message-ID: <52dc1c820903290635m6b0a41ag4fa056c85a22c034@mail.gmail.com>

It'd be worthy of fixing in 2.6 since the module exists.  Though honestly...
who cares about Irix?

On Sat, Mar 28, 2009 at 8:53 PM, R. David Murray <rdmurray at bitdance.com>wrote:

> I'm reviewing http://bugs.python.org/issue2591, which is marked as
> 'security' because it is a potential buffer overflow.  almodule.c has
> been dropped in py3k, so my impulse is to close the bug as "won't fix".
> But I thought I should check in to find out what the policy is before
> doing that since it is a 'security' bug.
>
> --
> R. David Murray     http://www.bitdance.com
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090329/f5db4889/attachment.htm>

From martin at v.loewis.de  Sun Mar 29 15:42:39 2009
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 29 Mar 2009 08:42:39 -0500
Subject: [Python-Dev] bdist_linux
In-Reply-To: <5b8d13220903290609p6d9e2acejeccbd8ec260a29d4@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>	
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>	
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>	
	<49CCCB73.7060706@egenix.com> <49CCCE7A.8090903@trueblade.com>	
	<ca471dc20903271256q212deee1j179a23a89b55d596@mail.gmail.com>	
	<49CD3B98.7060004@egenix.com>	
	<94bdd2610903271355p19719c92l3a56414d754bf637@mail.gmail.com>	
	<49CF6499.2080900@v.loewis.de>
	<5b8d13220903290609p6d9e2acejeccbd8ec260a29d4@mail.gmail.com>
Message-ID: <49CF7ACF.4030103@v.loewis.de>


> Maybe I don't understand what is meant by metadata, but I don't
> understand why we can't provide the same metadata as autotools

Likewise, *this* I do not understand. In what way does autotools
*provide* metadata? I can understand that it *uses* certain metadata,
but it doesn't *provide* them...

> So that making a
> FHS compliant package would be as simple as
> 
> python setup.py distutils --datadir=bla --htmldir=foo ....

What's the meaning of the distutils command?

Regards,
Martin

From cournape at gmail.com  Sun Mar 29 15:58:50 2009
From: cournape at gmail.com (David Cournapeau)
Date: Sun, 29 Mar 2009 22:58:50 +0900
Subject: [Python-Dev] bdist_linux
In-Reply-To: <49CF7ACF.4030103@v.loewis.de>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903262019w4fa2ff0fyd1fc8714db2c7563@mail.gmail.com>
	<49CCCB73.7060706@egenix.com> <49CCCE7A.8090903@trueblade.com>
	<ca471dc20903271256q212deee1j179a23a89b55d596@mail.gmail.com>
	<49CD3B98.7060004@egenix.com>
	<94bdd2610903271355p19719c92l3a56414d754bf637@mail.gmail.com>
	<49CF6499.2080900@v.loewis.de>
	<5b8d13220903290609p6d9e2acejeccbd8ec260a29d4@mail.gmail.com>
	<49CF7ACF.4030103@v.loewis.de>
Message-ID: <5b8d13220903290658x7f8a5f1dt62f9bb2d023f809f@mail.gmail.com>

On Sun, Mar 29, 2009 at 10:42 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
>> Maybe I don't understand what is meant by metadata, but I don't
>> understand why we can't provide the same metadata as autotools
>
> Likewise, *this* I do not understand. In what way does autotools
> *provide* metadata? I can understand that it *uses* certain metadata,
> but it doesn't *provide* them...

It does not provide them to external tools, true. Let me rephrase
this: why cannot distutils use and provide metadata corresponding to
the different categories as available in autotools ? It provides both
customization from the command line and a relatively straightforward
way to set which files go where.

Last time this point was discussed on distutils-sig, the main worry
came from people who do not care about tagging things appropriately. I
don't think it is a big problem, because people already do it in
setup.py, or distutils can do it semi automatically (it already has
different categories for .py, .pyc, data files, libraries). Also,
since packagers have to do it anyway, I think they would prefer
something which enable them to send those changes upstream instead of
every OS packager having to do it.

>> python setup.py distutils --datadir=bla --htmldir=foo ....
>
> What's the meaning of the distutils command?

Sorry, this should read python setup.py install ...

cheers,

David

From rdmurray at bitdance.com  Sun Mar 29 16:37:21 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 29 Mar 2009 10:37:21 -0400 (EDT)
Subject: [Python-Dev] Are buffer overflows in modules deleted in py3k
 ignorable?
In-Reply-To: <52dc1c820903290635m6b0a41ag4fa056c85a22c034@mail.gmail.com>
References: <Pine.LNX.4.64.0903282146390.26362@kimball.webabinitio.net>
	<52dc1c820903290635m6b0a41ag4fa056c85a22c034@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0903291035150.26362@kimball.webabinitio.net>

On Sun, 29 Mar 2009 at 08:35, Gregory P. Smith wrote:
> It'd be worthy of fixing in 2.6 since the module exists.  Though honestly...
> who cares about Irix?

Guido commented on the ticket and closed it, so I closed the other two like it.

--
R. David Murray         http://www.bitdance.com

From solipsis at pitrou.net  Sun Mar 29 16:42:44 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 29 Mar 2009 14:42:44 +0000 (UTC)
Subject: [Python-Dev] pyc files,
	constant folding and borderline portability issues
Message-ID: <loom.20090329T143625-223@post.gmane.org>

Hello,

There are a couple of ancillary portability concerns due to optimizations which
store system-dependent results of operations between constants in pyc files:

- Issue #5057: code like '\U00012345'[0] is optimized away and its result stored
as a constant in the pyc file, but the result should be different in UCS-2 and
UCS-4 builds.
- Issue #5593: code like 1e16+2.9999 is optimized away and its result stored as
a constant (again), but the result can vary slightly depending on the internal
FPU precision.

These problems have probably been there for a long time and almost no one seems
to complain, but I thought I'd report them here just in case.

Regards

Antoine.



From guido at python.org  Sun Mar 29 17:36:01 2009
From: guido at python.org (Guido van Rossum)
Date: Sun, 29 Mar 2009 10:36:01 -0500
Subject: [Python-Dev] pyc files,
	constant folding and borderline 	portability issues
In-Reply-To: <loom.20090329T143625-223@post.gmane.org>
References: <loom.20090329T143625-223@post.gmane.org>
Message-ID: <ca471dc20903290836n7420b81ep39bdb9bfd2373757@mail.gmail.com>

On Sun, Mar 29, 2009 at 9:42 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> There are a couple of ancillary portability concerns due to optimizations which
> store system-dependent results of operations between constants in pyc files:
>
> - Issue #5057: code like '\U00012345'[0] is optimized away and its result stored
> as a constant in the pyc file, but the result should be different in UCS-2 and
> UCS-4 builds.

Why would anyone write such code (disregarding the \U escape problem)?
So why do we bother optimizing this?

> - Issue #5593: code like 1e16+2.9999 is optimized away and its result stored as
> a constant (again), but the result can vary slightly depending on the internal
> FPU precision.

I would just not bother constant folding involving FP, or only if the
values involved have an exact representation in IEEE binary FP format.

> These problems have probably been there for a long time and almost no one seems
> to complain, but I thought I'd report them here just in case.

I would expect that constant folding isn't nearly effective in Python
as in other (less dynamic) languages because it doesn't do anything
for NAMED constants. E.g.

MINUTE = 60

def half_hour():
    return MINUTE*30

This should be folded to "return 1800" but doesn't because the
compiler doesn't know that MINUTE is a constant.

Has anyone ever profiled the effectiveness of constant folding on
real-world code? The only kind of constant folding that I expect to be
making a diference is things like unary operators, since e.g. "x = -2"
is technically an expression involving a unary minus.

ISTM that historically, almost every time we attempted some new form
of constant folding, we introduced a bug.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From vikas_it_pec at yahoo.co.in  Sun Mar 29 19:04:36 2009
From: vikas_it_pec at yahoo.co.in (garg vikas)
Date: Sun, 29 Mar 2009 22:34:36 +0530 (IST)
Subject: [Python-Dev] python core project
Message-ID: <988433.66488.qm@web95207.mail.in2.yahoo.com>

??? hi everybody , i am going to do the project? on nose compatibility with
core-python testing infrastructure. so i have written the abstract for this project if someone want
to mentor this project plesae contact me ?

???????????????????????? Core Python Testing Infrastructure | Nose compatibility 
? ? ? ?????????????????????  
ABSTRACT
 :-
????????????
Python is improving day by day by providing more features and facilites
to python programmers,but the developers of core-python has week
testing infrastructure, but as they are well known of the python code
so they can use their own hackish testing ,but "testing the core-python
code" is great problem for the new developers who want to participate
in developing core-python and are new to this core-python code.So
"core-python Testing Framework" requires improvements which can be
provided by using "nose - unit test discovery and execution framework"
which provides you selectively execute certain tests, capture and
collate error output, and add coverage and profiling information in
addition to finding and running your tests.so nose has such great
features but it has not campatible with core-python testing suit.So
that is my aim of project to provide nose framework to "core-python
testing suit".nose also provides one more good feature, that is plugins
support means you can write your plugins which helps in providing 

1. support for testing suits which are not compatible with nose
2. improve the functionality of nose to provide features to your testing infrastructure.
?
??????????
second aim of this project is not going to change anything of
core-python testing infrastructure or with nose but this project will
be an bridge between nose and core-python which will include nose
-plugins and wrappers for providing beautiful features like tag-based
execution,code-coverage and many more features to the core-python
testing infrastructure.it will make testing of code more relevent,make
easy fixing of bugs and as with its tag-based execution property will
help in running selective tests and which will also help in lowering
testing time.It's code-coverage property will make help in finding all
untested places in "code of core-python". 
?????????????????????????????????????????????????????????????????? My email:- vikas_it_pec at yahoo.co.in
Thanks
vikas garg



      Add more friends to your messenger and enjoy! Go to http://messenger.yahoo.com/invite/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090329/b99f6341/attachment.htm>

From jyasskin at gmail.com  Sun Mar 29 19:21:01 2009
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Sun, 29 Mar 2009 12:21:01 -0500
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
Message-ID: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>

I've heard some good things about cmake ? LLVM, googletest, and Boost
are all looking at switching to it ? so I wanted to see if we could
simplify our autoconf+makefile system by using it. The biggest wins I
see from going to cmake are:
 1. It can autogenerate the Visual Studio project files instead of
needing them to be maintained separately
 2. It lets you write functions and modules without understanding
autoconf's mix of shell and M4.
 3. Its generated Makefiles track header dependencies accurately so we
might be able to add private headers efficiently.

However, after trying it out during PyCon and looking over the
previous attempt at
<http://mail.python.org/pipermail/python-dev/2007-July/073912.html>, I
can't recommend switching to it.

 A. It has no equivalent of autoheader, so we'd have to maintain
pyconfig.h.in by hand.
 B. It does not allow the CMakeLists.txt file control the --help
output. This appears to be an intentional decision
(http://www.cmake.org/pipermail/cmake-promote/2006-May/000095.html).
To replace it, they have an interactive mode (which asks you about
each possible option) and a -LH flag (which runs the whole configure
process and then tells you about the flags you could have set if you
knew about them).
 C. It does not allow the CMakeLists.txt file to see the command line,
so we can't stay backward compatible with the configure script, and
we'd have to replace flags like --with-pydebug with
-Dpydebug:bool=true. We could write a separate configure script to
mitigate this and the previous problem, but we'd have to keep that in
sync with CMakeLists.txt manually.
 D. It doesn't have an expression language, so to replace
   	ac_md_system=`echo $ac_sys_system |
			   tr -d '[/ ]' | tr '[[A-Z]]' '[[a-z]]'`
  you have to write:
    string(REGEX REPLACE "[/ ]" "" ac_md_system ${ac_sys_system})
    string(TOLOWER ${ac_md_system} ac_md_system)

So, at the very least, it doesn't look like a big enough win to
justify switching, and may not be a win at all.


The other popular configure+make replacement is scons. The biggest
objection to scons before even looking at it is that it requires a
working Python interpreter in order to build Python, causing a
bootstrap problem. However, Brett Cannon and I talked, and we think
this is surmountable. First, nearly every desktop system comes with a
Python interpreter, which would avoid the bootstrap for ordinary
development. Second, we need a cross compilation story anyway for
embedded systems, and the same mechanism could be used to get Python
onto a new desktop platform. Third, Jython and IronPython are pretty
mature and either can run scons now or should be able to run it after
some relatively easy tweaks. They could be used to build CPython on a
new platform. I don't intend to look into scons myself any time soon,
but I'd be interested in anyone's experiences who does try it.

Jeffrey

From aahz at pythoncraft.com  Sun Mar 29 19:24:44 2009
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 29 Mar 2009 10:24:44 -0700
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
Message-ID: <20090329172443.GA22562@panix.com>

Nice report!  Thanks!
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it."  --Brian W. Kernighan

From solipsis at pitrou.net  Sun Mar 29 19:59:35 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 29 Mar 2009 17:59:35 +0000 (UTC)
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
Message-ID: <loom.20090329T175435-783@post.gmane.org>

Jeffrey Yasskin <jyasskin <at> gmail.com> writes:
> 
> The other popular configure+make replacement is scons.

I can only give uninformed information (!) here, but in one company I worked
with, the main project decided to switch from scons to cmake due to some huge
performance problems in scons. This was in 2005-2006, though, and I don't know
whether things have changed.

If you want to investigate Python-based build systems, there is waf (*), which
apparently started out as a fork of scons (precisely due to the aforementioned
performance problems). Again, I have never tried it.

(*) http://code.google.com/p/waf/

Regards

Antoine.



From cournape at gmail.com  Sun Mar 29 20:14:30 2009
From: cournape at gmail.com (David Cournapeau)
Date: Mon, 30 Mar 2009 03:14:30 +0900
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <loom.20090329T175435-783@post.gmane.org>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
	<loom.20090329T175435-783@post.gmane.org>
Message-ID: <5b8d13220903291114k17e9eff9v6d1a5eef1fb72332@mail.gmail.com>

On Mon, Mar 30, 2009 at 2:59 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Jeffrey Yasskin <jyasskin <at> gmail.com> writes:
>>
>> The other popular configure+make replacement is scons.
>
> I can only give uninformed information (!) here, but in one company I worked
> with, the main project decided to switch from scons to cmake due to some huge
> performance problems in scons. This was in 2005-2006, though, and I don't know
> whether things have changed.

They haven't - scons is still slow. Python is not that big, though
(from a build POV) ?

I would think the bootstrap problem to be much more significant. I
don't find the argument "many desktop have already python" very
convincing - what if you can't install it, for example ? AFAIK, scons
does not run on jython or ironpython.

>
> If you want to investigate Python-based build systems, there is waf (*), which
> apparently started out as a fork of scons (precisely due to the aforementioned
> performance problems). Again, I have never tried it.

Waf is definitely faster than scons - something like one order of
magnitude. I am yet very familiar with waf, but I like what I saw -
the architecture is much nicer than scons (waf core amount of code is
almost ten times smaller than scons core), but I would not call it a
mature project yet.

About cmake: I haven't looked at it recently, but I have a bit of hard
time believing python requires more from a build system than KDE. The
lack of autoheader is not accurate, if
only because kde projects have it:

http://www.cmake.org/Wiki/CMake_HowToDoPlatformChecks

Whether using it compared to the current system is really a win for
python, I have no idea.

David

From solipsis at pitrou.net  Sun Mar 29 20:18:04 2009
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 29 Mar 2009 18:18:04 +0000 (UTC)
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
	<loom.20090329T175435-783@post.gmane.org>
	<5b8d13220903291114k17e9eff9v6d1a5eef1fb72332@mail.gmail.com>
Message-ID: <loom.20090329T181554-672@post.gmane.org>

David Cournapeau <cournape <at> gmail.com> writes:
> 
> I would think the bootstrap problem to be much more significant. I
> don't find the argument "many desktop have already python" very
> convincing - what if you can't install it, for example ?

I agree. I had to build Python once on a corporate AIX box without any modern
facilities. If it had needed anything else than a standard C compiler, I
couldn't have done it.

> About cmake: I haven't looked at it recently, but I have a bit of hard
> time believing python requires more from a build system than KDE.

What are the compilation requirements for cmake itself? Does it only need a
standard C compiler and library, or are there other dependencies?



From cournape at gmail.com  Sun Mar 29 20:24:49 2009
From: cournape at gmail.com (David Cournapeau)
Date: Mon, 30 Mar 2009 03:24:49 +0900
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <loom.20090329T181554-672@post.gmane.org>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
	<loom.20090329T175435-783@post.gmane.org>
	<5b8d13220903291114k17e9eff9v6d1a5eef1fb72332@mail.gmail.com>
	<loom.20090329T181554-672@post.gmane.org>
Message-ID: <5b8d13220903291124w670ed5bdx45c2982505511072@mail.gmail.com>

On Mon, Mar 30, 2009 at 3:18 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> What are the compilation requirements for cmake itself? Does it only need a
> standard C compiler and library, or are there other dependencies?

CMake is written in C++. IIRC, that's the only dependency.

cheers,

David

From jyasskin at gmail.com  Sun Mar 29 20:59:05 2009
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Sun, 29 Mar 2009 13:59:05 -0500
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <5b8d13220903291114k17e9eff9v6d1a5eef1fb72332@mail.gmail.com>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
	<loom.20090329T175435-783@post.gmane.org>
	<5b8d13220903291114k17e9eff9v6d1a5eef1fb72332@mail.gmail.com>
Message-ID: <5d44f72f0903291159p7daa18e0w42cf05df7eb660d4@mail.gmail.com>

On Sun, Mar 29, 2009 at 1:14 PM, David Cournapeau <cournape at gmail.com> wrote:
> About cmake: I haven't looked at it recently, but I have a bit of hard
> time believing python requires more from a build system than KDE. The
> lack of autoheader is not accurate, if
> only because kde projects have it:
>
> http://www.cmake.org/Wiki/CMake_HowToDoPlatformChecks

That page says, "... So we write a source file named config.h.in...".
The purpose of autoheader is to write that file automatically. I might
have missed something, but you'll have to provide a more precise link.

The problems I found were enough to convince me that it wasn't worth
continuing to a full translation of configure.in and Makefile.pre.in.
If you or someone else wants to do that translation, I'd be happy to
look at the result in case it turns out not to be as inconvenient as I
currently expect.

Jeffrey

From db3l.net at gmail.com  Sun Mar 29 22:52:20 2009
From: db3l.net at gmail.com (David Bolen)
Date: Sun, 29 Mar 2009 16:52:20 -0400
Subject: [Python-Dev] Test failures under Windows?
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net>
	<49C9EEB5.2090804@gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>
	<d2155e360903250553u2e134d61rc005e8817ad82a60@mail.gmail.com>
	<m2ab791r8o.fsf@valheru.db3l.homeip.net>
Message-ID: <m2tz5cxda3.fsf@valheru.db3l.homeip.net>

David Bolen <db3l.net at gmail.com> writes:

>>From what I can see though, the tools/buildbot/test.bat file no longer
> adds the -n option that it used to, although I'm unclear on why it
> might have been removed.  Perhaps this was just a regression that was
> accidentally missed, as it appears to have disappeared during a large
> merger from trunk to the py3k branch mid-2008 (r64273) when the batch
> file line ending was changed to CRLF.
>
> It would be nice to also have this in the other active release branches.

This thread sort of died out ... would there be any objections to
restoring the -n option in the buildbot test.bat file for Windows
buildbots?

I just went through clearing a ton of popups again on my build slave,
but in the end had to again manually kill off all the individual
python_d processes, as the popups just seemed to keep getting created
anew.

I don't know why they are happening so frequently now when there was a
reasonable period when they weren't an issue (something about new I/O
support in 3.x perhaps?), but without preventing them it seems the
Windows build slaves are going to become (if not already) quite a bit
less useful.  Don't know about anyone else's but I can't watch mine
7x24.

-- David


From barry at python.org  Sun Mar 29 23:53:47 2009
From: barry at python.org (Barry Warsaw)
Date: Sun, 29 Mar 2009 16:53:47 -0500
Subject: [Python-Dev] Python 2.6.2
Message-ID: <E21726DE-05A8-4EA8-BB99-DAD257F0613C@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'd like to release Python 2.6.2 the week after the conference.  I've  
talked to a few people here about it and the general consensus is that  
we do one brown-paper-bag-avoiding release candidate first.  Looking  
at the calendar, I propose the following schedule:

Monday April 6: 2.6.2 rc1
Friday April 10: 2.6.2 final

As usual, I plan on freezing the tree to create the Subversion tags at  
UTC 2200 or 6pm US/Eastern the day before the release to give Martin  
time to build the Windows installers.  If you have stuff to get into  
2.6.2, please do so before rc1.  You'll need to get approval from me  
for all changes between 2.6.2rc1 and 2.6.2 final.

Please let me know if this schedule does not work for you.
Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSc/t63EjvBPtnXfVAQLL1QQAs1MQKs4x4Zvg5kMyzfcM/7Gtl7OmB8it
dYVz0F0xuaWoNrAVxWrSnnIA4jaorZtWGk7/E0pn2kJ1oDdZdyqYQa0T86pKh1q0
n8/2ml8jNph92SlWK8UvgijK92x21rTBO1DZ+KJPJ0JYuCD2QTOTJY25MGk9M5LV
k8E5IzSr7R0=
=3Ezw
-----END PGP SIGNATURE-----

From nyamatongwe at gmail.com  Mon Mar 30 00:09:21 2009
From: nyamatongwe at gmail.com (Neil Hodgson)
Date: Mon, 30 Mar 2009 09:09:21 +1100
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
Message-ID: <50862ebd0903291509g60c988feme0297d7d4e975129@mail.gmail.com>

Jeffrey Yasskin:

> ?1. It can autogenerate the Visual Studio project files instead of
> needing them to be maintained separately

   I have looked at a couple of build tools (scons was probably one)
that generate Visual Studio project files in the past and they
produced fairly poor project files, which would compile the code but
wouldn't be as capable as project files created by hand. Its been a
while so I can't remember the details. The current Python project
files are hierarchical, building several DLLs and an EXE and I think
this was outside the scope of the tools I looked at.

   Neil

From lists at cheimes.de  Mon Mar 30 01:22:53 2009
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 30 Mar 2009 01:22:53 +0200
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
Message-ID: <gqovsd$rnv$1@ger.gmane.org>

Jeffrey Yasskin wrote:

>  1. It can autogenerate the Visual Studio project files instead of
> needing them to be maintained separately

I'm familiar with the Unix and the Windows build system. More than a
year ago I went to a great deal of work to migrate the Windows builds
from VS 7.1 to VS 9.0. I'm in doubt that any automatic tool can create
configuration files that are as good as our hand made files.

The VS project files support debug, non debug and profile guided
optimization builds for X86 and AMD64 including cross compilation of
AMD64 binaries. The project files are using multiple inheritance to
avoid duplication of options.
The differences between Windows and Unix builds are fairly large, too.
On Windows lots of modules are built in and the remaining Python
extensions are build with VS directly. On Unix most modules are build as
shared libraries using distutils and setup.py.

In my opinion any change to an automated system is a waste of precious
developer times and makes our Windows support worse.

Christian


From asmodai at in-nomine.org  Mon Mar 30 07:53:07 2009
From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven)
Date: Mon, 30 Mar 2009 07:53:07 +0200
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
Message-ID: <20090330055307.GL21667@nexus.in-nomine.org>

-On [20090329 19:21], Jeffrey Yasskin (jyasskin at gmail.com) wrote:
>However, Brett Cannon and I talked, and we think this is surmountable.
>First, nearly every desktop system comes with a Python interpreter, which
>would avoid the bootstrap for ordinary development.

This is quite a major assumption.

Most FreeBSD, NetBSD, and OpenBSD users tend to install a minimal binary OS
(kernel plus system tools) and then proceed to install third party
applications via either ports or pkgsrc. This means that Python gets built
from scratch. So depending on Python to build Python seems a bad decision if
you care for my opinion.

-- 
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
????? ?????? ??? ?? ??????
http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B
I must be cruel, only to be kind...

From ocean-city at m2.ccsnet.ne.jp  Mon Mar 30 10:30:46 2009
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Mon, 30 Mar 2009 17:30:46 +0900
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <m2tz5cxda3.fsf@valheru.db3l.homeip.net>
References: <loom.20090324T134451-902@post.gmane.org>	<m2zlfa241x.fsf@valheru.db3l.homeip.net>	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net>	<49C9EEB5.2090804@gmail.com>	<930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>	<d2155e360903250553u2e134d61rc005e8817ad82a60@mail.gmail.com>	<m2ab791r8o.fsf@valheru.db3l.homeip.net>
	<m2tz5cxda3.fsf@valheru.db3l.homeip.net>
Message-ID: <49D08336.5070907@m2.ccsnet.ne.jp>


David Bolen wrote:
> I don't know why they are happening so frequently now when there was a
> reasonable period when they weren't an issue (something about new I/O
> support in 3.x perhaps?), but without preventing them it seems the
> Windows build slaves are going to become (if not already) quite a bit
> less useful.  Don't know about anyone else's but I can't watch mine
> 7x24.
> 
> -- David

CRT Assertion was totally disabled before, but recently was enabled,
and workarounds were patched for problematic functions. (ex: fdopen and 
dup) Probably this *patch* is not perfect. See 
http://bugs.python.org/issue4804

I'm now +3/4 for the idea disabling assertion by default,
and enabling by startup option or environment variable. (Or enabling
by default and disabling by environment variable?)


From baptiste13z at free.fr  Mon Mar 30 10:58:52 2009
From: baptiste13z at free.fr (Baptiste Carvello)
Date: Mon, 30 Mar 2009 10:58:52 +0200
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<49C96304.7050207@v.loewis.de>
	<49C99C7B.3080805@holdenweb.com>	<79990c6b0903250508oae7f35fp86da758d9c3eafae@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>	<87myb9w6sq.fsf@benfinney.id.au>
	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>
Message-ID: <gqq1kf$o7$1@ger.gmane.org>

Tennessee Leeuwenburg a ?crit :
> I would suggest there may be three use cases for Python installation 
> tools. Bonus -- I'm not a web developer! :)
> 
> Case One: Developer wishing to install additional functionality into the 
> system Python interpreter forever
> Case Two: Developer wishing to install additional functionality into the 
> system Python interpreter for a specific task
> Case Three: Person wanting to install an application which happens to be 
> written in Python
> 

a maybe more exotic Case Four: the enlightened open-source user, who wants to 
install an application, but might like to look under the hood, either to tweek 
it to his needs, or to investigate a bug.

In that case, each application comming with a private version of various system 
libraries is unwelcomed complication, so that using the platform's package 
manager is a much better solution.

Cheers,
B.


From baptiste13z at free.fr  Mon Mar 30 11:09:18 2009
From: baptiste13z at free.fr (Baptiste Carvello)
Date: Mon, 30 Mar 2009 11:09:18 +0200
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <gqgmp7$3ku$1@ger.gmane.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>	<loom.20090325T121432-793@post.gmane.org>	<94bdd2610903250636t4126194dqbdf0b7e7c7209533@mail.gmail.com>	<79990c6b0903250708t49813f7dvf6fd5f567d5832e1@mail.gmail.com>	<94bdd2610903250723w583b0cc2r57cec95936cdc7bb@mail.gmail.com>	<79990c6b0903250735t22377c87wf82446d50c96265d@mail.gmail.com>	<87myb9w6sq.fsf@benfinney.id.au>	<43c8685c0903251606h64e6061bw9f8c45e3534bc41b@mail.gmail.com>	<25D4BA0B-4A2C-4FE4-8FAD-36CFFD5166D0@python.org>	<gqglfq$und$2@ger.gmane.org>	<24ea26600903261247t32af6eb4xde2b957571d44018@mail.gmail.com>
	<gqgmp7$3ku$1@ger.gmane.org>
Message-ID: <gqq280$31l$1@ger.gmane.org>

Tres Seaver a ?crit :
> 
> Note that the kind of applications I work on tend to be the sort which
> will run as server apps, and which will (in production) be the entire
> rasion d'etre for the machine they run on, which makes the cost of
> isolation tiny compared to the consequences of failed isolation.
> 

Indeed. More fundamentally, different use cases call for the dependency 
management to happen in different places.

In the case of the web application, the dependencies must be resolved on the 
developper machine, and tested there, then the full bundle is pushed to the 
production server where failure is not an option.

This is in strong contrast with the Debian (for example) use case, where you 
want the dependencies to be resolved on the end user machine, because that way 
Debian has to support just N independant packages, not NxN possible user 
configurations.

Cheers,
B.


From csaba.balazs at researchstudio.at  Mon Mar 30 13:47:59 2009
From: csaba.balazs at researchstudio.at (Csaba Balazs)
Date: Mon, 30 Mar 2009 13:47:59 +0200
Subject: [Python-Dev] CPython and C++ object GC
Message-ID: <20090330134759.1u3l4o0ccgcwwg4w@webmail.researchstudio.at>

Hello Everybody,

I would like to use a C++ gui library with the following (simplified) interface
in Python.

#include <stdio.h>

class Gui;

class GuiObject {
public:
	GuiObject(Gui *Gui) {printf("creating GuiObject(gui: %X)\n", Gui);}
	~GuiObject() {printf("deleting GuiObject\n");}
	void Move(int x, int y) {printf("GuiObject move(%d, %d)\n", x, y);};
};

class Gui {
public:
	Gui()  {printf("creating Gui\n");}
	~Gui() {printf("deleting Gui\n");}
	GuiObject* AddImage() {
		GuiObject* ob = new GuiObject(this);
		return ob;
	}
	void Print() {printf("Gui: %X\n", this);}
};

int main() {
	Gui *gui = new Gui();
	gui->Print();
	GuiObject *obj = gui->AddImage();
	obj->Move(50, 50);
	/*GuiObject *obj2 = new GuiObject(gui); // not allowed
	delete obj2;*/
	delete obj;
	delete gui;
	return 0;
}


I created the Python Gui and GuiObject classes (PyTypeObject), and added it to
main module (PyModule_AddObject).
It works, but there is a problem at the Gui::AddImage(), with constructs a new
GuiObject, which is available in Python layer but finally it is not collected
and freed by GC:

...
obj = _PyObject_New(&GuiObjectType);
PyObject_Init(obj, &GuiObjectType);
...

I cannot invoke the GuiObject object constructor directly from Python,
because of the implementation of the C++ gui library (in this case it would be
collected).
I use the embedded CPython as an interpreter, so I cannot add additional
external .py file for it.

So the following Python code would be the target:

gui = GUI();

background = gui.AddImage();
#background = GuiObject(gui); <-- Collected but not allowed
background.ImageFile("bg.jpg");
background.Move(0, 0);
...

How could I implement the AddImage function in order to be freed the constructed
object at the end?

Thanks in advance!

All-the-best,

Csaba


From ncoghlan at gmail.com  Mon Mar 30 14:04:22 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 30 Mar 2009 22:04:22 +1000
Subject: [Python-Dev] CPython and C++ object GC
In-Reply-To: <20090330134759.1u3l4o0ccgcwwg4w@webmail.researchstudio.at>
References: <20090330134759.1u3l4o0ccgcwwg4w@webmail.researchstudio.at>
Message-ID: <49D0B546.2090003@gmail.com>

Csaba Balazs wrote:
> Hello Everybody,
> 
> I would like to use a C++ gui library with the following (simplified) interface
> in Python.

This is a question for python-list/comp.lang.python (i.e. development
*using* Python, including the C API), not python-dev (which is for
development of the language itself).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From eric at trueblade.com  Mon Mar 30 14:24:59 2009
From: eric at trueblade.com (Eric Smith)
Date: Mon, 30 Mar 2009 07:24:59 -0500
Subject: [Python-Dev] CPython and C++ object GC
In-Reply-To: <49D0B546.2090003@gmail.com>
References: <20090330134759.1u3l4o0ccgcwwg4w@webmail.researchstudio.at>
	<49D0B546.2090003@gmail.com>
Message-ID: <49D0BA1B.1070109@trueblade.com>

Nick Coghlan wrote:
> Csaba Balazs wrote:
>> Hello Everybody,
>>
>> I would like to use a C++ gui library with the following (simplified) interface
>> in Python.
> 
> This is a question for python-list/comp.lang.python (i.e. development
> *using* Python, including the C API), not python-dev (which is for
> development of the language itself).

There's also the capi-sig mailing list, 
http://mail.python.org/mailman/listinfo/capi-sig.


From olemis at gmail.com  Mon Mar 30 15:41:20 2009
From: olemis at gmail.com (Olemis Lang)
Date: Mon, 30 Mar 2009 08:41:20 -0500
Subject: [Python-Dev] "setuptools has divided the Python community"
In-Reply-To: <0B6ECAA5-0BA9-4D72-81A6-A8EF27FFBC76@python.org>
References: <fb6fbf560903081251m4219b117l63c6bcb71e199be6@mail.gmail.com>
	<ca471dc20903261328s5462674cj458a2793e6bdc5d6@mail.gmail.com>
	<20090326213047.6A8413A4097@sparrow.telecommunity.com>
	<49CC57FC.30907@trueblade.com>
	<20090327170202.4E5D03A40AF@sparrow.telecommunity.com>
	<49CD17F8.1020501@egenix.com> <49CD192D.1090402@trueblade.com>
	<24ea26600903271141h278fc5bfu7ed5f568bf470073@mail.gmail.com>
	<49CD2889.5050109@trueblade.com>
	<0B6ECAA5-0BA9-4D72-81A6-A8EF27FFBC76@python.org>
Message-ID: <24ea26600903300641l5b7f3e87ncd1a3891abb4f138@mail.gmail.com>

On Fri, Mar 27, 2009 at 4:27 PM, Barry Warsaw <barry at python.org> wrote:
> On Mar 27, 2009, at 2:27 PM, Eric Smith wrote:
>> Olemis Lang wrote:
>>>>
>>>> I also think the feature should go. If you want functionality that's so
>>>> difficult to provide when you install as a zip file, the answer is not
>>>> to
>>>> make things more complex, but to not install as zip files.
>>>>
>>> What about environments like Google App Engine ? I mean, AFAIK using
>>> ZIP files is the *official* solution to meet some quotas &
>>> requirements ... CMIIW anyway ...
>>
>> I mentioned yesterday (in the language summit) that we really need to get
>> all the requirements together before we start any work. I fear that there
>> are many hidden requirements (or ones that I'm personally not aware of, at

I remembered another similar use case (which is mentioned in PEP 302
-motivation AFAICR- ) ... what about importing Py modules/pkgs
packaged in .JAR files to be used in Jython ? ... Hmmmmm?

I dont think its a good idea so far to remove zip imports | installs
and similar ... at least I'll need further arguments and solutions to
concrete use cases to understand this decision a little bit better...
;)

>> least). I don't know gettext well enough give an answer yet.
>
> The class-based API for gettext takes streams, so resource_stream() would
> work just fine. ?I think i18n plugins for Python do not necessarily need to
> use the classic gettext API.
>

In fact ... I use Babel (... especially Translations & Locale classes
;) quite often ... :P

Besides web frameworks and apps also need specific artifacts (e.g.
Genshi template filters for i18n ...) to fully accomplish these tasks
... ;o)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Comandos : Pipe Viewer ... ?Qu? est? pasando por esta tuber?a?

From tseaver at palladion.com  Mon Mar 30 15:42:43 2009
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 30 Mar 2009 09:42:43 -0400
Subject: [Python-Dev] My summit notes (packaging)
In-Reply-To: <94bdd2610903280724l43910a9am807113e7b19c1ca8@mail.gmail.com>
References: <20090328012305.1B7043A4097@sparrow.telecommunity.com>	<E9C33337-7C4F-4C82-9F78-DC8ADC026E1F@bud.ca>
	<94bdd2610903280724l43910a9am807113e7b19c1ca8@mail.gmail.com>
Message-ID: <gqqi8o$nmp$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tarek Ziad? wrote:
> On Fri, Mar 27, 2009 at 9:54 PM, Kevin Teague <kevin at bud.ca> wrote:
> 
>>
>> Tarek, was there any further discussion on "Requires" vs "install_requires"
>> or any decisions made on what to do about this?
>>
>> (I've got a +1 ready for including 'install_requires' in the standard
>> package metadata and marking 'Requires' as deprecated if that was put forth
>> :P)
> 
> Yes that is what we were thinking of doing. (deprectating Requires and
> Provides and and install_requires)

'Provides' could actually be a potentially useful missing feature in
setuptools, if it meant "virtual distutils project" rather than
"specific module or package."


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ0MxT+gerLs4ltQ4RAgldAJ0Y0arjMeQJYK/G/6pkLqj2WV/Y/gCeL8b9
WnN4ZUWoaZUcFEysEWYsyXE=
=wOf2
-----END PGP SIGNATURE-----


From jnoller at gmail.com  Mon Mar 30 16:44:34 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 30 Mar 2009 09:44:34 -0500
Subject: [Python-Dev] 3to2 Project
Message-ID: <4222a8490903300744t498e79daodea9cff32e4a94c1@mail.gmail.com>

During the Language summit this past Thursday, pretty much everyone
agreed that a python 3 to python 2 tool would be a very large
improvement in helping developers be able to write "pure" python 3
code. The idea being a large project such as Django could completely
cut over to Python3, but then run the 3to2 tool on the code based to
continue to support version 2.x.

I raised my hand to help move this along, I've spoke to Benjamin
Peterson, and he's amendable to mentoring a GSoC student for this
project and he's  already received at least one proposal for this.

Additionally, there's been a number of developers here at PyCon who
are more than ready to help contribute.

So, if people are interested in helping, coordinating work/etc - feel
free to sync up with Benjamin - he's started a wiki page here:

http://wiki.python.org/moin/3to2

I'll help coordinate where I can!

-Jesse

From guido at python.org  Mon Mar 30 16:58:38 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Mar 2009 09:58:38 -0500
Subject: [Python-Dev] And the winner is...
Message-ID: <ca471dc20903300758s47fc922bpacaa4a5fb10a1dac@mail.gmail.com>

Dear Python developers,

The decision is made! I've selected a DVCS to use for Python. We're
switching to Mercurial (Hg).

The implementation and schedule is still up in the air -- I am hoping
that we can switch before the summer.

It's hard to explain my reasons for choosing -- like most language
decisions (especially the difficult ones) it's mostly a matter of gut
feelings. One thing I know is that it's better to decide now than to
spend another year discussing the pros and cons. All that could be
said has been said, pretty much, and my mind is made up.

To me, the advantages of using *some* DVCS are obvious. At PyCon,
Brett already announced that Git was no longer being considered --
while it has obviously many fans, it also provokes strong antipathies.
So it was between Hg and Bzr (both of which happen to be implemented
in Python FWIW). Based on a completely unscientific poll (basically
whatever feedback I received in my personal inbox or on Twitter), Hg
has a strong following among Python developers and few detractors,
while few (except Canonical employees) seem to like Bzr. In addition,
most timing experiments point towards Hg being faster than Bzr for
most operations, and Hg is (again, subjectively) easier to learn for
SVN users than Bzr.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From bill.hoffman at kitware.com  Mon Mar 30 17:50:13 2009
From: bill.hoffman at kitware.com (Bill Hoffman)
Date: Mon, 30 Mar 2009 11:50:13 -0400
Subject: [Python-Dev] CMake Evaluation
Message-ID: <49D0EA35.1030806@kitware.com>

Hi,

I noticed the thread on CMake evaluation, and as a CMake developer I am 
interested in helping you evaluate CMake.

I guess four issues were raised:

 >A. It has no equivalent of autoheader, so we'd have to maintain
pyconfig.h.in by hand.

This is currently true, but if this were a deal breaker, I am sure 
something could be added to CMake to accommodate the need. It might even 
be possible to create a CMake function that does something like this 
without having to modify CMake itself.

 >B. It does not allow the CMakeLists.txt file control the --help
output. This appears to be an intentional decision
(http://www.cmake.org/pipermail/cmake-promote/2006-May/000095.html).
To replace it, they have an interactive mode (which asks you about
each possible option) and a -LH flag (which runs the whole configure
process and then tells you about the flags you could have set if you
knew about them).


CMake now has a cmake-gui (qt based application) for configuring a 
project.

C. It does not allow the CMakeLists.txt file to see the command line,
so we can't stay backward compatible with the configure script, and
we'd have to replace flags like --with-pydebug with
-Dpydebug:bool=true. We could write a separate configure script to
mitigate this and the previous problem, but we'd have to keep that in
sync with CMakeLists.txt manually.

We have talked about adding this to CMake several times, and it is a 
good idea.  However, it has not been a huge issue for many projects.

D. It doesn't have an expression language, so to replace
ac_md_system=`echo $ac_sys_system |
tr -d '[/ ]' | tr '[[A-Z]]' '[[a-z]]'`
you have to write:
string(REGEX REPLACE "[/ ]" "" ac_md_system ${ac_sys_system})
string(TOLOWER ${ac_md_system} ac_md_system)

However, the above CMake script will work on any platform with CMake 
installed, and does not use tr.  Also, the CMake code for the above 
should actually run faster as it is not running two processes with pipe 
communication.

I would not think that this list of issues would be a deal breaker.  The 
benefits of CMake are a single build system that works cross platform. 
Developers can use the native tools that they are used to using (VS IDE, 
Xcode IDE, Eclipse IDE, nmake command line, gmake command line, etc.). 
The original Python CMake work done by Alex, was done to support the 
building of ParaView (www.paraview.org) on a Cray Xt5 computer using 
CMake's cross-compilation support.  This worked very well.  The choice 
is obviously yours to make, but if you have any specific questions about 
CMake, I will lurk on this mailing list and try to answer them.  I think 
it would be a good fit for CMake to build Python, and make things easier 
for some of our customers building ParaView which depends on Python on 
machines like the Cray Xt5.

Thanks.

-Bill

From jcea at jcea.es  Mon Mar 30 18:20:30 2009
From: jcea at jcea.es (Jesus Cea)
Date: Mon, 30 Mar 2009 18:20:30 +0200
Subject: [Python-Dev] And the winner is...
In-Reply-To: <ca471dc20903300758s47fc922bpacaa4a5fb10a1dac@mail.gmail.com>
References: <ca471dc20903300758s47fc922bpacaa4a5fb10a1dac@mail.gmail.com>
Message-ID: <49D0F14E.9030102@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Guido van Rossum wrote:
> The decision is made! I've selected a DVCS to use for Python. We're
> switching to Mercurial (Hg).

Bravo.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSdDxSplgi5GaxT1NAQLacgP9GdVfg1LWpy4UakHrGC1MFMRV3PCZ9HuL
B63k368rX/QOzYc67Y6smzTzKJUFsGCGCUsg70NTIxNwGLJIspBjQ46xKrcNRHYS
nXvzT/WKRz5HgEMEHwDLXMFdsXWBsVAT5ZkiXZeGIa1WnPvWxmhqwJPd105JusqE
BRH5dhg8MbU=
=dnwD
-----END PGP SIGNATURE-----

From jcea at jcea.es  Mon Mar 30 18:48:07 2009
From: jcea at jcea.es (Jesus Cea)
Date: Mon, 30 Mar 2009 18:48:07 +0200
Subject: [Python-Dev] =?iso-8859-15?q?=BFNueva_reuni=F3n_mensual_en_Madrid?=
	=?iso-8859-15?q?=3F?=
Message-ID: <49D0F7C7.8070005@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Va tocando hacer una nueva reuni?n mensual en Madrid. ?Propuestas de
fecha y lugar?.

La ?ltima vez estuvo bastante bien, aunque ?ramos poquitos. ?Hacemos
otra a la vuelta de semana santa?.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSdD3x5lgi5GaxT1NAQI0PgQAoPtOHTNfc5HOqj6YhChWalVt7JHidouh
v779NvzJ4BxepT/g6bs0N59twUX0zkodiRshc/9Hc4zKpXzTFJ3ku4f8yg2KFfL0
jH7A4EvRHrr2xWlcbYtB4qeorPhdRnsuuxfAX09LRdjD9m5VxGso/nVBSt0N5BMd
nni9xP72/gM=
=oInB
-----END PGP SIGNATURE-----

From jcea at jcea.es  Mon Mar 30 19:05:29 2009
From: jcea at jcea.es (Jesus Cea)
Date: Mon, 30 Mar 2009 19:05:29 +0200
Subject: [Python-Dev]
 =?iso-8859-15?q?=BFNueva_reuni=F3n_mensual_en_Madrid?=
 =?iso-8859-15?q?=3F?=
In-Reply-To: <49D0F7C7.8070005@jcea.es>
References: <49D0F7C7.8070005@jcea.es>
Message-ID: <49D0FBD9.2050806@jcea.es>

Excuse me. Wrong mailing list.

From db3l.net at gmail.com  Mon Mar 30 19:30:56 2009
From: db3l.net at gmail.com (David Bolen)
Date: Mon, 30 Mar 2009 13:30:56 -0400
Subject: [Python-Dev] Test failures under Windows?
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net>
	<49C9EEB5.2090804@gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>
	<d2155e360903250553u2e134d61rc005e8817ad82a60@mail.gmail.com>
	<m2ab791r8o.fsf@valheru.db3l.homeip.net>
	<m2tz5cxda3.fsf@valheru.db3l.homeip.net>
	<49D08336.5070907@m2.ccsnet.ne.jp>
Message-ID: <m2ljqmyl2n.fsf@valheru.db3l.homeip.net>

Hirokazu Yamamoto <ocean-city at m2.ccsnet.ne.jp> writes:

> CRT Assertion was totally disabled before, but recently was enabled,
> and workarounds were patched for problematic functions. (ex: fdopen
> and dup) Probably this *patch* is not perfect. See
> http://bugs.python.org/issue4804

Ah - that ticket may explain why my buildbot hadn't been having
problems until recently, even though the -n option in test.bat was
removed quite a while back.

> I'm now +3/4 for the idea disabling assertion by default,
> and enabling by startup option or environment variable. (Or enabling
> by default and disabling by environment variable?)

This is a bit more than I'm looking to have changed.

I'm only discussing disabling the GUI assertions during buildbot test
runs.  It's fine if there are other circumstances when one might also
want a mechanism to disable them, but I'd rather not intermingle such
cases since there may be different pros and cons to different cases.

The nature of the development process is that you're going to have
issues at times which is why you're testing, and given the
consequences of a pop-up box on an unattended build slave, it just
seems very strange to me that there could be any good reason to leave
a potential for such popups (which at this point is easy to disable)
in the buildbot environment.

-- David


From alex.neundorf at kitware.com  Mon Mar 30 19:34:50 2009
From: alex.neundorf at kitware.com (Alexander Neundorf)
Date: Mon, 30 Mar 2009 19:34:50 +0200
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <gqovsd$rnv$1@ger.gmane.org>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
	<gqovsd$rnv$1@ger.gmane.org>
Message-ID: <806d41050903301034i8018472pbdb3f550a1629886@mail.gmail.com>

Hi,


On Mon, Mar 30, 2009 at 1:22 AM, Christian Heimes <lists at cheimes.de> wrote:
> Jeffrey Yasskin wrote:
>
>>  1. It can autogenerate the Visual Studio project files instead of
>> needing them to be maintained separately
>
> I'm familiar with the Unix and the Windows build system. More than a
> year ago I went to a great deal of work to migrate the Windows builds
> from VS 7.1 to VS 9.0. I'm in doubt that any automatic tool can create
> configuration files that are as good as our hand made files.

This of course depends on the definition of "as good as" ;-)
Well, I have met Windows-only developers which use CMake because it is
able to generate project files for different versions of Visual
Studio, and praise it for that.

> The VS project files support debug, non debug and profile guided
> optimization builds for X86 and AMD64 including cross compilation of
> AMD64 binaries.

CMake supports different build configurations.

> The project files are using multiple inheritance to
> avoid duplication of options.

No idea. Maybe it wouldn't be necessary ? With CMake you can just
generate different buildtrees with different options, so you can get
different behaviour in these trees.

> The differences between Windows and Unix builds are fairly large, too.
> On Windows lots of modules are built in and the remaining Python
> extensions are build with VS directly. On Unix most modules are build as
> shared libraries using distutils and setup.py.

That's right. Is there actually a real reason for this ?
When I posted the cmake files for python in 2007 I think they also
worked for Windows,, but I didn't test this that much.

> In my opinion any change to an automated system is a waste of precious
> developer times and makes our Windows support worse.

Seriously, I don't think so.
In KDE, our (small group of) Windows developers are the ones which
appreciate CMake most.
At Kitware, support for Windows and MSVC is a primary feature of
CMake, not an afterthought.

Alex

From alex.neundorf at kitware.com  Mon Mar 30 19:37:17 2009
From: alex.neundorf at kitware.com (Alexander Neundorf)
Date: Mon, 30 Mar 2009 19:37:17 +0200
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <50862ebd0903291509g60c988feme0297d7d4e975129@mail.gmail.com>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
	<50862ebd0903291509g60c988feme0297d7d4e975129@mail.gmail.com>
Message-ID: <806d41050903301037k1f38a20bkb9faa98fc8a26a70@mail.gmail.com>

On Mon, Mar 30, 2009 at 12:09 AM, Neil Hodgson <nyamatongwe at gmail.com> wrote:
...
> while so I can't remember the details. The current Python project
> files are hierarchical, building several DLLs and an EXE and I think
> this was outside the scope of the tools I looked at.

Not sure I understand.
Having a project which builds (shared) libraries and executables which
use them (and which maybe have to be executed later on during the
build) is no problem for CMake, also with the VisualStudio projects.
>From what I remember when I wrote the CMake files for python it was
quite straight forward.

Alex

From collinw at gmail.com  Mon Mar 30 19:37:47 2009
From: collinw at gmail.com (Collin Winter)
Date: Mon, 30 Mar 2009 10:37:47 -0700
Subject: [Python-Dev] 3to2 Project
In-Reply-To: <4222a8490903300744t498e79daodea9cff32e4a94c1@mail.gmail.com>
References: <4222a8490903300744t498e79daodea9cff32e4a94c1@mail.gmail.com>
Message-ID: <43aa6ff70903301037y215d979he36246d36c987493@mail.gmail.com>

On Mon, Mar 30, 2009 at 7:44 AM, Jesse Noller <jnoller at gmail.com> wrote:
> During the Language summit this past Thursday, pretty much everyone
> agreed that a python 3 to python 2 tool would be a very large
> improvement in helping developers be able to write "pure" python 3
> code. The idea being a large project such as Django could completely
> cut over to Python3, but then run the 3to2 tool on the code based to
> continue to support version 2.x.
>
> I raised my hand to help move this along, I've spoke to Benjamin
> Peterson, and he's amendable to mentoring a GSoC student for this
> project and he's ?already received at least one proposal for this.
>
> Additionally, there's been a number of developers here at PyCon who
> are more than ready to help contribute.
>
> So, if people are interested in helping, coordinating work/etc - feel
> free to sync up with Benjamin - he's started a wiki page here:
>
> http://wiki.python.org/moin/3to2

If anyone is interested in working on this during the PyCon sprints or
otherwise, here are some easy, concrete starter projects that would
really help move this along:
- The core refactoring engine needs to be broken out from 2to3. In
particular, the tests/ and fixes/ need to get pulled up a directory,
out of lib2to3/.
- Once that's done, lib2to3 should then be renamed to something like
librefactor or something else that indicates its more general nature.
This will allow both 2to3 and 3to2 to more easily share the core
components.
- If you're more performance-minded, 2to3 and 3to2 would benefit
heavily from some work on the pattern matching system. The current
pattern matcher is a fairly simple AST interpreter; compiling the
patterns down to pure Python code would be a win, I believe. This is
all pretty heavily tested, so you wouldn't run much risk of breaking
it.

Collin

From rrr at ronadam.com  Mon Mar 30 19:54:42 2009
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 30 Mar 2009 12:54:42 -0500
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <20090327170154.AC1753A40A7@sparrow.telecommunity.com>
References: <49C48D03.7030709@gmail.com>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	<49C966A0.6090401@canterbury.ac.nz>	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>	<49C9C947.3080608@canterbury.ac.nz>	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>	<loom.20090326T102716-650@post.gmane.org>	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>	<20090326211646.3392B3A4097@sparrow.telecommunity.com>	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>	<20090327041138.EEF893A4097@sparrow.telecommunity.com>	<49CC8037.1010909@scottdial.com>
	<20090327170154.AC1753A40A7@sparrow.telecommunity.com>
Message-ID: <49D10762.2000607@ronadam.com>



P.J. Eby wrote:

> Sure.  But right now, the return value of a generator function *is the 
> generator*.  And you're free to ignore that, sure.
> 
> But this is a "second" return value that only goes to a special place 
> with special syntax -- without that syntax, you can't access it.
> 
> But in the use cases where you'd actually want to make such a function 
> return a value to begin with, it's because that value is the value you 
> *really* want from the function -- the only reason it's a generator is 
> because it needs to be paused and resumed along the way to getting that 
> return value.


How about if 'yield from' returns the generator object, and the return 
value is accessed with an attribute.

    g = yield from gen
    x = g.__value__

Or

    x = (yield from gen).__value__



Another possibility is to be able to break from a 'yield from' at some 
point and then continue it to get any final values.

    # yield values of sub generator
    g = yield from gen

    # get remaining unused value of sub generator
    x = g.next()












From cournape at gmail.com  Mon Mar 30 20:04:26 2009
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 31 Mar 2009 03:04:26 +0900
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <806d41050903301037k1f38a20bkb9faa98fc8a26a70@mail.gmail.com>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
	<50862ebd0903291509g60c988feme0297d7d4e975129@mail.gmail.com>
	<806d41050903301037k1f38a20bkb9faa98fc8a26a70@mail.gmail.com>
Message-ID: <5b8d13220903301104v39262171v55f27ac6029ac5a5@mail.gmail.com>

On Tue, Mar 31, 2009 at 2:37 AM, Alexander Neundorf
<alex.neundorf at kitware.com> wrote:
> On Mon, Mar 30, 2009 at 12:09 AM, Neil Hodgson <nyamatongwe at gmail.com> wrote:
> ...
>> while so I can't remember the details. The current Python project
>> files are hierarchical, building several DLLs and an EXE and I think
>> this was outside the scope of the tools I looked at.
>
> Not sure I understand.
> Having a project which builds (shared) libraries and executables which
> use them (and which maybe have to be executed later on during the
> build) is no problem for CMake, also with the VisualStudio projects.
> >From what I remember when I wrote the CMake files for python it was
> quite straight forward.

I think Christian meant that since on windows, those are built with
visual studio project files, but everywhere else, it is built with
distutils, you can't use a common system without first converting
everything to cmake for all the other platforms.

Also, when converting a project from one build system to another,
doing the 80 % takes 20 % in my experience. The most time consuming
part is all small the details on not so common platforms.

David

From jnoller at gmail.com  Mon Mar 30 20:05:31 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 30 Mar 2009 13:05:31 -0500
Subject: [Python-Dev] 3to2 Project
In-Reply-To: <43aa6ff70903301037y215d979he36246d36c987493@mail.gmail.com>
References: <4222a8490903300744t498e79daodea9cff32e4a94c1@mail.gmail.com>
	<43aa6ff70903301037y215d979he36246d36c987493@mail.gmail.com>
Message-ID: <4222a8490903301105j43bf38c9s2a0a2a682960c12@mail.gmail.com>

On Mon, Mar 30, 2009 at 12:37 PM, Collin Winter <collinw at gmail.com> wrote:
> On Mon, Mar 30, 2009 at 7:44 AM, Jesse Noller <jnoller at gmail.com> wrote:
>> During the Language summit this past Thursday, pretty much everyone
>> agreed that a python 3 to python 2 tool would be a very large
>> improvement in helping developers be able to write "pure" python 3
>> code. The idea being a large project such as Django could completely
>> cut over to Python3, but then run the 3to2 tool on the code based to
>> continue to support version 2.x.
>>
>> I raised my hand to help move this along, I've spoke to Benjamin
>> Peterson, and he's amendable to mentoring a GSoC student for this
>> project and he's ?already received at least one proposal for this.
>>
>> Additionally, there's been a number of developers here at PyCon who
>> are more than ready to help contribute.
>>
>> So, if people are interested in helping, coordinating work/etc - feel
>> free to sync up with Benjamin - he's started a wiki page here:
>>
>> http://wiki.python.org/moin/3to2
>
> If anyone is interested in working on this during the PyCon sprints or
> otherwise, here are some easy, concrete starter projects that would
> really help move this along:
> - The core refactoring engine needs to be broken out from 2to3. In
> particular, the tests/ and fixes/ need to get pulled up a directory,
> out of lib2to3/.
> - Once that's done, lib2to3 should then be renamed to something like
> librefactor or something else that indicates its more general nature.
> This will allow both 2to3 and 3to2 to more easily share the core
> components.
> - If you're more performance-minded, 2to3 and 3to2 would benefit
> heavily from some work on the pattern matching system. The current
> pattern matcher is a fairly simple AST interpreter; compiling the
> patterns down to pure Python code would be a win, I believe. This is
> all pretty heavily tested, so you wouldn't run much risk of breaking
> it.
>

A second note on performance, is that Benjamin was about to checkin a
version of 3to2 which used multiprocessing to help speed it up a bit -
he had to catch a plane, so he should have it done some time shortly.

-jesse

From rrr at ronadam.com  Mon Mar 30 20:06:50 2009
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 30 Mar 2009 13:06:50 -0500
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49D10762.2000607@ronadam.com>
References: <49C48D03.7030709@gmail.com>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	<49C966A0.6090401@canterbury.ac.nz>	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>	<49C9C947.3080608@canterbury.ac.nz>	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>	<loom.20090326T102716-650@post.gmane.org>	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>	<20090326211646.3392B3A4097@sparrow.telecommunity.com>	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>	<20090327041138.EEF893A4097@sparrow.telecommunity.com>	<49CC8037.1010909@scottdial.com>	<20090327170154.AC1753A40A7@sparrow.telecommunity.com>
	<49D10762.2000607@ronadam.com>
Message-ID: <49D10A3A.90509@ronadam.com>



Ron Adam wrote:
> 
> 
> P.J. Eby wrote:
> 
>> Sure.  But right now, the return value of a generator function *is the 
>> generator*.  And you're free to ignore that, sure.
>>
>> But this is a "second" return value that only goes to a special place 
>> with special syntax -- without that syntax, you can't access it.
>>
>> But in the use cases where you'd actually want to make such a function 
>> return a value to begin with, it's because that value is the value you 
>> *really* want from the function -- the only reason it's a generator is 
>> because it needs to be paused and resumed along the way to getting 
>> that return value.
> 
> 
> How about if 'yield from' returns the generator object, and the return 
> value is accessed with an attribute.
> 
>    g = yield from gen
>    x = g.__value__
> 
> Or
> 
>    x = (yield from gen).__value__
> 
> 
> 
> Another possibility is to be able to break from a 'yield from' at some 
> point and then continue it to get any final values.
> 
>    # yield values of sub generator
>    g = yield from gen
> 
>    # get remaining unused value of sub generator
>    x = g.next()
> 

This could possibly be done in one line as well...

      x = (yield from gen).next()



From rrr at ronadam.com  Mon Mar 30 20:06:50 2009
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 30 Mar 2009 13:06:50 -0500
Subject: [Python-Dev] PEP 380 (yield from a subgenerator) comments
In-Reply-To: <49D10762.2000607@ronadam.com>
References: <49C48D03.7030709@gmail.com>	<ca471dc20903241344r71c64179tb6ce4b9b4131fd7f@mail.gmail.com>	<49C966A0.6090401@canterbury.ac.nz>	<ca471dc20903241623g686a7f15h46dd1b7819339184@mail.gmail.com>	<49C9C947.3080608@canterbury.ac.nz>	<ca471dc20903252139k3887d63en59a838113c96a14@mail.gmail.com>	<loom.20090326T102716-650@post.gmane.org>	<20090326171640.EC27D3A40A5@sparrow.telecommunity.com>	<ca471dc20903261227u946f42bma094177c46b12cdf@mail.gmail.com>	<20090326211646.3392B3A4097@sparrow.telecommunity.com>	<ca471dc20903262039t4f37cc21j4265f722a790fe29@mail.gmail.com>	<20090327041138.EEF893A4097@sparrow.telecommunity.com>	<49CC8037.1010909@scottdial.com>	<20090327170154.AC1753A40A7@sparrow.telecommunity.com>
	<49D10762.2000607@ronadam.com>
Message-ID: <49D10A3A.90509@ronadam.com>



Ron Adam wrote:
> 
> 
> P.J. Eby wrote:
> 
>> Sure.  But right now, the return value of a generator function *is the 
>> generator*.  And you're free to ignore that, sure.
>>
>> But this is a "second" return value that only goes to a special place 
>> with special syntax -- without that syntax, you can't access it.
>>
>> But in the use cases where you'd actually want to make such a function 
>> return a value to begin with, it's because that value is the value you 
>> *really* want from the function -- the only reason it's a generator is 
>> because it needs to be paused and resumed along the way to getting 
>> that return value.
> 
> 
> How about if 'yield from' returns the generator object, and the return 
> value is accessed with an attribute.
> 
>    g = yield from gen
>    x = g.__value__
> 
> Or
> 
>    x = (yield from gen).__value__
> 
> 
> 
> Another possibility is to be able to break from a 'yield from' at some 
> point and then continue it to get any final values.
> 
>    # yield values of sub generator
>    g = yield from gen
> 
>    # get remaining unused value of sub generator
>    x = g.next()
> 

This could possibly be done in one line as well...

      x = (yield from gen).next()



From barry at python.org  Mon Mar 30 20:16:00 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 30 Mar 2009 13:16:00 -0500
Subject: [Python-Dev] Python 3.0.2
Message-ID: <81FD1C74-1283-4BFB-A19F-34C3879BD86D@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

We made a decision at the sprints today about Python 3.0.  We've  
agreed that there will be one more release, 3.0.2 and then that's it.   
Because of the earlier decision to drop all support for Python 3.0  
once 3.1 is released, we won't be doing any more releases after that.

I don't have an ETA for 3.0.2, but I'm happy to do it whenever it  
makes sense.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSdEMYXEjvBPtnXfVAQLNLQP/XY9Wv3QvoqvEhgberz45inXqMpiMuhAL
knw+tjtpwtizim4yXFR4EQxa0OhsAgqMPCe+m4CDTGSqcasQAa5iRkk5He5h0Z2C
gBP3/DxfECb84r1aDuP4t1wCjqPmNq2T7kIiB7rTRTZktma/1vL2NH3zA4AErQnf
b/wjc4zIExg=
=kbT0
-----END PGP SIGNATURE-----

From alex.neundorf at kitware.com  Mon Mar 30 20:16:06 2009
From: alex.neundorf at kitware.com (Alexander Neundorf)
Date: Mon, 30 Mar 2009 20:16:06 +0200
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <5b8d13220903301104v39262171v55f27ac6029ac5a5@mail.gmail.com>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
	<50862ebd0903291509g60c988feme0297d7d4e975129@mail.gmail.com>
	<806d41050903301037k1f38a20bkb9faa98fc8a26a70@mail.gmail.com>
	<5b8d13220903301104v39262171v55f27ac6029ac5a5@mail.gmail.com>
Message-ID: <806d41050903301116w380b2c68m45201d3682a9469a@mail.gmail.com>

On Mon, Mar 30, 2009 at 8:04 PM, David Cournapeau <cournape at gmail.com> wrote:
> On Tue, Mar 31, 2009 at 2:37 AM, Alexander Neundorf
> <alex.neundorf at kitware.com> wrote:
...
>> Not sure I understand.
>> Having a project which builds (shared) libraries and executables which
>> use them (and which maybe have to be executed later on during the
>> build) is no problem for CMake, also with the VisualStudio projects.
>> >From what I remember when I wrote the CMake files for python it was
>> quite straight forward.
>
> I think Christian meant that since on windows, those are built with
> visual studio project files, but everywhere else, it is built with
> distutils, you can't use a common system without first converting
> everything to cmake for all the other platforms.

Can you please explain ? What is "those" ?

> Also, when converting a project from one build system to another,
> doing the 80 % takes 20 % in my experience.

Getting it working took me like 2 days, if that's 20% it's not too bad ;-)

> The most time consuming
> part is all small the details on not so common platforms.

Yes.

Alex

From cournape at gmail.com  Mon Mar 30 20:26:11 2009
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 31 Mar 2009 03:26:11 +0900
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <806d41050903301116w380b2c68m45201d3682a9469a@mail.gmail.com>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
	<50862ebd0903291509g60c988feme0297d7d4e975129@mail.gmail.com>
	<806d41050903301037k1f38a20bkb9faa98fc8a26a70@mail.gmail.com>
	<5b8d13220903301104v39262171v55f27ac6029ac5a5@mail.gmail.com>
	<806d41050903301116w380b2c68m45201d3682a9469a@mail.gmail.com>
Message-ID: <5b8d13220903301126m21c440bete4729b56978ceebe@mail.gmail.com>

On Tue, Mar 31, 2009 at 3:16 AM, Alexander Neundorf
<alex.neundorf at kitware.com> wrote:
>
> Can you please explain ? What is "those" ?

Everything in Lib. On windows, I believe this is done through project
files, but on linux at least, and I guess on most other OS, those are
handled by distutils. I guess the lack of autoconf on windows is one
reason for this difference ?

>
>> Also, when converting a project from one build system to another,
>> doing the 80 % takes 20 % in my experience.
>
> Getting it working took me like 2 days, if that's 20% it's not too bad ;-)

So it means ten days of work to convert to a new system that maybe
most python maintainers do not know. What does it bring ?

I think supporting cross compilation would be more worthwhile, for
example, in the build department.

cheers,

David

From lists at cheimes.de  Mon Mar 30 20:34:21 2009
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 30 Mar 2009 20:34:21 +0200
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <806d41050903301034i8018472pbdb3f550a1629886@mail.gmail.com>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>	<gqovsd$rnv$1@ger.gmane.org>
	<806d41050903301034i8018472pbdb3f550a1629886@mail.gmail.com>
Message-ID: <49D110AD.3080703@cheimes.de>

Hallo Alexander!

Alexander Neundorf wrote:
> This of course depends on the definition of "as good as" ;-)
> Well, I have met Windows-only developers which use CMake because it is
> able to generate project files for different versions of Visual
> Studio, and praise it for that.

So far I haven't heard any complains about or feature requests for the
project files. ;) Most Windows related issue where about the SxS
assembly issues with the new MSVCRT library.

The multitude of project files formats are an issue for most open source
libraries. Each minor version of Python has only one supported version
of Visual Studio due to possible issues with mixed MSVCRTs. 99% of all
Windows users are using the official MSI package provided by Martin. We
also have project files for VS 6.0, 7.1 and 8.0 in the PC/ subdirectory.
The VS 8.0 are created from the VS 9.0 project files. The remaining
projects are maintained manually.

>> The VS project files support debug, non debug and profile guided
>> optimization builds for X86 and AMD64 including cross compilation of
>> AMD64 binaries.
> 
> CMake supports different build configurations.

Does it support all eight build configurations? (Debug, Release, PGO
instrument, PGO update for X86 and X64)

> No idea. Maybe it wouldn't be necessary ? With CMake you can just
> generate different buildtrees with different options, so you can get
> different behaviour in these trees.

I don't know, too.

> That's right. Is there actually a real reason for this ?
> When I posted the cmake files for python in 2007 I think they also
> worked for Windows,, but I didn't test this that much.

As far as I remember DLL take considerable more time to load than shared
libraries on Unix OSes. The VS project driven build also makes it
possible to cross compile builds for Itanium and AMD64 platforms on a
X86 Windows. Distutils doesn't support cross compilation so far.
You have to ask Martin von L?wis and Mark Hammond for detailed information.

>> In my opinion any change to an automated system is a waste of precious
>> developer times and makes our Windows support worse.
> 
> Seriously, I don't think so.
> In KDE, our (small group of) Windows developers are the ones which
> appreciate CMake most.
> At Kitware, support for Windows and MSVC is a primary feature of
> CMake, not an afterthought.

Please understand my previous mail in the context of Python core
development. For KDE it makes perfectly sense to use cmake.
For Python on Windows I just don't see any relevant benefits. I'm not
against a CMake approach if somebody else is able to pull it off. So +0
from me.

Christian

From lists at cheimes.de  Mon Mar 30 20:49:19 2009
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 30 Mar 2009 20:49:19 +0200
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <5b8d13220903301126m21c440bete4729b56978ceebe@mail.gmail.com>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>	<50862ebd0903291509g60c988feme0297d7d4e975129@mail.gmail.com>	<806d41050903301037k1f38a20bkb9faa98fc8a26a70@mail.gmail.com>	<5b8d13220903301104v39262171v55f27ac6029ac5a5@mail.gmail.com>	<806d41050903301116w380b2c68m45201d3682a9469a@mail.gmail.com>
	<5b8d13220903301126m21c440bete4729b56978ceebe@mail.gmail.com>
Message-ID: <49D1142F.5090509@cheimes.de>

David Cournapeau wrote:
> On Tue, Mar 31, 2009 at 3:16 AM, Alexander Neundorf
> <alex.neundorf at kitware.com> wrote:
>> Can you please explain ? What is "those" ?
> 
> Everything in Lib. On windows, I believe this is done through project
> files, but on linux at least, and I guess on most other OS, those are
> handled by distutils. I guess the lack of autoconf on windows is one
> reason for this difference ?

All modules under Modules/ and PC/ are build inside the VS project. Some
dependencies like gzip, bzip2, openssl etc. are build inside the
project, too. Other dependencies are using shell scripts and nmake.

On Unix the builtin modules are compiled with a custom build system
(Modules/Setup.* and friends). The shared libraries are build by a
distutils script (setup.py) in the root folder of the Python distribution.

I already explained the reasons for the Windows build in a previous mail
about 15 minutes ago.

Christian

From kippesp at gmail.com  Mon Mar 30 20:16:11 2009
From: kippesp at gmail.com (Paul Kippes)
Date: Mon, 30 Mar 2009 13:16:11 -0500
Subject: [Python-Dev] 3to2 Project
In-Reply-To: <4222a8490903301105j43bf38c9s2a0a2a682960c12@mail.gmail.com>
References: <4222a8490903300744t498e79daodea9cff32e4a94c1@mail.gmail.com>
	<43aa6ff70903301037y215d979he36246d36c987493@mail.gmail.com>
	<4222a8490903301105j43bf38c9s2a0a2a682960c12@mail.gmail.com>
Message-ID: <c89e39d50903301116y285b0adahd3840d0a1ea825a0@mail.gmail.com>

Ron Duplain and I have been working on this at the sprint.  We're
working on getting the necessary 3to2 fixes in place and verified
first since dropping this in was quite simple.  The fixes don't look
to be too involved which should provide time for the recommended
refactorings.

Paul

On Mon, Mar 30, 2009 at 1:05 PM, Jesse Noller <jnoller at gmail.com> wrote:
> On Mon, Mar 30, 2009 at 12:37 PM, Collin Winter <collinw at gmail.com> wrote:
>> On Mon, Mar 30, 2009 at 7:44 AM, Jesse Noller <jnoller at gmail.com> wrote:
>>> During the Language summit this past Thursday, pretty much everyone
>>> agreed that a python 3 to python 2 tool would be a very large
>>> improvement in helping developers be able to write "pure" python 3
>>> code. The idea being a large project such as Django could completely
>>> cut over to Python3, but then run the 3to2 tool on the code based to
>>> continue to support version 2.x.
>>>
>>> I raised my hand to help move this along, I've spoke to Benjamin
>>> Peterson, and he's amendable to mentoring a GSoC student for this
>>> project and he's ?already received at least one proposal for this.
>>>
>>> Additionally, there's been a number of developers here at PyCon who
>>> are more than ready to help contribute.
>>>
>>> So, if people are interested in helping, coordinating work/etc - feel
>>> free to sync up with Benjamin - he's started a wiki page here:
>>>
>>> http://wiki.python.org/moin/3to2
>>
>> If anyone is interested in working on this during the PyCon sprints or
>> otherwise, here are some easy, concrete starter projects that would
>> really help move this along:
>> - The core refactoring engine needs to be broken out from 2to3. In
>> particular, the tests/ and fixes/ need to get pulled up a directory,
>> out of lib2to3/.
>> - Once that's done, lib2to3 should then be renamed to something like
>> librefactor or something else that indicates its more general nature.
>> This will allow both 2to3 and 3to2 to more easily share the core
>> components.
>> - If you're more performance-minded, 2to3 and 3to2 would benefit
>> heavily from some work on the pattern matching system. The current
>> pattern matcher is a fairly simple AST interpreter; compiling the
>> patterns down to pure Python code would be a win, I believe. This is
>> all pretty heavily tested, so you wouldn't run much risk of breaking
>> it.
>>
>
> A second note on performance, is that Benjamin was about to checkin a
> version of 3to2 which used multiprocessing to help speed it up a bit -
> he had to catch a plane, so he should have it done some time shortly.
>
> -jesse
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/kippesp%2Bpythondev%40gmail.com
>

From tjreedy at udel.edu  Mon Mar 30 21:02:53 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 30 Mar 2009 15:02:53 -0400
Subject: [Python-Dev] Python 3.0.2
In-Reply-To: <81FD1C74-1283-4BFB-A19F-34C3879BD86D@python.org>
References: <81FD1C74-1283-4BFB-A19F-34C3879BD86D@python.org>
Message-ID: <gqr50o$285$1@ger.gmane.org>

Barry Warsaw wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> We made a decision at the sprints today about Python 3.0.  We've agreed 
> that there will be one more release, 3.0.2 and then that's it.  Because 
> of the earlier decision to drop all support for Python 3.0 once 3.1 is 
> released, we won't be doing any more releases after that.
> 
> I don't have an ETA for 3.0.2, but I'm happy to do it whenever it makes 
> sense.

Perhaps midway between 3.0.1 and 3.1, which would likely be late April.


From alex.neundorf at kitware.com  Mon Mar 30 21:36:01 2009
From: alex.neundorf at kitware.com (Alexander Neundorf)
Date: Mon, 30 Mar 2009 21:36:01 +0200
Subject: [Python-Dev] Evaluated cmake as an autoconf replacement
In-Reply-To: <49D1142F.5090509@cheimes.de>
References: <5d44f72f0903291021u352e9864x5db3c3f9d7b32b76@mail.gmail.com>
	<50862ebd0903291509g60c988feme0297d7d4e975129@mail.gmail.com>
	<806d41050903301037k1f38a20bkb9faa98fc8a26a70@mail.gmail.com>
	<5b8d13220903301104v39262171v55f27ac6029ac5a5@mail.gmail.com>
	<806d41050903301116w380b2c68m45201d3682a9469a@mail.gmail.com>
	<5b8d13220903301126m21c440bete4729b56978ceebe@mail.gmail.com>
	<49D1142F.5090509@cheimes.de>
Message-ID: <806d41050903301236r490e4324s3df7a2a6e41140f@mail.gmail.com>

On Mon, Mar 30, 2009 at 8:49 PM, Christian Heimes <lists at cheimes.de> wrote:
> David Cournapeau wrote:
>> On Tue, Mar 31, 2009 at 3:16 AM, Alexander Neundorf
>> <alex.neundorf at kitware.com> wrote:
>>> Can you please explain ? What is "those" ?
>>
>> Everything in Lib. On windows, I believe this is done through project
>> files, but on linux at least, and I guess on most other OS, those are
>> handled by distutils. I guess the lack of autoconf on windows is one
>> reason for this difference ?
>
> All modules under Modules/ and PC/ are build inside the VS project. Some
> dependencies like gzip, bzip2, openssl etc. are build inside the
> project, too. Other dependencies are using shell scripts and nmake.

AFAIK we also have cmake-based builds for gzip, bzip2 and other
dependencies KDE needs somewhere.

> On Unix the builtin modules are compiled with a custom build system
> (Modules/Setup.* and friends). The shared libraries are build by a
> distutils script (setup.py) in the root folder of the Python distribution.

With CMake I was also building the Modules directly with CMake, i.e.
not using distutils, so these were already taken care of.

In Lib/ there are only python files, and they should be compiled to
pyc files, right ? I didn't do this back then.
We are right now getting better support for this in KDE, which would
probably help here:
http://lists.kde.org/?l=kde-buildsystem&m=123795136609498&w=2

Alex

From brett at python.org  Mon Mar 30 23:40:52 2009
From: brett at python.org (Brett Cannon)
Date: Mon, 30 Mar 2009 14:40:52 -0700
Subject: [Python-Dev] Python 3.0.2
In-Reply-To: <gqr50o$285$1@ger.gmane.org>
References: <81FD1C74-1283-4BFB-A19F-34C3879BD86D@python.org> 
	<gqr50o$285$1@ger.gmane.org>
Message-ID: <bbaeab100903301440v51e51495w9a347e9ff0fd6b77@mail.gmail.com>

On Mon, Mar 30, 2009 at 12:02, Terry Reedy <tjreedy at udel.edu> wrote:

> Barry Warsaw wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> We made a decision at the sprints today about Python 3.0.  We've agreed
>> that there will be one more release, 3.0.2 and then that's it.  Because of
>> the earlier decision to drop all support for Python 3.0 once 3.1 is
>> released, we won't be doing any more releases after that.
>>
>> I don't have an ETA for 3.0.2, but I'm happy to do it whenever it makes
>> sense.
>>
>
> Perhaps midway between 3.0.1 and 3.1, which would likely be late April.


I would think that after 3.1 would be best to get any and all fixes that go
into 3.1.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090330/69c3e04b/attachment.htm>

From barry at python.org  Mon Mar 30 23:51:11 2009
From: barry at python.org (Barry Warsaw)
Date: Mon, 30 Mar 2009 16:51:11 -0500
Subject: [Python-Dev] Python 3.0.2
In-Reply-To: <bbaeab100903301440v51e51495w9a347e9ff0fd6b77@mail.gmail.com>
References: <81FD1C74-1283-4BFB-A19F-34C3879BD86D@python.org>
	<gqr50o$285$1@ger.gmane.org>
	<bbaeab100903301440v51e51495w9a347e9ff0fd6b77@mail.gmail.com>
Message-ID: <3ACA4638-C03A-4396-85F2-F5BEC8559F0B@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mar 30, 2009, at 4:40 PM, Brett Cannon wrote:

> I would think that after 3.1 would be best to get any and all fixes  
> that go into 3.1.

That fits nicely with our general policy, so that the only difference  
with 3.0 is that we won't do any security-only releases after the  
final patch release.

Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSdE+0HEjvBPtnXfVAQL4AQQAnYyOeFlVfitJvsJF5gPXJbz5D2BqH9OA
2B9lEuUpVBooaZpsu/CHHgTCUYAVsrqvVBYbRCwsfsvy1mMet7503703vCaJbzxc
MyEBzaHllX6q8froEDc26tQC8aYUs2/kLUL9C9O/inhLAN/i/9HQI9keQ1oP5ubi
JTM5Ryp1nVc=
=2VJ8
-----END PGP SIGNATURE-----

From guido at python.org  Tue Mar 31 01:17:53 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Mar 2009 18:17:53 -0500
Subject: [Python-Dev] Broken import?
In-Reply-To: <ca471dc20903301544l3749c80i3d166fbb8780f502@mail.gmail.com>
References: <ca471dc20903301544l3749c80i3d166fbb8780f502@mail.gmail.com>
Message-ID: <ca471dc20903301617ydbb4bc9pfb221c2cba8df7e4@mail.gmail.com>

[Adding python-dev. I'm quoting the entire original message.]

> On Thu, Mar 19, 2009 at 6:40 PM, Fredrik Lundh <fredriklundh at google.com> wrote:
>> PS. Is it just me, or is import broken in 3.0? ?Consider this:
>>
>> $ more package\a.py
>> print("in a")
>>
>> import b
>>
>> def a():
>> ? ?print("here")
>>
>> def main():
>> ? ?b.b()
>>
>> $ more package\b.py
>> print("in b")
>>
>> import a
>>
>> def b():
>> ? ?a.a()
>>
>> Under 2.X, this prints "in a" "in b" and "here", as expected. ?Under
>> 3.0, using the "from . import" form for relative imports, it bombs out
>> with a:
>>
>> in a
>> in b
>> Traceback (most recent call last):
>> ?File "main.py", line 1, in <module>
>> ? ?from package import a
>> ?File "package/a.py", line 3, in <module>
>> ? ?from . import b
>> ?File "package\b.py", line 3, in <module>
>> ? ?from . import a
>> ImportError: cannot import name a
>>
>> Sure, it's a recursive import, but there's no recursive dependency
>> here - nobody will access the module contents until the main program
>> calls the library. ?What am I missing?

On Mon, Mar 30, 2009 at 5:44 PM, Guido van Rossum <guido at python.org> wrote:
> I reproduced this, but it seems to have more to do with "from . import
> ..." than with the Python version. If I add the "from ." before each
> of the imports, "python -c 'import p.a' " fails with roughly the above
> traceback for any version of Python that supports this syntax, while
> without that it passes for any 2.x.
>
> If I use the "from ." syntax in a.py but not in b.py, "import p.a"
> passes but "import p.b" fails.
>
> I'll see if anyone present at the sprints has a clue.

Made some progress. Anything using "from <whatever> import b" (where
<whatever> is either '.' or 'p') will fail when b's import is not
completed. OTOH using "import p.b" works. I reduced it to:

p/a.py == "from p import b"
p/b.py == "import a"
python -c "import p.b"

The reason seems to be that until the outermost import (in this case
p.b) is completed, while sys.modules has the (incomplete) modules 'p',
'p.a' and 'p.b', the attributes p.a and p.b aren't added until after
their import is completed. Which it isn't during recursive import.
Apparently 'from <anything> import <something>' looks for the
<something> attribute in the <parent> object. This is because
"from...import" can also be used to import objects other than modules
(e.g. "from M import C"). I'm guessing that setting the attribute is
delayed until the import is totally complete, because upon a failed
import we remove the half-imported module object from sys.modules, but
apparently we didn 't want to be in the business of removing the
attribute from the parent package, so that's only set after the import
is deemed successful.

At least, this is my hypothesis, thinking about it -- I might look at
the code later. :-)

The most portable solution is to avoid "from...import" and instead
write something like

import p.b as b

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From murman at gmail.com  Tue Mar 31 01:26:31 2009
From: murman at gmail.com (Michael Urman)
Date: Mon, 30 Mar 2009 18:26:31 -0500
Subject: [Python-Dev] And the winner is...
In-Reply-To: <ca471dc20903300758s47fc922bpacaa4a5fb10a1dac@mail.gmail.com>
References: <ca471dc20903300758s47fc922bpacaa4a5fb10a1dac@mail.gmail.com>
Message-ID: <dcbbbb410903301626x3d707b49pa8184a765e6fcb59@mail.gmail.com>

> We're switching to Mercurial (Hg).

And two hours later, GNOME announces their migration to git is
underway. I'd suspect a series of April Fools jokes, if it weren't two
days early. :)

-- 
Michael Urman

From guido at python.org  Tue Mar 31 01:52:36 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Mar 2009 18:52:36 -0500
Subject: [Python-Dev] Broken import?
In-Reply-To: <ca471dc20903301617ydbb4bc9pfb221c2cba8df7e4@mail.gmail.com>
References: <ca471dc20903301544l3749c80i3d166fbb8780f502@mail.gmail.com>
	<ca471dc20903301617ydbb4bc9pfb221c2cba8df7e4@mail.gmail.com>
Message-ID: <ca471dc20903301652w63dac59dn3275620c3b220fd3@mail.gmail.com>

On Mon, Mar 30, 2009 at 6:17 PM, Guido van Rossum <guido at python.org> wrote:
> [Adding python-dev. I'm quoting the entire original message.]
>
>> On Thu, Mar 19, 2009 at 6:40 PM, Fredrik Lundh <fredriklundh at google.com> wrote:
>>> PS. Is it just me, or is import broken in 3.0? ?Consider this:
>>>
>>> $ more package\a.py
>>> print("in a")
>>>
>>> import b
>>>
>>> def a():
>>> ? ?print("here")
>>>
>>> def main():
>>> ? ?b.b()
>>>
>>> $ more package\b.py
>>> print("in b")
>>>
>>> import a
>>>
>>> def b():
>>> ? ?a.a()
>>>
>>> Under 2.X, this prints "in a" "in b" and "here", as expected. ?Under
>>> 3.0, using the "from . import" form for relative imports, it bombs out
>>> with a:
>>>
>>> in a
>>> in b
>>> Traceback (most recent call last):
>>> ?File "main.py", line 1, in <module>
>>> ? ?from package import a
>>> ?File "package/a.py", line 3, in <module>
>>> ? ?from . import b
>>> ?File "package\b.py", line 3, in <module>
>>> ? ?from . import a
>>> ImportError: cannot import name a
>>>
>>> Sure, it's a recursive import, but there's no recursive dependency
>>> here - nobody will access the module contents until the main program
>>> calls the library. ?What am I missing?
>
> On Mon, Mar 30, 2009 at 5:44 PM, Guido van Rossum <guido at python.org> wrote:
>> I reproduced this, but it seems to have more to do with "from . import
>> ..." than with the Python version. If I add the "from ." before each
>> of the imports, "python -c 'import p.a' " fails with roughly the above
>> traceback for any version of Python that supports this syntax, while
>> without that it passes for any 2.x.
>>
>> If I use the "from ." syntax in a.py but not in b.py, "import p.a"
>> passes but "import p.b" fails.
>>
>> I'll see if anyone present at the sprints has a clue.
>
> Made some progress. Anything using "from <whatever> import b" (where
> <whatever> is either '.' or 'p') will fail when b's import is not
> completed. OTOH using "import p.b" works. I reduced it to:
>
> p/a.py == "from p import b"
> p/b.py == "import a"
> python -c "import p.b"
>
> The reason seems to be that until the outermost import (in this case
> p.b) is completed, while sys.modules has the (incomplete) modules 'p',
> 'p.a' and 'p.b', the attributes p.a and p.b aren't added until after
> their import is completed. Which it isn't during recursive import.
> Apparently 'from <anything> import <something>' looks for the
> <something> attribute in the <parent> object. This is because
> "from...import" can also be used to import objects other than modules
> (e.g. "from M import C"). I'm guessing that setting the attribute is
> delayed until the import is totally complete, because upon a failed
> import we remove the half-imported module object from sys.modules, but
> apparently we didn 't want to be in the business of removing the
> attribute from the parent package, so that's only set after the import
> is deemed successful.
>
> At least, this is my hypothesis, thinking about it -- I might look at
> the code later. :-)
>
> The most portable solution is to avoid "from...import" and instead
> write something like
>
> import p.b as b

So it turns out that "from X import Y" compiles into this bytecode:

              0 LOAD_CONST               0 (-1)
              3 LOAD_CONST               1 (('Y',))
              6 IMPORT_NAME              0 (X)
              9 IMPORT_FROM              1 (Y)
             12 STORE_NAME               1 (Y)
             15 POP_TOP

The first three opcodes (through IMPORT_NAME) call __import__('X',
None, None, ('Y',)) and push the result on top of the stack; this
result is the toplevel package X. The IMPORT_FROM opcode is
essentially a getattr call that turns an AttributeError into an
ImportError exception. I changed p/a.py into

p = __import__('p', None, None, ['b'])
print(p.b)

and confirmed that it fails on the print() line in p.b.

Does anyone feel that this ought to be fixed?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tjreedy at udel.edu  Tue Mar 31 02:35:47 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 30 Mar 2009 20:35:47 -0400
Subject: [Python-Dev] And the winner is...
In-Reply-To: <dcbbbb410903301626x3d707b49pa8184a765e6fcb59@mail.gmail.com>
References: <ca471dc20903300758s47fc922bpacaa4a5fb10a1dac@mail.gmail.com>
	<dcbbbb410903301626x3d707b49pa8184a765e6fcb59@mail.gmail.com>
Message-ID: <gqrogv$3ri$1@ger.gmane.org>

Michael Urman wrote:
>> We're switching to Mercurial (Hg).
> 
> And two hours later, GNOME announces their migration to git is
> underway. I'd suspect a series of April Fools jokes, if it weren't two
> days early. :)

Like Python, Gnome was/is using SVN and tested (at least) GIT, bzr, and 
hg mirrors, starting somewhat earlier than Python, for DVCS migration. 
As announced in January, the majority of *their* developers preferred 
GIT.  They started conversion then, in January, and made a progress 
announcement on March 19 (not yesterday).
http://permalink.gmane.org/gmane.comp.gnome.devel.announce/15

I fail to see any joke.  Different people have different tool preferences.

tjr


From tutufan at gmail.com  Tue Mar 31 02:59:59 2009
From: tutufan at gmail.com (Mike Coleman)
Date: Mon, 30 Mar 2009 19:59:59 -0500
Subject: [Python-Dev] And the winner is...
Message-ID: <3c6c07c20903301759u209f1b0dyb46c933e5f25f0b2@mail.gmail.com>

Just for curiosity's sake, could someone outline the five (or so) most
significant pluses of hg relative to git?


(My personal feeling is that any of the three is a huge improvement
over subversion.  I think git probably should have been written in
Python with some stuff in C where necessary, and (perhaps) the hg guy
really is right when he claims that Linus should have skipped git and
used hg from the start.  That notwithstanding, though, it kind of
looks like git has won the mindshare war at this point, and I think
the best hg can hope for from this point forward is a sort of *BSD to
git's Linux.  I do hope that it lives on, shutouts being fascist, etc.

Aside: I once worked with the guy maintaining git, and he might have
the greatest sum of talent plus humility of any programmer I ever
met.)

From tjreedy at udel.edu  Tue Mar 31 03:17:23 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 30 Mar 2009 21:17:23 -0400
Subject: [Python-Dev] Broken import?
In-Reply-To: <ca471dc20903301652w63dac59dn3275620c3b220fd3@mail.gmail.com>
References: <ca471dc20903301544l3749c80i3d166fbb8780f502@mail.gmail.com>	<ca471dc20903301617ydbb4bc9pfb221c2cba8df7e4@mail.gmail.com>
	<ca471dc20903301652w63dac59dn3275620c3b220fd3@mail.gmail.com>
Message-ID: <gqrqv0$8v2$1@ger.gmane.org>

Guido van Rossum wrote:
> On Mon, Mar 30, 2009 at 6:17 PM, Guido van Rossum <guido at python.org> wrote:
>> [Adding python-dev. I'm quoting the entire original message.]
>>
>>> On Thu, Mar 19, 2009 at 6:40 PM, Fredrik Lundh <fredriklundh at google.com> wrote:
>>>> PS. Is it just me, or is import broken in 3.0?  Consider this:
[snip]

>>>> Sure, it's a recursive import, but there's no recursive dependency
>>>> here - nobody will access the module contents until the main program
>>>> calls the library.  What am I missing?

Problems with recursive imports are a perennial topic on Python list.  A 
common suggestion is to refactor to avoid them.

>> On Mon, Mar 30, 2009 at 5:44 PM, Guido van Rossum <guido at python.org> wrote:
>>> I reproduced this, but it seems to have more to do with "from . import
>>> ..." than with the Python version. If I add the "from ." before each
>>> of the imports, "python -c 'import p.a' " fails with roughly the above
>>> traceback for any version of Python that supports this syntax, while
>>> without that it passes for any 2.x.
>>>
>>> If I use the "from ." syntax in a.py but not in b.py, "import p.a"
>>> passes but "import p.b" fails.
>>>
>>> I'll see if anyone present at the sprints has a clue.
>> Made some progress. Anything using "from <whatever> import b" (where
>> <whatever> is either '.' or 'p') will fail when b's import is not
>> completed. OTOH using "import p.b" works. I reduced it to:
>>
>> p/a.py == "from p import b"
>> p/b.py == "import a"
>> python -c "import p.b"
>>
>> The reason seems to be that until the outermost import (in this case
>> p.b) is completed, while sys.modules has the (incomplete) modules 'p',
>> 'p.a' and 'p.b', the attributes p.a and p.b aren't added until after
>> their import is completed. Which it isn't during recursive import.
>> Apparently 'from <anything> import <something>' looks for the
>> <something> attribute in the <parent> object. This is because
>> "from...import" can also be used to import objects other than modules
>> (e.g. "from M import C"). I'm guessing that setting the attribute is
>> delayed until the import is totally complete, because upon a failed
>> import we remove the half-imported module object from sys.modules, but
>> apparently we didn 't want to be in the business of removing the
>> attribute from the parent package, so that's only set after the import
>> is deemed successful.
>>
>> At least, this is my hypothesis, thinking about it -- I might look at
>> the code later. :-)
>>
>> The most portable solution is to avoid "from...import"

When doing recursive imports (it seems to work fine otherwise).

> and instead write something like
>>
>> import p.b as b
> 
> So it turns out that "from X import Y" compiles into this bytecode:
> 
>               0 LOAD_CONST               0 (-1)
>               3 LOAD_CONST               1 (('Y',))
>               6 IMPORT_NAME              0 (X)
>               9 IMPORT_FROM              1 (Y)
>              12 STORE_NAME               1 (Y)
>              15 POP_TOP
> 
> The first three opcodes (through IMPORT_NAME) call __import__('X',
> None, None, ('Y',)) and push the result on top of the stack; this
> result is the toplevel package X. The IMPORT_FROM opcode is
> essentially a getattr call that turns an AttributeError into an
> ImportError exception. I changed p/a.py into
> 
> p = __import__('p', None, None, ['b'])
> print(p.b)
> 
> and confirmed that it fails on the print() line in p.b.

If I understand, you are saying that

from x import y

is equivalent in effect to

import x
y = x.y
del x

except that the binding of 'x' never happens.

This is pretty much what the (3.0.1) doc says: "The from form does not 
bind the module name: it goes through the list of identifiers, looks 
each one of them up in the module found in step (1), and binds the name 
in the local namespace to the object thus found. " where step 1 is the 
(completed) initialization of the module.

So it seems to me that the behavior Fredrik noticed is implied by the 
doc.  It could be main plainer though.  I have not read Brett's proposed 
import doc yet.

> Does anyone feel that this ought to be fixed?

What would be the new doc?

Terry Jan Reedy




From tjreedy at udel.edu  Tue Mar 31 03:37:27 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 30 Mar 2009 21:37:27 -0400
Subject: [Python-Dev] Broken import?
In-Reply-To: <gqrqv0$8v2$1@ger.gmane.org>
References: <ca471dc20903301544l3749c80i3d166fbb8780f502@mail.gmail.com>	<ca471dc20903301617ydbb4bc9pfb221c2cba8df7e4@mail.gmail.com>	<ca471dc20903301652w63dac59dn3275620c3b220fd3@mail.gmail.com>
	<gqrqv0$8v2$1@ger.gmane.org>
Message-ID: <gqrs4i$b1l$1@ger.gmane.org>

Terry Reedy wrote:
> Guido van Rossum wrote:

>>> The reason seems to be that until the outermost import (in this case
>>> p.b) is completed, while sys.modules has the (incomplete) modules 'p',
>>> 'p.a' and 'p.b', the attributes p.a and p.b aren't added until after
>>> their import is completed. Which it isn't during recursive import.
>>> Apparently 'from <anything> import <something>' looks for the
>>> <something> attribute in the <parent> object. This is because
>>> "from...import" can also be used to import objects other than modules
>>> (e.g. "from M import C"). I'm guessing that setting the attribute is
>>> delayed until the import is totally complete, because upon a failed
>>> import we remove the half-imported module object from sys.modules, but
>>> apparently we didn 't want to be in the business of removing the
>>> attribute from the parent package, so that's only set after the import
>>> is deemed successful.

I remember a pydev discussion on this point.


From benjamin at python.org  Tue Mar 31 04:29:12 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 30 Mar 2009 21:29:12 -0500
Subject: [Python-Dev] 3to2 Project
In-Reply-To: <43aa6ff70903301037y215d979he36246d36c987493@mail.gmail.com>
References: <4222a8490903300744t498e79daodea9cff32e4a94c1@mail.gmail.com>
	<43aa6ff70903301037y215d979he36246d36c987493@mail.gmail.com>
Message-ID: <1afaf6160903301929l4120abe5g96e2ca2fdb722896@mail.gmail.com>

2009/3/30 Collin Winter <collinw at gmail.com>:
> If anyone is interested in working on this during the PyCon sprints or
> otherwise, here are some easy, concrete starter projects that would
> really help move this along:
> - The core refactoring engine needs to be broken out from 2to3. In
> particular, the tests/ and fixes/ need to get pulled up a directory,
> out of lib2to3/.
> - Once that's done, lib2to3 should then be renamed to something like
> librefactor or something else that indicates its more general nature.
> This will allow both 2to3 and 3to2 to more easily share the core
> components.

FWIW, I think it is unfortunately too late to make this change. We've
already released it as lib2to3 in the standard library and I have
actually seen it used in other projects. (PythonScope, for example.)



-- 
Regards,
Benjamin

From guido at python.org  Tue Mar 31 04:54:47 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Mar 2009 21:54:47 -0500
Subject: [Python-Dev] And the winner is...
In-Reply-To: <3c6c07c20903301759u209f1b0dyb46c933e5f25f0b2@mail.gmail.com>
References: <3c6c07c20903301759u209f1b0dyb46c933e5f25f0b2@mail.gmail.com>
Message-ID: <ca471dc20903301954g435309dex26c8117e9316355f@mail.gmail.com>

On Mon, Mar 30, 2009 at 7:59 PM, Mike Coleman <tutufan at gmail.com> wrote:
> Just for curiosity's sake, could someone outline the five (or so) most
> significant pluses of hg relative to git?
>
>
> (My personal feeling is that any of the three is a huge improvement
> over subversion. ?I think git probably should have been written in
> Python with some stuff in C where necessary, and (perhaps) the hg guy
> really is right when he claims that Linus should have skipped git and
> used hg from the start. ?That notwithstanding, though, it kind of
> looks like git has won the mindshare war at this point, and I think
> the best hg can hope for from this point forward is a sort of *BSD to
> git's Linux. ?I do hope that it lives on, shutouts being fascist, etc.

Yeah, I also think I'll just stop developing Python now and suggest
that you all switch to Java, which has clearly won the mindshare war
for languages. :-)

> Aside: I once worked with the guy maintaining git, and he might have
> the greatest sum of talent plus humility of any programmer I ever
> met.)

But is his humility enough to cancel out Linus's attitude?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rdmurray at bitdance.com  Tue Mar 31 05:57:03 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 30 Mar 2009 23:57:03 -0400 (EDT)
Subject: [Python-Dev] CMake Evaluation
In-Reply-To: <49D0EA35.1030806@kitware.com>
References: <49D0EA35.1030806@kitware.com>
Message-ID: <Pine.LNX.4.64.0903302353120.26362@kimball.webabinitio.net>

On Mon, 30 Mar 2009 at 11:50, Bill Hoffman wrote:
>> B. It does not allow the CMakeLists.txt file control the --help
> output. This appears to be an intentional decision
> (http://www.cmake.org/pipermail/cmake-promote/2006-May/000095.html).
> To replace it, they have an interactive mode (which asks you about
> each possible option) and a -LH flag (which runs the whole configure
> process and then tells you about the flags you could have set if you
> knew about them).
>
>
> CMake now has a cmake-gui (qt based application) for configuring a project.

A GUI is not a replacement for useful command line help :)

--
R. David Murray             http://www.bitdance.com

From aahz at pythoncraft.com  Tue Mar 31 05:57:27 2009
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 30 Mar 2009 20:57:27 -0700
Subject: [Python-Dev] And the winner is...
In-Reply-To: <gqrogv$3ri$1@ger.gmane.org>
References: <ca471dc20903300758s47fc922bpacaa4a5fb10a1dac@mail.gmail.com>
	<dcbbbb410903301626x3d707b49pa8184a765e6fcb59@mail.gmail.com>
	<gqrogv$3ri$1@ger.gmane.org>
Message-ID: <20090331035727.GB928@panix.com>

On Mon, Mar 30, 2009, Terry Reedy wrote:
> Michael Urman wrote:
>> Guido:
>>>
>>> We're switching to Mercurial (Hg).
>>
>> And two hours later, GNOME announces their migration to git is
>> underway. I'd suspect a series of April Fools jokes, if it weren't two
>> days early. :)
>
> Like Python, Gnome was/is using SVN and tested (at least) GIT, bzr, and  
> hg mirrors, starting somewhat earlier than Python, for DVCS migration.  
> As announced in January, the majority of *their* developers preferred  
> GIT.  They started conversion then, in January, and made a progress  
> announcement on March 19 (not yesterday).
> http://permalink.gmane.org/gmane.comp.gnome.devel.announce/15
>
> I fail to see any joke.  Different people have different tool preferences.

As Michael said, joke suspicion comes from the timing.  Combine that with
Guido's previous post about the Van Lindberg clause...
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it."  --Brian W. Kernighan

From guido at python.org  Tue Mar 31 06:13:38 2009
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Mar 2009 23:13:38 -0500
Subject: [Python-Dev] And the winner is...
In-Reply-To: <20090331035727.GB928@panix.com>
References: <ca471dc20903300758s47fc922bpacaa4a5fb10a1dac@mail.gmail.com>
	<dcbbbb410903301626x3d707b49pa8184a765e6fcb59@mail.gmail.com>
	<gqrogv$3ri$1@ger.gmane.org> <20090331035727.GB928@panix.com>
Message-ID: <ca471dc20903302113y4966184em99c45b579832d855@mail.gmail.com>

On Mon, Mar 30, 2009 at 10:57 PM, Aahz <aahz at pythoncraft.com> wrote:
> On Mon, Mar 30, 2009, Terry Reedy wrote:
>> Michael Urman wrote:
>>> Guido:
>>>>
>>>> We're switching to Mercurial (Hg).
>>>
>>> And two hours later, GNOME announces their migration to git is
>>> underway. I'd suspect a series of April Fools jokes, if it weren't two
>>> days early. :)
>>
>> Like Python, Gnome was/is using SVN and tested (at least) GIT, bzr, and
>> hg mirrors, starting somewhat earlier than Python, for DVCS migration.
>> As announced in January, the majority of *their* developers preferred
>> GIT. ?They started conversion then, in January, and made a progress
>> announcement on March 19 (not yesterday).
>> http://permalink.gmane.org/gmane.comp.gnome.devel.announce/15
>>
>> I fail to see any joke. ?Different people have different tool preferences.
>
> As Michael said, joke suspicion comes from the timing.

AFAIC the only joke was that I announced the decision on Twitter first...

> Combine that with Guido's previous post about the Van Lindberg clause...

Um, that was on psf-members, which has  much smaller audience than python-dev.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steve at holdenweb.com  Tue Mar 31 06:24:10 2009
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 31 Mar 2009 00:24:10 -0400
Subject: [Python-Dev] And the winner is...
In-Reply-To: <ca471dc20903301954g435309dex26c8117e9316355f@mail.gmail.com>
References: <3c6c07c20903301759u209f1b0dyb46c933e5f25f0b2@mail.gmail.com>
	<ca471dc20903301954g435309dex26c8117e9316355f@mail.gmail.com>
Message-ID: <gqs5tb$tml$1@ger.gmane.org>

Guido van Rossum wrote:
> On Mon, Mar 30, 2009 at 7:59 PM, Mike Coleman <tutufan at gmail.com> wrote:
>> Just for curiosity's sake, could someone outline the five (or so) most
>> significant pluses of hg relative to git?
>>
>>
>> (My personal feeling is that any of the three is a huge improvement
>> over subversion.  I think git probably should have been written in
>> Python with some stuff in C where necessary, and (perhaps) the hg guy
>> really is right when he claims that Linus should have skipped git and
>> used hg from the start.  That notwithstanding, though, it kind of
>> looks like git has won the mindshare war at this point, and I think
>> the best hg can hope for from this point forward is a sort of *BSD to
>> git's Linux.  I do hope that it lives on, shutouts being fascist, etc.
> 
> Yeah, I also think I'll just stop developing Python now and suggest
> that you all switch to Java, which has clearly won the mindshare war
> for languages. :-)
> 
>> Aside: I once worked with the guy maintaining git, and he might have
>> the greatest sum of talent plus humility of any programmer I ever
>> met.)
> 
> But is his humility enough to cancel out Linus's attitude?
> 
All the humility in the world pales besides Linus's attitude. But that's
probably just because we are all fools.

regards
 Steve
-- 
Steve Holden           +1 571 484 6266   +1 800 494 3119
Holden Web LLC                 http://www.holdenweb.com/
Want to know? Come to PyCon - soon! http://us.pycon.org/


From stephen at xemacs.org  Tue Mar 31 07:31:13 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 31 Mar 2009 14:31:13 +0900
Subject: [Python-Dev] And the winner is...
In-Reply-To: <3c6c07c20903301759u209f1b0dyb46c933e5f25f0b2@mail.gmail.com>
References: <3c6c07c20903301759u209f1b0dyb46c933e5f25f0b2@mail.gmail.com>
Message-ID: <874oxaw95q.fsf@xemacs.org>

Mike Coleman writes:

 > Just for curiosity's sake, could someone outline the five (or so)
 > most significant pluses of hg relative to git?

I think really it comes down to Guido's intuition.

However, without attempting to channel Guido, as the git proponent in
the PEP I'd like to go on record as saying that I'm quite satisfied
with the outcome.

The main thing is that git strongly encourages direct manipulation of
the commit DAG, in the way that Lisp encourages direct manipulation of
lists (even more so than Python does!)  This opens the door to
dramatic changes in the public workflow over time, viz. movement
toward a Linux-kernel-like workflow.  But the BDFL is not Linus, and
Python is not the Linux kernel.  My feeling (as the git proponent in
the PEP who was surprised about the pushback I felt) is that some
Python developers are visceral conservatives about workflow.  Even
cracking that door is unnerving.  And most would rather avoid changes
in the *community* workflow, despite eagerly looking forward to the
changes in *personal* workflow that any of the distributed VCSes will
enable.

My feeling is that in that context, it's not a matter of which is
"best".  They're all good.  But from the point of view of maintaining
the good points of the current workflow, while enabling experiment and
improvement by individual developers, I think Mercurial is most
conservative alternative of the three.

I also just wrote a long post about the comparison of bzr to hg
responding to a comment on bazaar at canonical.com.  I won't recap it
here but it might be of interest.

 > I think git probably should have been written in Python with some
 > stuff in C where necessary,

It wouldn't be that hard to do a rewrite in Python, but the git
programmers are mostly kernel people.  They write in C and shell.
No big deal -- except to Pythonistas.<wink>

 > and (perhaps) the hg guy really is right when he claims that Linus
 > should have skipped git and used hg from the start.

Unlikely.  As Terry says, people have different preferences for
tools.  The important one here is whether you see history as immutable
fact and direct manipulations of the commit DAG as falsification, or not.

People who lean toward the DAG as *recording* history will prefer
Mercurial or Bazaar. People who tend to see the DAG as a tool for
*presenting* changes will prefer git.


From matthieu.brucher at gmail.com  Tue Mar 31 07:57:28 2009
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Tue, 31 Mar 2009 07:57:28 +0200
Subject: [Python-Dev] CMake Evaluation
In-Reply-To: <Pine.LNX.4.64.0903302353120.26362@kimball.webabinitio.net>
References: <49D0EA35.1030806@kitware.com>
	<Pine.LNX.4.64.0903302353120.26362@kimball.webabinitio.net>
Message-ID: <e76aa17f0903302257q50a46bf9uf41c8106287a559f@mail.gmail.com>

2009/3/31 R. David Murray <rdmurray at bitdance.com>:
> On Mon, 30 Mar 2009 at 11:50, Bill Hoffman wrote:
>>>
>>> B. It does not allow the CMakeLists.txt file control the --help
>>
>> output. This appears to be an intentional decision
>> (http://www.cmake.org/pipermail/cmake-promote/2006-May/000095.html).
>> To replace it, they have an interactive mode (which asks you about
>> each possible option) and a -LH flag (which runs the whole configure
>> process and then tells you about the flags you could have set if you
>> knew about them).
>>
>>
>> CMake now has a cmake-gui (qt based application) for configuring a
>> project.
>
> A GUI is not a replacement for useful command line help :)

You can also use ccmake, IIRC.

Matthieu
-- 
Information System Engineer, Ph.D.
Website: http://matthieu-brucher.developpez.com/
Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn: http://www.linkedin.com/in/matthieubrucher

From ncoghlan at gmail.com  Tue Mar 31 14:23:48 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 31 Mar 2009 22:23:48 +1000
Subject: [Python-Dev] Broken import?
In-Reply-To: <gqrs4i$b1l$1@ger.gmane.org>
References: <ca471dc20903301544l3749c80i3d166fbb8780f502@mail.gmail.com>	<ca471dc20903301617ydbb4bc9pfb221c2cba8df7e4@mail.gmail.com>	<ca471dc20903301652w63dac59dn3275620c3b220fd3@mail.gmail.com>	<gqrqv0$8v2$1@ger.gmane.org>
	<gqrs4i$b1l$1@ger.gmane.org>
Message-ID: <49D20B54.1010108@gmail.com>

Terry Reedy wrote:
> Terry Reedy wrote:
>> Guido van Rossum wrote:
> 
>>>> The reason seems to be that until the outermost import (in this case
>>>> p.b) is completed, while sys.modules has the (incomplete) modules 'p',
>>>> 'p.a' and 'p.b', the attributes p.a and p.b aren't added until after
>>>> their import is completed. Which it isn't during recursive import.
>>>> Apparently 'from <anything> import <something>' looks for the
>>>> <something> attribute in the <parent> object. This is because
>>>> "from...import" can also be used to import objects other than modules
>>>> (e.g. "from M import C"). I'm guessing that setting the attribute is
>>>> delayed until the import is totally complete, because upon a failed
>>>> import we remove the half-imported module object from sys.modules, but
>>>> apparently we didn 't want to be in the business of removing the
>>>> attribute from the parent package, so that's only set after the import
>>>> is deemed successful.
> 
> I remember a pydev discussion on this point.

The quote from Fredrik in Guido's original message sounded familiar.
Checking my nosy list on the tracker brought me to the following issue:

http://bugs.python.org/issue992389

Jim Fulton's example in that tracker issue shows that with a bit of
creativity you can provoke this behaviour *without* using a from-style
import. Torsten Bronger later brought up the same issue that Fredrik did
- it prevents some kinds of explicit relative import that look like they
should be fine.

It's starting to look to me like the "set in advance and delete on
failure" approach taken with sys.modules may make sense for the
attributes in the parent package after all. Given the way that import is
defined, I suspect that is easier said than done though...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Tue Mar 31 14:42:27 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 31 Mar 2009 22:42:27 +1000
Subject: [Python-Dev] And the winner is...
In-Reply-To: <3c6c07c20903301759u209f1b0dyb46c933e5f25f0b2@mail.gmail.com>
References: <3c6c07c20903301759u209f1b0dyb46c933e5f25f0b2@mail.gmail.com>
Message-ID: <49D20FB3.9050400@gmail.com>

Mike Coleman wrote:
> Just for curiosity's sake, could someone outline the five (or so) most
> significant pluses of hg relative to git?

Every single git command line example I have seen gives me exactly the
same gut reaction I get whenever I have to read Perl code. You can extol
the tool's virtues to me all day long, but you're never going to
eliminate that visceral horror at its interface, any more than someone
that loves Perl is going to have any luck convincing me that it really
can be a sane choice of language for anything more than
write-once-read-never throwaway scripts.

Note that it *isn't* the idea of a using a directed acyclic graph in
general that bothers me (since all DVCSs are pretty much forced to do
that): it's specifically the way the sensibilities of git's original
audience are reflected in the CLI, and the subsequent offense to my own
personal sense of aesthetics :)

The Mercurial and Bazaar interfaces on the other hand, both seemed
perfectly palatable (e.g. a bit more inclined to use words over arcane
symbols), and Hg appears to be a clear winner against Bazaar when it
comes to performance *right now*. So Guido's intuition actually sounds
perfectly reasonable to me.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From kristjan at ccpgames.com  Tue Mar 31 15:27:33 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Tue, 31 Mar 2009 13:27:33 +0000
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <d2155e360903250553u2e134d61rc005e8817ad82a60@mail.gmail.com>
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net> <49C9EEB5.2090804@gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>
	<d2155e360903250553u2e134d61rc005e8817ad82a60@mail.gmail.com>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F056D5272C8@exchis.ccp.ad.local>

Right, my question to microsoft was more about making sure that a __crtMessageBox() actually does nothing, when running unattended as a service.

Also, we should be seeing the same problem in non-debug versions, since the _set_invalid_parameter_handler() is no longer called, and the default one also puts up a message box.  The Assertion is then performed _in_addition_ on debug builds:
>From ms-help://MS.VSCC.v80/MS.MSDN.v80/MS.VisualStudio.v80.en/dv_vccrt/html/c0e67934-1a41-4016-ad8e-972828f3ac11.htm
In the following example, an invalid parameter error handler is used to print the function that received the invalid parameter and the file and line in CRT sources. When the debug CRT library is used, invalid parameter errors also throw an assertion, which is disabled in this example using _CrtSetReportMode.

Btw, I am working on finding out the test suite failures for test_multiprocessing.
K
-----Original Message-----
From: Curt Hagenlocher [mailto:curt at hagenlocher.org] 
Sent: 25. mars 2009 12:54
To: Kristj?n Valur J?nsson
Cc: mhammond at skippinet.com.au; David Bolen; python-dev at python.org
Subject: Re: [Python-Dev] Test failures under Windows?

2009/3/25 Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
>
> I'm going to poke my contacts at Microsoft and ask them if there is
> a way to disable popups like this for a process that runs unattended
> and/or is running as a windows service.

MSVC has shipped with runtime library source since the 16-bit days, so
the easier thing is just to read the source code.  :)

It looks like there are two possible code paths for the assertion.
The first code path uses functions in dbgrpt[t].c and its behavior is
controlled by CrtSetReportMode and CrtSetReportFile.  The second uses
functions in assert.c and its behavior is controlled by
_set_error_mode.  Both code paths use __crtMessageBox in crtmbox.c to
actually display the message box.  This function -- when running as a
service -- will helpfully set MB_SERVICE_NOTIFICATION before calling
the USER32 MessageBox, which ensures that you'll see exactly why your
service has hung the next time you actually log in with sufficient
rights. :/

The variation that goes through assert.c should write to stderr for a
console-mode application, so it's reasonable to assume that we're
hitting the other code path -- and that Mark's suggestion to use
CrtSetReportMode would address the issue.

--
Curt Hagenlocher
curt at hagenlocher.org


From bcannon at gmail.com  Tue Mar 31 16:35:37 2009
From: bcannon at gmail.com (Brett Cannon)
Date: Tue, 31 Mar 2009 07:35:37 -0700
Subject: [Python-Dev] And the winner is...
In-Reply-To: <49D20FB3.9050400@gmail.com>
References: <3c6c07c20903301759u209f1b0dyb46c933e5f25f0b2@mail.gmail.com> 
	<49D20FB3.9050400@gmail.com>
Message-ID: <bbaeab100903310735k7afa1fdft2ad2f7cf3927f231@mail.gmail.com>

2009/3/31 Nick Coghlan <ncoghlan at gmail.com>

> Mike Coleman wrote:
> > Just for curiosity's sake, could someone outline the five (or so) most
> > significant pluses of hg relative to git?
>
> Every single git command line example I have seen gives me exactly the
> same gut reaction I get whenever I have to read Perl code. You can extol
> the tool's virtues to me all day long, but you're never going to
> eliminate that visceral horror at its interface, any more than someone
> that loves Perl is going to have any luck convincing me that it really
> can be a sane choice of language for anything more than
> write-once-read-never throwaway scripts.
>
> Note that it *isn't* the idea of a using a directed acyclic graph in
> general that bothers me (since all DVCSs are pretty much forced to do
> that): it's specifically the way the sensibilities of git's original
> audience are reflected in the CLI, and the subsequent offense to my own
> personal sense of aesthetics :)
>
> The Mercurial and Bazaar interfaces on the other hand, both seemed
> perfectly palatable (e.g. a bit more inclined to use words over arcane
> symbols), and Hg appears to be a clear winner against Bazaar when it
> comes to performance *right now*. So Guido's intuition actually sounds
> perfectly reasonable to me.



It's also about what the community prefers. Git was eliminated because it
didn't offer some stellar feature that  warranted forcing core developers to
use it when my little survey clearly showed it was the most disliked. Hg was
chosen (in my view) because the community wanted it; after I said Git was
out I had a lot of people come up to me stating their preference for
Mercurial. Once again, while Bazaar would have been fine, there was not
leaping out at me to cause me to think that I should potentially alienate
part of the community by going against their preference.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090331/20156c49/attachment.htm>

From eric at trueblade.com  Tue Mar 31 16:48:58 2009
From: eric at trueblade.com (Eric Smith)
Date: Tue, 31 Mar 2009 09:48:58 -0500
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F056D5272C8@exchis.ccp.ad.local>
References: <loom.20090324T134451-902@post.gmane.org>	<m2zlfa241x.fsf@valheru.db3l.homeip.net>	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net>
	<49C9EEB5.2090804@gmail.com>	<930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>	<d2155e360903250553u2e134d61rc005e8817ad82a60@mail.gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D5272C8@exchis.ccp.ad.local>
Message-ID: <49D22D5A.6090507@trueblade.com>

Kristj?n Valur J?nsson wrote:
> Btw, I am working on finding out the test suite failures for test_multiprocessing.

Some of those are caused by force builds on a branch, so make sure the 
errors are still occurring before you put too much effort into this. We 
made the branch before some recent fixes to the py3k branch.

But of course there may still be errors that exist.

Eric.


From guido at python.org  Tue Mar 31 17:12:11 2009
From: guido at python.org (Guido van Rossum)
Date: Tue, 31 Mar 2009 10:12:11 -0500
Subject: [Python-Dev] [Python-ideas] CapPython's use of unbound methods
In-Reply-To: <20090331112247.4a10624a@o>
References: <ca471dc20903121433p783ea549k9dcdc7114709ffd9@mail.gmail.com>
	<20090319.231249.343185657.mrs@localhost.localdomain>
	<ca471dc20903221531q2b65d21ew3aa962543e6cdc55@mail.gmail.com>
	<20090329.225703.432823651.mrs@localhost.localdomain>
	<ca471dc20903301419p4305e05cy926f709b37124890@mail.gmail.com>
	<20090331112247.4a10624a@o>
Message-ID: <ca471dc20903310812x7e385af0pc878b8a4f129e519@mail.gmail.com>

[Adding back python-dev, I don't want this discussion fragmented.]

Denis,

I am arguing with Mark because he and others claim that it is possible
to add capabilities to Python *without* changing the flavor of the
language (much), and because he believes that using a subset of Python
is somehow helpful for Python programmers (or helps Python programmers
transitioning to CapPython). I'm trying to point out the limitations
of that approach.

In the past capability zealots have also requested forcefully that
Python should support capabilities natively. This sounds like an
unrealistic evolutionary path for the language, and I'm pointing that
out. (Certainly I don't see CapPython as a step in that direction --
perhaps Tav's approach could be.) If they are happy with a different
language that happens to resemble Python is some syntactic details
that would be fine of course, but then they shouldn't whine that Py3k
breaks their implementation.

I also suspect that Mark's approach might be easily crackable because
he doesn't know the CPython implementation well enough to be aware of
all possible attack vectors.

--Guido

On Tue, Mar 31, 2009 at 4:22 AM, spir <denis.spir at free.fr> wrote:
> Le Mon, 30 Mar 2009 16:19:51 -0500,
> Guido van Rossum <guido at python.org> s'exprima ainsi:
>
> [...]
>> > Fortunately CapPython does not have to make this kind of semantic
>> > change.
>>
>> Well of course it makes a much more severe semantic change by
>> declaring illegal all use of attribute names starting with underscore.
> [...]
>
> Hello,
>
> I wonder what's the meaning of this exchange. As I understand it, the point of CapPython is precisely to build a different dialect which semantic field is restricted, in order to comply with the so-called "Object-Capability" security guidelines.
> We may like it or not. So what? I won't use CapPython, but I'm pleased to see people do language experiments on a Python basis.
> Now, I also understand this: if ever for you CapPython breaks fondamental design choices, then you may not like, as Python inventor, this projects to be called *Python. I would even agree, from a moral point of view, with a demand for a name change.
>
> (Sorry if this sounds a bit rude. Not sure of idiom connotations in english. I prefere clarity to hypocrisy.)
>
> Denis
>
> PS: I had a kind of ProtoPython in mind. If ever someone comments that it's a major semantic change; that it does not let pythonistas use their beloved class-ic idioms; then I'll just laugh... Yes, it's even a major cognitive change. This is precisely the intent. So what?
>
> ------
> la vita e estrany
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tjreedy at udel.edu  Tue Mar 31 18:21:52 2009
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 31 Mar 2009 12:21:52 -0400
Subject: [Python-Dev] And the winner is...
In-Reply-To: <20090331035727.GB928@panix.com>
References: <ca471dc20903300758s47fc922bpacaa4a5fb10a1dac@mail.gmail.com>	<dcbbbb410903301626x3d707b49pa8184a765e6fcb59@mail.gmail.com>	<gqrogv$3ri$1@ger.gmane.org>
	<20090331035727.GB928@panix.com>
Message-ID: <gqtfv0$ucn$1@ger.gmane.org>

Aahz wrote:
> On Mon, Mar 30, 2009, Terry Reedy wrote:
>> Michael Urman wrote:
>>> Guido:
>>>> We're switching to Mercurial (Hg).
>>> And two hours later, GNOME announces their migration to git is
>>> underway. I'd suspect a series of April Fools jokes, if it weren't two
>>> days early. :)
>> Like Python, Gnome was/is using SVN and tested (at least) GIT, bzr, and  
>> hg mirrors, starting somewhat earlier than Python, for DVCS migration.  
>> As announced in January, the majority of *their* developers preferred  
>> GIT.  They started conversion then, in January, and made a progress  
>> announcement on March 19 (not yesterday).
>> http://permalink.gmane.org/gmane.comp.gnome.devel.announce/15
>>
>> I fail to see any joke.  Different people have different tool preferences.
> 
> As Michael said, joke suspicion comes from the timing.

In private email, he explained that the py-dev announcement from Guido 
landed in his mailbox right next to a 'gnome-dev' request to cease 
commits until the GIT changeover was complete.  That coincidence, 
coupled with his personal git preference and not having closely followed 
the evaluating process of either group, lead to his reaction.  He also 
disavowed any intention to start a new bikeshed discussion.

Terry


From tutufan at gmail.com  Tue Mar 31 18:39:17 2009
From: tutufan at gmail.com (Mike Coleman)
Date: Tue, 31 Mar 2009 11:39:17 -0500
Subject: [Python-Dev] And the winner is...
In-Reply-To: <ca471dc20903301954g435309dex26c8117e9316355f@mail.gmail.com>
References: <3c6c07c20903301759u209f1b0dyb46c933e5f25f0b2@mail.gmail.com>
	<ca471dc20903301954g435309dex26c8117e9316355f@mail.gmail.com>
Message-ID: <3c6c07c20903310939o6fc2bde6gd66c292362a3b204@mail.gmail.com>

On Mon, Mar 30, 2009 at 9:54 PM, Guido van Rossum <guido at python.org> wrote:
> Yeah, I also think I'll just stop developing Python now and suggest
> that you all switch to Java, which has clearly won the mindshare war
> for languages. :-)

Heh.  :-)

Guess I should have said "mindshare among people whose technical
opinions I give weight to".  In that sense, Python mindshare seems to
have been and to still be increasing steadily.  (My Magic 8-ball says
"future unclear" for Java.)

The TIOBE index is entertaining, if you haven't seen it before:

    http://www.tiobe.com/content/paperinfo/tpci/index.html


> But is his humility enough to cancel out Linus's attitude?

Why would I want to do that?  :-)


Seriously--thanks for all of your responses.  If it wasn't clear, I
was asking because I was curious about whether and why I should look
some more at hg.  I would never dream of trying to change anyone's
mind...

Mike

From stephen at xemacs.org  Tue Mar 31 18:59:11 2009
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 01 Apr 2009 01:59:11 +0900
Subject: [Python-Dev] And the winner is...
In-Reply-To: <49D20FB3.9050400@gmail.com>
References: <3c6c07c20903301759u209f1b0dyb46c933e5f25f0b2@mail.gmail.com>
	<49D20FB3.9050400@gmail.com>
Message-ID: <87y6ulvdb4.fsf@xemacs.org>

Nick Coghlan writes:

 > Every single git command line example I have seen gives me exactly the
 > same gut reaction I get whenever I have to read Perl code.

Every single one?  Sounds to me like the cause is probably something
you ate, not anything you read.  In the examples in the PEP, about 80%
of the commands were syntactically identical across VCSes.

I hope nobody is put off either git or bzr by the result of this PEP.
If there's anything striking about the PEP's examples, it's how
similar the usage of the VCSes would be in the context of Python's
workflow.  There are important differences, and I agree with Guido's
choice, for Python, on March 30, 2009.  But all three are capable
VCSes, with advantages and disadvantages, and were this PEP started
next June rather than last December, the result could have been very
different.

From tutufan at gmail.com  Tue Mar 31 20:04:47 2009
From: tutufan at gmail.com (Mike Coleman)
Date: Tue, 31 Mar 2009 13:04:47 -0500
Subject: [Python-Dev] And the winner is...
In-Reply-To: <874oxaw95q.fsf@xemacs.org>
References: <3c6c07c20903301759u209f1b0dyb46c933e5f25f0b2@mail.gmail.com>
	<874oxaw95q.fsf@xemacs.org>
Message-ID: <3c6c07c20903311104i6b50a9eeg3362ade5cf981c5c@mail.gmail.com>

On Tue, Mar 31, 2009 at 12:31 AM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
> I also just wrote a long post about the comparison of bzr to hg
> responding to a comment on bazaar at canonical.com. ?I won't recap it
> here but it might be of interest.

I found the post interesting.  Here's a link to the start of the thread:

    https://lists.ubuntu.com/archives/bazaar/2009q1/055805.html

There's a bit of bafflement there regarding Python culture.  I can
relate--although I love Python, I don't feel like I understand the
culture either.

> It wouldn't be that hard to do a rewrite in Python, but the git
> programmers are mostly kernel people. ?They write in C and shell.

I mentioned this once on the git list and Linus' response was
something like "C lets me see exactly what's going on".  I'm not
unsympathetic to this point of view--I'm really growing to loathe C++
partly because it *doesn't* let me see exactly what's going on--but
I'm not convinced, either.

It looks like there might be a Python clone sprouting here:

    http://gitorious.org/projects/git-python/


> People who lean toward the DAG as *recording* history will prefer
> Mercurial or Bazaar. People who tend to see the DAG as a tool for
> *presenting* changes will prefer git.

I've noticed this tension as well.  It seems to me that both uses are
important, so I suspect all three will eventually steal each other's
features with respect to this over time.

Mike

From benjamin at python.org  Tue Mar 31 21:09:30 2009
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 31 Mar 2009 14:09:30 -0500
Subject: [Python-Dev] 3.1a2
Message-ID: <1afaf6160903311209w43623e04mb35f15883b1d2560@mail.gmail.com>

Hi,
I'd like to release the second alpha of 3.1 as planned on Saturday,
April 4th. There are currently two release blockers, issues #4847 and
#5470. #5470 appears to be Martin's issue.

I haven't looked at #4847 in depth, but appears that the csv module
will need some API changes to deal with encodings. Perhaps somebody
would like to sprint on it?

-- 
Regards,
Benjamin

From larry at hastings.org  Tue Mar 31 21:14:57 2009
From: larry at hastings.org (Larry Hastings)
Date: Tue, 31 Mar 2009 14:14:57 -0500
Subject: [Python-Dev] Let's update CObject API so it is safe and regular!
Message-ID: <49D26BB1.8050108@hastings.org>


The CObject API has two flaws.

First, there is no usable type safety mechanism.  You can store a void
*object, and a void *description.  There is no established schema for
the description; it could be an integer cast to a pointer, or it could
point to memory of any configuration, or it could be NULL.  Thus users
of the CObject API generally ignore it--thus working without any type
safety whatsoever.  A programmer could crash the interpreter from pure
Python by mixing and matching CObjects from different modules (e.g. give
"curses" a CObject from "_ctypes").

Second, the destructor callback is defined as taking *either* one *or*
two parameters, depending on whether the "descr" pointer is non-NULL. 
One can debate the finer points of what is and isn't defined behavior in
C, but at its heart this is a sloppy API.

MvL and I discussed this last night and decided to float a revision of
the API.  I wrote the patch, though, so don't blame Martin if you don't
like my specific approach.

The color of this particular bike shed is:
* The PyCObject is now a private data structure; you must use accessors.
  I added accessors for all the members.
* The constructors and the main accessor (PyCObject_AsVoidPtr) now all
  *require* a "const char *type" parameter, which must be a non-NULL C
  string of non-zero length.  If you call that accessor and the "type"
  is invalid *or doesn't match*, it fails.
* The destructor now takes the PyObject *, not the PyCObject *.  You
  must use accessors to get your hands on the data inside to free it.

Yes, you can easily skip around the "matching type" restriction by
calling PyCObject_AsVoidPtr(cobj, PyCObject_GetType(cobj)).  The point
of my API changes is to *encourage* correct use.

I've posted a patch implementing this change in the 3.1 trunk to the
bug tracker:

    http://bugs.python.org/issue5630

I look forward to your comments!


/larry/


From ocean-city at m2.ccsnet.ne.jp  Tue Mar 31 21:28:20 2009
From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto)
Date: Wed, 01 Apr 2009 04:28:20 +0900
Subject: [Python-Dev] 3.1a2
In-Reply-To: <1afaf6160903311209w43623e04mb35f15883b1d2560@mail.gmail.com>
References: <1afaf6160903311209w43623e04mb35f15883b1d2560@mail.gmail.com>
Message-ID: <49D26ED4.7090205@m2.ccsnet.ne.jp>


Benjamin Peterson wrote:
> Hi,
> I'd like to release the second alpha of 3.1 as planned on Saturday,
> April 4th. There are currently two release blockers, issues #4847 and
> #5470. #5470 appears to be Martin's issue.
> 
> I haven't looked at #4847 in depth, but appears that the csv module
> will need some API changes to deal with encodings. Perhaps somebody
> would like to sprint on it?
> 

I added #5499 to release blocker because it needs specification 
decision. (It's too strong?)

From rdmurray at bitdance.com  Tue Mar 31 21:49:30 2009
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 31 Mar 2009 15:49:30 -0400 (EDT)
Subject: [Python-Dev] regrtest behavior change
Message-ID: <Pine.LNX.4.64.0903311220340.26362@kimball.webabinitio.net>

At the sprint the developers decided that a long standing issue with
regrtest should be fixed: the fact that invalid imports in tests can
result in a test being skipped instead of producing an error.

I've just checked in a change to regrtest on the trunk.  It no longer
assumes that an ImportError means the test should be marked as skipped.
I have done my best to wrap all imports where errors _should_ cause
skips in the appropriate wrappers from test_support (import_module or
get_attribute).  Of course, since I can't run the tests on all possible
systems I may have missed some.  It's also possible that making this
change will unmask some errors that have been hidden by the previous
behavior, although I haven't found any instances of that so far.

Another issue is that there may be modules that don't exist on other VM
implementations where those modules would have been skipped on a test run
previous to this, and will now generate an ImportError.  We will happily
wrap imports for those modules when informed that such should be done.

I'll be watching the buildbots, but please let me know about any issues
that you find.  I'll be merging it to py3k as well, but there will be
no backport.

--
R. David Murray             http://www.bitdance.com

From skip at pobox.com  Tue Mar 31 22:11:12 2009
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 31 Mar 2009 15:11:12 -0500 (CDT)
Subject: [Python-Dev] This seems like a bug - main thread has None ident???
Message-ID: <20090331201112.38AF7EF1A54@montanaro.dyndns.org>

Looking for some quick feedback on this.  I've bumped into what looks like a
bug in the threading module.  From the interpreter prompt:

    >>> import threading
    >>> threading.currentThread()
    <_MainThread(MainThread, started)>
    >>> print threading.currentThread().ident
    None

The ident attribute is documented as being a number unless the thread has
yet to be started.  Shouldn't the main thread's ident attribute *always*
be non-None?  Clearly, it appears the main thread has been started.

Am I missing something obvious or have I hit a bug?  This is a fully updated
2.7a0 build, trunk:70878M.

Skip

From db3l.net at gmail.com  Tue Mar 31 23:07:21 2009
From: db3l.net at gmail.com (David Bolen)
Date: Tue, 31 Mar 2009 17:07:21 -0400
Subject: [Python-Dev] Test failures under Windows?
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net>
	<49C9EEB5.2090804@gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>
	<d2155e360903250553u2e134d61rc005e8817ad82a60@mail.gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D5272C8@exchis.ccp.ad.local>
Message-ID: <m21vsdxuye.fsf@valheru.db3l.homeip.net>

Kristj?n Valur J?nsson <kristjan at ccpgames.com> writes:

> Btw, I am working on finding out the test suite failures for
> test_multiprocessing.

This is all well and good, but I still haven't seen any plausible
reason for not preventing these popups (in favor of stderr failures)
during buildbot test runs?  I don't get it - it just seems a no
brainer to help protect the Windows buildbots.  Surely there may be
similar issues down the road that also introduce the potential for
assertions?

I guess I'll stop asking after this note, but can anyone give a final
verdict on whether the older "-n" option can be restored to the
buildbot test.bat (from the revision history I'm not actually sure it
was intentionally removed in the first place)?  Unlike normal Windows
popups I can't protect against it easily through local modifications
to my build slave, since test.bat gets updated as part of the source
tree.

-- David


From skip at pobox.com  Tue Mar 31 23:08:56 2009
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 31 Mar 2009 16:08:56 -0500
Subject: [Python-Dev] This seems like a bug - main thread has None
	ident???
In-Reply-To: <266253949@toto.iv>
Message-ID: <18898.34408.22596.268311@montanaro.dyndns.org>


    skip> Am I missing something obvious or have I hit a bug?  This is a
    skip> fully updated 2.7a0 build, trunk:70878M.

After noting that thread.get_ident() returned a thread id but that
threading.currentThread().ident was None I concluded that it is, in fact, a
bug in the threading module.  I opened a ticket:

    http://bugs.python.org/issue5632

Skip

From jnoller at gmail.com  Tue Mar 31 23:27:21 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 31 Mar 2009 16:27:21 -0500
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <m21vsdxuye.fsf@valheru.db3l.homeip.net>
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net> <49C9EEB5.2090804@gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>
	<d2155e360903250553u2e134d61rc005e8817ad82a60@mail.gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D5272C8@exchis.ccp.ad.local>
	<m21vsdxuye.fsf@valheru.db3l.homeip.net>
Message-ID: <4222a8490903311427n295139fax25bec49afd149d85@mail.gmail.com>

On Tue, Mar 31, 2009 at 4:07 PM, David Bolen <db3l.net at gmail.com> wrote:
> Kristj?n Valur J?nsson <kristjan at ccpgames.com> writes:
>
>> Btw, I am working on finding out the test suite failures for
>> test_multiprocessing.
>
> This is all well and good, but I still haven't seen any plausible
> reason for not preventing these popups (in favor of stderr failures)
> during buildbot test runs? ?I don't get it - it just seems a no
> brainer to help protect the Windows buildbots. ?Surely there may be
> similar issues down the road that also introduce the potential for
> assertions?
>
> I guess I'll stop asking after this note, but can anyone give a final
> verdict on whether the older "-n" option can be restored to the
> buildbot test.bat (from the revision history I'm not actually sure it
> was intentionally removed in the first place)? ?Unlike normal Windows
> popups I can't protect against it easily through local modifications
> to my build slave, since test.bat gets updated as part of the source
> tree.
>
> -- David
>

I guess I missed the multiprocessing part of this discussion, but in
any case, see bug: http://bugs.python.org/issue5619 - once Martin and
I review it, it should be in trunk and elsewhere as needed.

-jesse

From jnoller at gmail.com  Tue Mar 31 23:31:10 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 31 Mar 2009 16:31:10 -0500
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F056D5272C8@exchis.ccp.ad.local>
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net> <49C9EEB5.2090804@gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>
	<d2155e360903250553u2e134d61rc005e8817ad82a60@mail.gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D5272C8@exchis.ccp.ad.local>
Message-ID: <4222a8490903311431u351b8d7kfb1e6cca716b1976@mail.gmail.com>

2009/3/31 Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
> Right, my question to microsoft was more about making sure that a __crtMessageBox() actually does nothing, when running unattended as a service.
>
> Also, we should be seeing the same problem in non-debug versions, since the _set_invalid_parameter_handler() is no longer called, and the default one also puts up a message box. ?The Assertion is then performed _in_addition_ on debug builds:
> >From ms-help://MS.VSCC.v80/MS.MSDN.v80/MS.VisualStudio.v80.en/dv_vccrt/html/c0e67934-1a41-4016-ad8e-972828f3ac11.htm
> In the following example, an invalid parameter error handler is used to print the function that received the invalid parameter and the file and line in CRT sources. When the debug CRT library is used, invalid parameter errors also throw an assertion, which is disabled in this example using _CrtSetReportMode.
>
> Btw, I am working on finding out the test suite failures for test_multiprocessing.
> K

I spoke to Martin about this yesterday, apparently a specific test in
the multiprocessing suite was triggering these dialogs (the test
invalid handle test). We're working on a patch for multiprocessing so
that the children will obey the flag

-jesse

From ncoghlan at gmail.com  Tue Mar 31 23:44:48 2009
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 01 Apr 2009 07:44:48 +1000
Subject: [Python-Dev] Broken import?
In-Reply-To: <gqrs4i$b1l$1@ger.gmane.org>
References: <ca471dc20903301544l3749c80i3d166fbb8780f502@mail.gmail.com>	<ca471dc20903301617ydbb4bc9pfb221c2cba8df7e4@mail.gmail.com>	<ca471dc20903301652w63dac59dn3275620c3b220fd3@mail.gmail.com>	<gqrqv0$8v2$1@ger.gmane.org>
	<gqrs4i$b1l$1@ger.gmane.org>
Message-ID: <49D28ED0.4050704@gmail.com>

Terry Reedy wrote:
> Terry Reedy wrote:
>> Guido van Rossum wrote:
> 
>>>> The reason seems to be that until the outermost import (in this case
>>>> p.b) is completed, while sys.modules has the (incomplete) modules 'p',
>>>> 'p.a' and 'p.b', the attributes p.a and p.b aren't added until after
>>>> their import is completed. Which it isn't during recursive import.
>>>> Apparently 'from <anything> import <something>' looks for the
>>>> <something> attribute in the <parent> object. This is because
>>>> "from...import" can also be used to import objects other than modules
>>>> (e.g. "from M import C"). I'm guessing that setting the attribute is
>>>> delayed until the import is totally complete, because upon a failed
>>>> import we remove the half-imported module object from sys.modules, but
>>>> apparently we didn 't want to be in the business of removing the
>>>> attribute from the parent package, so that's only set after the import
>>>> is deemed successful.
> 
> I remember a pydev discussion on this point.

A half-baked idea that I just posted to the tracker item:

What if the import semantics were adjusted so that, as a last gasp
effort before bailing out with an ImportError, the import process
checked sys.modules again with the full module name?

Would that be enough to eliminate this error?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From kristjan at ccpgames.com  Tue Mar 31 23:49:03 2009
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Tue, 31 Mar 2009 21:49:03 +0000
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <4222a8490903311431u351b8d7kfb1e6cca716b1976@mail.gmail.com>
References: <loom.20090324T134451-902@post.gmane.org>
	<m2zlfa241x.fsf@valheru.db3l.homeip.net>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net> <49C9EEB5.2090804@gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>
	<d2155e360903250553u2e134d61rc005e8817ad82a60@mail.gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D5272C8@exchis.ccp.ad.local>
	<4222a8490903311431u351b8d7kfb1e6cca716b1976@mail.gmail.com>
Message-ID: <930F189C8A437347B80DF2C156F7EC7F056D527435@exchis.ccp.ad.local>

I found a different problem in multiprocessing, for the py3k.
In import.c, get_file.c, it was knowingly leaking FILE objects, while the underlying fh was being correctly closed.  This caused the CRT to assert when cleaning up FILE pointers on subprocess exit.
I fixed this this afternoon in a submission to the py3k branch.

K

-----Original Message-----
From: Jesse Noller [mailto:jnoller at gmail.com] 
Sent: 31. mars 2009 21:31
To: Kristj?n Valur J?nsson
Cc: Curt Hagenlocher; mhammond at skippinet.com.au; David Bolen; python-dev at python.org
Subject: Re: [Python-Dev] Test failures under Windows?

2009/3/31 Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
> Right, my question to microsoft was more about making sure that a __crtMessageBox() actually does nothing, when running unattended as a service.
>
> Also, we should be seeing the same problem in non-debug versions, since the _set_invalid_parameter_handler() is no longer called, and the default one also puts up a message box. ?The Assertion is then performed _in_addition_ on debug builds:
> >From ms-help://MS.VSCC.v80/MS.MSDN.v80/MS.VisualStudio.v80.en/dv_vccrt/html/c0e67934-1a41-4016-ad8e-972828f3ac11.htm
> In the following example, an invalid parameter error handler is used to print the function that received the invalid parameter and the file and line in CRT sources. When the debug CRT library is used, invalid parameter errors also throw an assertion, which is disabled in this example using _CrtSetReportMode.
>
> Btw, I am working on finding out the test suite failures for test_multiprocessing.
> K

I spoke to Martin about this yesterday, apparently a specific test in
the multiprocessing suite was triggering these dialogs (the test
invalid handle test). We're working on a patch for multiprocessing so
that the children will obey the flag

-jesse


From jnoller at gmail.com  Tue Mar 31 23:59:43 2009
From: jnoller at gmail.com (Jesse Noller)
Date: Tue, 31 Mar 2009 16:59:43 -0500
Subject: [Python-Dev] Test failures under Windows?
In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F056D527435@exchis.ccp.ad.local>
References: <loom.20090324T134451-902@post.gmane.org>
	<930F189C8A437347B80DF2C156F7EC7F05068D5486@exchis.ccp.ad.local>
	<m2vdpy21u0.fsf@valheru.db3l.homeip.net>
	<m2ljqu1pzh.fsf@valheru.db3l.homeip.net> <49C9EEB5.2090804@gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D526790@exchis.ccp.ad.local>
	<d2155e360903250553u2e134d61rc005e8817ad82a60@mail.gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D5272C8@exchis.ccp.ad.local>
	<4222a8490903311431u351b8d7kfb1e6cca716b1976@mail.gmail.com>
	<930F189C8A437347B80DF2C156F7EC7F056D527435@exchis.ccp.ad.local>
Message-ID: <4222a8490903311459m68e7b9f8m8cfcf27aa71b92ac@mail.gmail.com>

Does it need to be backported? I wonder when that was introduced.
Also, what CL was it so I can review it?

2009/3/31 Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
> I found a different problem in multiprocessing, for the py3k.
> In import.c, get_file.c, it was knowingly leaking FILE objects, while the underlying fh was being correctly closed. ?This caused the CRT to assert when cleaning up FILE pointers on subprocess exit.
> I fixed this this afternoon in a submission to the py3k branch.
>
> K
>
> -----Original Message-----
> From: Jesse Noller [mailto:jnoller at gmail.com]
> Sent: 31. mars 2009 21:31
> To: Kristj?n Valur J?nsson
> Cc: Curt Hagenlocher; mhammond at skippinet.com.au; David Bolen; python-dev at python.org
> Subject: Re: [Python-Dev] Test failures under Windows?
>
> 2009/3/31 Kristj?n Valur J?nsson <kristjan at ccpgames.com>:
>> Right, my question to microsoft was more about making sure that a __crtMessageBox() actually does nothing, when running unattended as a service.
>>
>> Also, we should be seeing the same problem in non-debug versions, since the _set_invalid_parameter_handler() is no longer called, and the default one also puts up a message box. ?The Assertion is then performed _in_addition_ on debug builds:
>> >From ms-help://MS.VSCC.v80/MS.MSDN.v80/MS.VisualStudio.v80.en/dv_vccrt/html/c0e67934-1a41-4016-ad8e-972828f3ac11.htm
>> In the following example, an invalid parameter error handler is used to print the function that received the invalid parameter and the file and line in CRT sources. When the debug CRT library is used, invalid parameter errors also throw an assertion, which is disabled in this example using _CrtSetReportMode.
>>
>> Btw, I am working on finding out the test suite failures for test_multiprocessing.
>> K
>
> I spoke to Martin about this yesterday, apparently a specific test in
> the multiprocessing suite was triggering these dialogs (the test
> invalid handle test). We're working on a patch for multiprocessing so
> that the children will obey the flag
>
> -jesse
>
>