From roundup-admin at psf.upfronthosting.co.za  Mon Oct  1 00:05:32 2012
From: roundup-admin at psf.upfronthosting.co.za (Python tracker)
Date: Sun, 30 Sep 2012 22:05:32 +0000
Subject: [Python-Dev] Failed issue tracker submission
Message-ID: <20120930220532.158E71CAAE@psf.upfronthosting.co.za>



The node specified by the designator in the subject of your message
("16304") does not exist.

Subject was: "[issue16304]"



Mail Gateway Help
=================
Incoming messages are examined for multiple parts:
 . In a multipart/mixed message or part, each subpart is extracted and
   examined. The text/plain subparts are assembled to form the textual
   body of the message, to be stored in the file associated with a "msg"
   class node. Any parts of other types are each stored in separate files
   and given "file" class nodes that are linked to the "msg" node.
 . In a multipart/alternative message or part, we look for a text/plain
   subpart and ignore the other parts.
 . A message/rfc822 is treated similar tomultipart/mixed (except for
   special handling of the first text part) if unpack_rfc822 is set in
   the mailgw config section.

Summary
-------
The "summary" property on message nodes is taken from the first non-quoting
section in the message body. The message body is divided into sections by
blank lines. Sections where the second and all subsequent lines begin with
a ">" or "|" character are considered "quoting sections". The first line of
the first non-quoting section becomes the summary of the message.

Addresses
---------
All of the addresses in the To: and Cc: headers of the incoming message are
looked up among the user nodes, and the corresponding users are placed in
the "recipients" property on the new "msg" node. The address in the From:
header similarly determines the "author" property of the new "msg"
node. The default handling for addresses that don't have corresponding
users is to create new users with no passwords and a username equal to the
address. (The web interface does not permit logins for users with no
passwords.) If we prefer to reject mail from outside sources, we can simply
register an auditor on the "user" class that prevents the creation of user
nodes with no passwords.

Actions
-------
The subject line of the incoming message is examined to determine whether
the message is an attempt to create a new item or to discuss an existing
item. A designator enclosed in square brackets is sought as the first thing
on the subject line (after skipping any "Fwd:" or "Re:" prefixes).

If an item designator (class name and id number) is found there, the newly
created "msg" node is added to the "messages" property for that item, and
any new "file" nodes are added to the "files" property for the item.

If just an item class name is found there, we attempt to create a new item
of that class with its "messages" property initialized to contain the new
"msg" node and its "files" property initialized to contain any new "file"
nodes.

Triggers
--------
Both cases may trigger detectors (in the first case we are calling the
set() method to add the message to the item's spool; in the second case we
are calling the create() method to create a new node). If an auditor raises
an exception, the original message is bounced back to the sender with the
explanatory message given in the exception.
-------------- next part --------------
Return-Path: <python-dev at python.org>
X-Original-To: report at bugs.python.org
Delivered-To: roundup+tracker at psf.upfronthosting.co.za
Received: from mail.python.org (mail.python.org [82.94.164.166])
	by psf.upfronthosting.co.za (Postfix) with ESMTPS id C96C81CA8C
	for <report at bugs.python.org>; Mon,  1 Oct 2012 00:05:31 +0200 (CEST)
Received: from albatross.python.org (localhost [127.0.0.1])
	by mail.python.org (Postfix) with ESMTP id 3XVLKR40JNzNYF
	for <report at bugs.python.org>; Mon,  1 Oct 2012 00:05:31 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=python.org; s=200901;
	t=1349042731; bh=VvsXcY86pi/c+BeJIZc+70zwVxaiJ7YyKuEt7uYQ3Tc=;
	h=MIME-Version:Content-Type:Content-Transfer-Encoding:From:To:
	 Subject:Message-Id:Date;
	b=JIb9A8mKbs/8mEf6lMLk3yCyEqivXstaCqDmc1Bx/B8VveRxdmlBnXCIdktlJpDXA
	 e+ffB/whCObLiza0eN0IM8SVH012oUCUHsjGyE/9KMYXzgDLjqGhVQl65OD7lZ0did
	 Go+K7962YIYqsKcJFoEDsTBu0OKNroH9FqedZq1M=
Received: from localhost (HELO mail.python.org) (127.0.0.1)
  by albatross.python.org with SMTP; 01 Oct 2012 00:05:31 +0200
Received: from virt-7yvsjn.psf.osuosl.org (virt-7yvsjn.psf.osuosl.org [140.211.10.72])
	by mail.python.org (Postfix) with ESMTP
	for <report at bugs.python.org>; Mon,  1 Oct 2012 00:05:30 +0200 (CEST)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
From: python-dev at python.org
To: report at bugs.python.org
Subject: [issue16304]
Message-Id: <3XVLKR40JNzNYF at mail.python.org>
Date: Mon,  1 Oct 2012 00:05:31 +0200 (CEST)

TmV3IGNoYW5nZXNldCA0MjU4MjQ4YTQ0YzcgYnkgTmFkZWVtIFZhd2RhIGluIGJyYW5jaCAnMy4z
JzoKSXNzdWUgIzE2MzA0OiBGdXJ0aGVyIHBlcmZvcm1hbmNlIGltcHJvdmVtZW50cyBmb3IgQloy
RmlsZS4KaHR0cDovL2hnLnB5dGhvbi5vcmcvY3B5dGhvbi9yZXYvNDI1ODI0OGE0NGM3CgoKTmV3
IGNoYW5nZXNldCBhYmI1YzViZGU4NzIgYnkgTmFkZWVtIFZhd2RhIGluIGJyYW5jaCAnZGVmYXVs
dCc6Ck1lcmdlOiAjMTYzMDQ6IEZ1cnRoZXIgcGVyZm9ybWFuY2UgaW1wcm92ZW1lbnRzIGZvciBC
WjJGaWxlLgpodHRwOi8vaGcucHl0aG9uLm9yZy9jcHl0aG9uL3Jldi9hYmI1YzViZGU4NzIK

From doko at ubuntu.com  Mon Oct  1 00:17:09 2012
From: doko at ubuntu.com (Matthias Klose)
Date: Mon, 01 Oct 2012 00:17:09 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
Message-ID: <5068C4E5.9020606@ubuntu.com>

On 30.09.2012 20:18, Gregory P. Smith wrote:
> On Sun, Sep 30, 2012 at 8:33 AM, Benjamin Peterson <benjamin at python.org>wrote:
> 
>> 2012/9/30 Xavier Morel <python-dev at masklinn.net>:
>>> But at worst, an outdated unicode database will be missing data right?
>>>
>>> Doesn't an outdated timezone db have the risk of returning *incorrect*
>> data?
>>
>> Unicode updates also include corrections; however, it seems there are
>> not significant enough or about obscure enough scripts that not many
>> notice. :)
>>
> 
> We never hear anyone complain because the corrections are not for English
> or other "western" languages that the majority of us speak.  ;)
> 
> Regardless, I think including a version of the database on windows releases
> makes sense.  Update it on a best effort basis before each .x minor
> release.  The documentation can make it clear that it is a changing
> database how to use an updated version with your application.
> 
> One additional thing I'd like to see: Don't let the successful importing of
> a 'pytzdata' module be the only way to get an updated tz database.  Create
> an API for someone to supply one themselves at runtime that is cleaner than
> shoving something into sys.modules['pytzdata'].  And define in which order
> they'll be used:
> 
> priority:
>   1) api call supplying tz data to the process.
>   2) pytzdata module if it exists
>   3) tz data from the underlying operating system
>   4) error.

I disagree on this order, at least for Linux systems. the tzdata database is
well managed on major Linux distributions and should be used for this reason.

  Matthias


From greg at krypto.org  Mon Oct  1 00:50:30 2012
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 30 Sep 2012 15:50:30 -0700
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <5068C4E5.9020606@ubuntu.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068C4E5.9020606@ubuntu.com>
Message-ID: <CAGE7PNKoC-BGd+8_4PaOZNTmdmA6UF1_3oGrpbFUSc3au6h-cg@mail.gmail.com>

On Sun, Sep 30, 2012 at 3:17 PM, Matthias Klose <doko at ubuntu.com> wrote:

> On 30.09.2012 20:18, Gregory P. Smith wrote:
> > On Sun, Sep 30, 2012 at 8:33 AM, Benjamin Peterson <benjamin at python.org
> >wrote:
> >
> >> 2012/9/30 Xavier Morel <python-dev at masklinn.net>:
> >>> But at worst, an outdated unicode database will be missing data right?
> >>>
> >>> Doesn't an outdated timezone db have the risk of returning *incorrect*
> >> data?
> >>
> >> Unicode updates also include corrections; however, it seems there are
> >> not significant enough or about obscure enough scripts that not many
> >> notice. :)
> >>
> >
> > We never hear anyone complain because the corrections are not for English
> > or other "western" languages that the majority of us speak.  ;)
> >
> > Regardless, I think including a version of the database on windows
> releases
> > makes sense.  Update it on a best effort basis before each .x minor
> > release.  The documentation can make it clear that it is a changing
> > database how to use an updated version with your application.
> >
> > One additional thing I'd like to see: Don't let the successful importing
> of
> > a 'pytzdata' module be the only way to get an updated tz database.
>  Create
> > an API for someone to supply one themselves at runtime that is cleaner
> than
> > shoving something into sys.modules['pytzdata'].  And define in which
> order
> > they'll be used:
> >
> > priority:
> >   1) api call supplying tz data to the process.
> >   2) pytzdata module if it exists
> >   3) tz data from the underlying operating system
> >   4) error.
>
> I disagree on this order, at least for Linux systems. the tzdata database
> is
> well managed on major Linux distributions and should be used for this
> reason.
>

Agreed, but that is why anyone writing code for Linux will simply never use
1) or 2).  Those exist primarily to handle the Windows and people with
software running on non well managed systems.

I'd add a 3.5) to the above list: tz data bundled with the Python
distribution.  Bundled tz data would likely simply not even be included in
a Python package on a Linux system.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/bad5a23b/attachment.html>

From regebro at gmail.com  Mon Oct  1 00:51:07 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 00:51:07 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <5068C4E5.9020606@ubuntu.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068C4E5.9020606@ubuntu.com>
Message-ID: <CAL0kPAXH8Kev6HR2SR3K+42YeOYOgr7xVijvaiORYRgHWjudzQ@mail.gmail.com>

On Mon, Oct 1, 2012 at 12:17 AM, Matthias Klose <doko at ubuntu.com> wrote:

> > priority:
> >   1) api call supplying tz data to the process.
> >   2) pytzdata module if it exists
> >   3) tz data from the underlying operating system
> >   4) error.
>
> I disagree on this order, at least for Linux systems. the tzdata database
> is
> well managed on major Linux distributions and should be used for this
> reason.
>

Sure, but in the cases where it is not, overriding should be possible. The
OS tz data is last because it is the default.

//Lennart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/95f145b8/attachment.html>

From doko at ubuntu.com  Mon Oct  1 00:57:31 2012
From: doko at ubuntu.com (Matthias Klose)
Date: Mon, 01 Oct 2012 00:57:31 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAGE7PNKoC-BGd+8_4PaOZNTmdmA6UF1_3oGrpbFUSc3au6h-cg@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068C4E5.9020606@ubuntu.com>
	<CAGE7PNKoC-BGd+8_4PaOZNTmdmA6UF1_3oGrpbFUSc3au6h-cg@mail.gmail.com>
Message-ID: <5068CE5B.4010200@ubuntu.com>

On 01.10.2012 00:50, Gregory P. Smith wrote:
> On Sun, Sep 30, 2012 at 3:17 PM, Matthias Klose <doko at ubuntu.com> wrote:
> 
>> On 30.09.2012 20:18, Gregory P. Smith wrote:
>>> On Sun, Sep 30, 2012 at 8:33 AM, Benjamin Peterson <benjamin at python.org
>>> wrote:
>>>
>>>> 2012/9/30 Xavier Morel <python-dev at masklinn.net>:
>>>>> But at worst, an outdated unicode database will be missing data right?
>>>>>
>>>>> Doesn't an outdated timezone db have the risk of returning *incorrect*
>>>> data?
>>>>
>>>> Unicode updates also include corrections; however, it seems there are
>>>> not significant enough or about obscure enough scripts that not many
>>>> notice. :)
>>>>
>>>
>>> We never hear anyone complain because the corrections are not for English
>>> or other "western" languages that the majority of us speak.  ;)
>>>
>>> Regardless, I think including a version of the database on windows
>> releases
>>> makes sense.  Update it on a best effort basis before each .x minor
>>> release.  The documentation can make it clear that it is a changing
>>> database how to use an updated version with your application.
>>>
>>> One additional thing I'd like to see: Don't let the successful importing
>> of
>>> a 'pytzdata' module be the only way to get an updated tz database.
>>  Create
>>> an API for someone to supply one themselves at runtime that is cleaner
>> than
>>> shoving something into sys.modules['pytzdata'].  And define in which
>> order
>>> they'll be used:
>>>
>>> priority:
>>>   1) api call supplying tz data to the process.
>>>   2) pytzdata module if it exists
>>>   3) tz data from the underlying operating system
>>>   4) error.
>>
>> I disagree on this order, at least for Linux systems. the tzdata database
>> is
>> well managed on major Linux distributions and should be used for this
>> reason.
>>
> 
> Agreed, but that is why anyone writing code for Linux will simply never use
> 1) or 2).  Those exist primarily to handle the Windows and people with
> software running on non well managed systems.
> 
> I'd add a 3.5) to the above list: tz data bundled with the Python
> distribution.  Bundled tz data would likely simply not even be included in
> a Python package on a Linux system.

I would add

 - a 2.5) if pytzdata does exist and uses the tz data from the underlying
operating system.
 - a 3.5) if pytzdata does exist and doesn't use the tz data from the underlying
operating system.

however I see our patch to pytzdata to use the os data is a local patch, and not
found upstream.

  Matthias


From doko at ubuntu.com  Mon Oct  1 01:00:10 2012
From: doko at ubuntu.com (Matthias Klose)
Date: Mon, 01 Oct 2012 01:00:10 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAL0kPAXH8Kev6HR2SR3K+42YeOYOgr7xVijvaiORYRgHWjudzQ@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068C4E5.9020606@ubuntu.com>
	<CAL0kPAXH8Kev6HR2SR3K+42YeOYOgr7xVijvaiORYRgHWjudzQ@mail.gmail.com>
Message-ID: <5068CEFA.9040509@ubuntu.com>

On 01.10.2012 00:51, Lennart Regebro wrote:
> On Mon, Oct 1, 2012 at 12:17 AM, Matthias Klose <doko at ubuntu.com> wrote:
> 
>>> priority:
>>>   1) api call supplying tz data to the process.
>>>   2) pytzdata module if it exists
>>>   3) tz data from the underlying operating system
>>>   4) error.
>>
>> I disagree on this order, at least for Linux systems. the tzdata database
>> is
>> well managed on major Linux distributions and should be used for this
>> reason.
>>
> 
> Sure, but in the cases where it is not, overriding should be possible. The
> OS tz data is last because it is the default.

but not overriding with an outdated database.

the new venv stuff makes the distinction to just use the core python
environment, or use the core and site/dist-packages. I don't want to see an
exception to not use something which is not in the core.

  Matthias


From rosuav at gmail.com  Mon Oct  1 01:07:35 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 1 Oct 2012 09:07:35 +1000
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <5068C4E5.9020606@ubuntu.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068C4E5.9020606@ubuntu.com>
Message-ID: <CAPTjJmr60O6Knar3CM3EgNezesHJExteppcgWOeQbRnaD7exGA@mail.gmail.com>

On Mon, Oct 1, 2012 at 8:17 AM, Matthias Klose <doko at ubuntu.com> wrote:
> On 30.09.2012 20:18, Gregory P. Smith wrote:
>> priority:
>>   1) api call supplying tz data to the process.
>>   2) pytzdata module if it exists
>>   3) tz data from the underlying operating system
>>   4) error.
>
> I disagree on this order, at least for Linux systems. the tzdata database is
> well managed on major Linux distributions and should be used for this reason.

There's no guarantee that an individual sysadmin will have OS updates
up-to-date. If, on Linux, the pytzdata module is not installed unless
explicitly called for, that would define pytzdata as high specificity,
ergo it should override the lower specificity of the OS-provided data.

The normal case on Linux will happily use the well-managed and
frequently-updated tzdata.

ChrisA

From solipsis at pitrou.net  Mon Oct  1 01:11:04 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 1 Oct 2012 01:11:04 +0200
Subject: [Python-Dev] benchmarks: Force map to a list to guarantee the
 calculations are performed under
References: <3XVMY61gFXzQYR@mail.python.org>
Message-ID: <20121001011104.6a200c26@pitrou.net>

On Mon,  1 Oct 2012 01:00:42 +0200 (CEST)
brett.cannon <python-checkins at python.org> wrote:
> http://hg.python.org/benchmarks/rev/e9f911fd9bd3
> changeset:   184:e9f911fd9bd3
> user:        Brett Cannon <brett at python.org>
> date:        Sun Sep 30 19:00:32 2012 -0400
> summary:
>   Force map to a list to guarantee the calculations are performed under
> Python 3.
[...]
>  def eval_times_u(func, u):
> -    return map(func, ((i,u) for i in xrange(len(list(u)))))
> +    return list(imap(func, ((i,u) for i in xrange(len(list(u))))))

Ugh, can't you use a list comprehension instead? The code would be much
more idiomatic.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From brett at python.org  Mon Oct  1 01:12:47 2012
From: brett at python.org (Brett Cannon)
Date: Sun, 30 Sep 2012 19:12:47 -0400
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide build)
Message-ID: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>

I am presenting the talk "Python 3.3: Trust Me, It's Better Than 2.7" as
PyCon Argentina and Brasil (and US if they accept the talk). As part of
that talk I need to be able to benchmark Python 3.3 against 2.7 (both from
tip) using the unladen benchmarks (which now include benchmarks from PyPy
that can be relatively easily ported to Python 3).

To make sure the unladen benchmarks run fine against Python 3.3, I did a
fast run of the benchmarks. I figured people might be interested in the
quick-and-dirty results on my 2 GHz Intel Core i7 MacBook Pro w/ 8 GB RAM
and no attempt to control for performance beyond not actively browsing the
web. As I said, quick-and-dirty and not authoritative; all done just to
make sure all the benchmarks could run to completion (including the django,
html5lib, and genshi benchmarks which are only on my laptop ATM until those
projects cut a release with official Python 3 support).

One thing to keep in mind is that many benchmarks use a raw str for things,
so the benchmarks often compare Python 2.7 str vs. Python 3.3 str (i.e. str
vs. unicode). While this might seem unfair, this is what real-world
comparisons in performance will be from users so it's an (somewhat unfair)
comparison that we just have to live with. I might take the time to try to
make some tests run under both raw strings and unicode so both comparisons
are available.

If you care about helping out with the benchmarks (e.g. helping spot where
the iteration counts should be higher, etc.) then head over to the
speed at mailing list.



> python3 perf.py -T --basedir ../benchmarks -f -b py3k
../cpython/builds/2.7-wide/bin/python ../cpython/builds/3.3/bin/python3.3

... output about the command line for the benchmarks ...

### 2to3 ###
0.785234 -> 0.722169: 1.09x faster

### call_method ###
Min: 0.491433 -> 0.414841: 1.18x faster
Avg: 0.493640 -> 0.416564: 1.19x faster
Significant (t=127.21)
Stddev: 0.00170 -> 0.00162: 1.0513x smaller

### call_method_slots ###
Min: 0.492749 -> 0.416280: 1.18x faster
Avg: 0.497888 -> 0.419275: 1.19x faster
Significant (t=61.72)
Stddev: 0.00433 -> 0.00237: 1.8304x smaller

### call_method_unknown ###
Min: 0.575536 -> 0.427234: 1.35x faster
Avg: 0.577286 -> 0.433428: 1.33x faster
Significant (t=66.09)
Stddev: 0.00117 -> 0.00835: 7.1621x larger

### call_simple ###
Min: 0.413011 -> 0.338923: 1.22x faster
Avg: 0.415862 -> 0.340699: 1.22x faster
Significant (t=111.94)
Stddev: 0.00223 -> 0.00134: 1.6616x smaller

### chaos ###
Min: 0.375286 -> 0.435456: 1.16x slower
Avg: 0.382798 -> 0.459515: 1.20x slower
Significant (t=-5.01)
Stddev: 0.01116 -> 0.03234: 2.8980x larger

### fastpickle ###
Min: 0.853560 -> 0.770580: 1.11x faster
Avg: 0.879498 -> 0.776249: 1.13x faster
Significant (t=8.24)
Stddev: 0.02771 -> 0.00407: 6.7995x smaller

### float ###
Min: 0.476596 -> 0.391101: 1.22x faster
Avg: 0.486164 -> 0.411553: 1.18x faster
Significant (t=9.07)
Stddev: 0.01049 -> 0.01511: 1.4411x larger

### formatted_logging ###
Min: 0.346703 -> 0.451643: 1.30x slower
Avg: 0.351218 -> 0.454626: 1.29x slower
Significant (t=-51.50)
Stddev: 0.00376 -> 0.00246: 1.5265x smaller

### genshi ###
Min: 0.275107 -> 0.294309: 1.07x slower
Avg: 0.287433 -> 0.299026: 1.04x slower
Significant (t=-3.82)
Stddev: 0.01077 -> 0.00467: 2.3044x smaller

### go ###
Min: 0.719160 -> 0.781042: 1.09x slower
Avg: 0.729322 -> 0.798135: 1.09x slower
Significant (t=-8.54)
Stddev: 0.01300 -> 0.01248: 1.0415x smaller

### hexiom2 ###
203.842661 -> 187.107363: 1.09x faster

### iterative_count ###
Min: 0.145088 -> 0.153285: 1.06x slower
Avg: 0.146369 -> 0.154425: 1.06x slower
Significant (t=-9.21)
Stddev: 0.00134 -> 0.00142: 1.0569x larger

### json_dump_v2 ###
Min: 3.512367 -> 4.040813: 1.15x slower
Avg: 3.521879 -> 4.057966: 1.15x slower
Significant (t=-64.29)
Stddev: 0.01071 -> 0.01526: 1.4247x larger

### json_load ###
Min: 1.024560 -> 0.642353: 1.60x faster
Avg: 1.025255 -> 0.644000: 1.59x faster
Significant (t=426.59)
Stddev: 0.00049 -> 0.00194: 3.9240x larger

### mako_v2 ###
Min: 0.137584 -> 0.287701: 2.09x slower
Avg: 0.140620 -> 0.293204: 2.09x slower
Significant (t=-296.14)
Stddev: 0.00243 -> 0.00272: 1.1195x larger

### meteor_contest ###
Min: 0.284739 -> 0.254285: 1.12x faster
Avg: 0.286174 -> 0.255323: 1.12x faster
Significant (t=38.02)
Stddev: 0.00124 -> 0.00133: 1.0725x larger

### nbody ###
Min: 0.491416 -> 0.336127: 1.46x faster
Avg: 0.493339 -> 0.337467: 1.46x faster
Significant (t=185.50)
Stddev: 0.00164 -> 0.00092: 1.7927x smaller

### normal_startup ###
Min: 0.639285 -> 0.898157: 1.40x slower
Avg: 0.645513 -> 0.901586: 1.40x slower
Significant (t=-90.10)
Stddev: 0.00575 -> 0.00270: 2.1309x smaller

### nqueens ###
Min: 0.399351 -> 0.429575: 1.08x slower
Avg: 0.403643 -> 0.430284: 1.07x slower
Significant (t=-9.83)
Stddev: 0.00603 -> 0.00053: 11.3092x smaller

### pathlib ###
Min: 0.137462 -> 0.170506: 1.24x slower
Avg: 0.145370 -> 0.172849: 1.19x slower
Significant (t=-11.09)
Stddev: 0.01232 -> 0.00128: 9.6403x smaller

### pidigits ###
Min: 0.400265 -> 0.379307: 1.06x faster
Avg: 0.401755 -> 0.381171: 1.05x faster
Significant (t=14.65)
Stddev: 0.00259 -> 0.00178: 1.4496x smaller

### raytrace ###
Min: 1.770596 -> 1.958350: 1.11x slower
Avg: 1.773719 -> 1.968401: 1.11x slower
Significant (t=-44.19)
Stddev: 0.00439 -> 0.00882: 2.0099x larger

### regex_effbot ###
Min: 0.076566 -> 0.098124: 1.28x slower
Avg: 0.077491 -> 0.098696: 1.27x slower
Significant (t=-54.47)
Stddev: 0.00052 -> 0.00069: 1.3227x larger

### regex_v8 ###
Min: 0.091530 -> 0.109116: 1.19x slower
Avg: 0.092308 -> 0.113627: 1.23x slower
Significant (t=-5.72)
Stddev: 0.00088 -> 0.00829: 9.4271x larger

### richards ###
Min: 0.257974 -> 0.232134: 1.11x faster
Avg: 0.259248 -> 0.234325: 1.11x faster
Significant (t=23.80)
Stddev: 0.00144 -> 0.00185: 1.2823x larger

### simple_logging ###
Min: 0.326569 -> 0.416797: 1.28x slower
Avg: 0.331694 -> 0.418844: 1.26x slower
Significant (t=-36.32)
Stddev: 0.00523 -> 0.00122: 4.3004x smaller

### spectral_norm ###
Min: 0.483011 -> 0.741558: 1.54x slower
Avg: 0.487128 -> 0.749741: 1.54x slower
Significant (t=-57.40)
Stddev: 0.00512 -> 0.00886: 1.7299x larger

### startup_nosite ###
Min: 0.220444 -> 0.374521: 1.70x slower
Avg: 0.222773 -> 0.376785: 1.69x slower
Significant (t=-176.17)
Stddev: 0.00166 -> 0.00221: 1.3331x larger

### threaded_count ###
Min: 0.171352 -> 0.151892: 1.13x faster
Avg: 0.183180 -> 0.153634: 1.19x faster
Significant (t=8.12)
Stddev: 0.00801 -> 0.00140: 5.7241x smaller

### unpack_sequence ###
Min: 0.000075 -> 0.000061: 1.23x faster
Avg: 0.000101 -> 0.000065: 1.54x faster
Significant (t=206.90)
Stddev: 0.00001 -> 0.00000: 3.2374x smaller

The following not significant results are hidden, use -v to show them:
chameleon, fannkuch, fastunpickle, regex_compile, silent_logging

### django ###
Min: 0.868956 -> 0.894571: 1.03x slower
Avg: 0.873620 -> 0.905274: 1.04x slower
Significant (t=-6.97)
Stddev: 0.00313 -> 0.00966: 3.0912x larger

### genshi ###
Min: 0.269615 -> 0.286348: 1.06x slower
Avg: 0.272206 -> 0.290708: 1.07x slower
Significant (t=-12.29)
Stddev: 0.00253 -> 0.00526: 2.0793x larger

### html5lib ###
12.279808 -> 11.862586: 1.04x faster
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/ea3e95d3/attachment.html>

From doko at ubuntu.com  Mon Oct  1 01:10:38 2012
From: doko at ubuntu.com (Matthias Klose)
Date: Mon, 01 Oct 2012 01:10:38 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAPTjJmr60O6Knar3CM3EgNezesHJExteppcgWOeQbRnaD7exGA@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068C4E5.9020606@ubuntu.com>
	<CAPTjJmr60O6Knar3CM3EgNezesHJExteppcgWOeQbRnaD7exGA@mail.gmail.com>
Message-ID: <5068D16E.3030509@ubuntu.com>

On 01.10.2012 01:07, Chris Angelico wrote:
> On Mon, Oct 1, 2012 at 8:17 AM, Matthias Klose <doko at ubuntu.com> wrote:
>> On 30.09.2012 20:18, Gregory P. Smith wrote:
>>> priority:
>>>   1) api call supplying tz data to the process.
>>>   2) pytzdata module if it exists
>>>   3) tz data from the underlying operating system
>>>   4) error.
>>
>> I disagree on this order, at least for Linux systems. the tzdata database is
>> well managed on major Linux distributions and should be used for this reason.
> 
> There's no guarantee that an individual sysadmin will have OS updates
> up-to-date. If, on Linux, the pytzdata module is not installed unless
> explicitly called for, that would define pytzdata as high specificity,
> ergo it should override the lower specificity of the OS-provided data.
> 
> The normal case on Linux will happily use the well-managed and
> frequently-updated tzdata.

I don't care as long as pytzdata is a third party module. I do care if it
belongs to the standard library.

I don't want to see a python and perl/java/foo application seeing different
local times.

  Matthias


From brett at python.org  Mon Oct  1 01:18:26 2012
From: brett at python.org (Brett Cannon)
Date: Sun, 30 Sep 2012 19:18:26 -0400
Subject: [Python-Dev] benchmarks: Force map to a list to guarantee the
 calculations are performed under
In-Reply-To: <20121001011104.6a200c26@pitrou.net>
References: <3XVMY61gFXzQYR@mail.python.org>
	<20121001011104.6a200c26@pitrou.net>
Message-ID: <CAP1=2W4T81rC11iijFoDnbCYB7KrVqLB=xXNvAv4-ESKrMj5kQ@mail.gmail.com>

On Sun, Sep 30, 2012 at 7:11 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Mon,  1 Oct 2012 01:00:42 +0200 (CEST)
> brett.cannon <python-checkins at python.org> wrote:
> > http://hg.python.org/benchmarks/rev/e9f911fd9bd3
> > changeset:   184:e9f911fd9bd3
> > user:        Brett Cannon <brett at python.org>
> > date:        Sun Sep 30 19:00:32 2012 -0400
> > summary:
> >   Force map to a list to guarantee the calculations are performed under
> > Python 3.
> [...]
> >  def eval_times_u(func, u):
> > -    return map(func, ((i,u) for i in xrange(len(list(u)))))
> > +    return list(imap(func, ((i,u) for i in xrange(len(list(u))))))
>
> Ugh, can't you use a list comprehension instead? The code would be much
> more idiomatic.
>

Done. Just went with the fastest fix without thinking.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/9f2a3778/attachment.html>

From christian at python.org  Mon Oct  1 01:24:09 2012
From: christian at python.org (Christian Heimes)
Date: Mon, 01 Oct 2012 01:24:09 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
Message-ID: <5068D499.6000802@python.org>

Am 30.09.2012 20:18, schrieb Gregory P. Smith:
> We never hear anyone complain because the corrections are not for
> English or other "western" languages that the majority of us speak.  ;)
> 
> Regardless, I think including a version of the database on windows
> releases makes sense.  Update it on a best effort basis before each .x
> minor release.  The documentation can make it clear that it is a
> changing database how to use an updated version with your application.
> 
> One additional thing I'd like to see: Don't let the successful importing
> of a 'pytzdata' module be the only way to get an updated tz database.
>  Create an API for someone to supply one themselves at runtime that is
> cleaner than shoving something into sys.modules['pytzdata'].  And define
> in which order they'll be used:
> 
> priority:
>   1) api call supplying tz data to the process.
>   2) pytzdata module if it exists
>   3) tz data from the underlying operating system
>   4) error.

I like your basic approach but I'm suggesting a slightly different
approach. Before I explain my proposal I like to get a naming convention
straight.

integrated tz database
----------------------

A copy of the Olson database that is shipped with every version of
Python (not just Windows). The integrated database is updated with every
feature release of Python.


system tz database
------------------

That's an interface to the operating system's or platform's tz database.
We probably have to provide multiple backends for different OSes, Java etc.


update tz database
------------------

A PyPI package that contains a current version of the Olson database. A
new version of the update tz should be provided by the Python core
developer team every time a new Olson database is available. The
updatetz must never be distributed with the Python source code and shall
not be installed by a distributor.
Optionally we can include the code that creates an update tz package
from a Olson database.


By default Python tries to load the updatetz first, then systemtz (if
available) and finally the integratedtz. A user can query the status and
version of the databases, set and get the currently used database with
three functions. The used database can also be set with an environment
variable:

 datetime.gettzdatabase() -> "integrated" or "system" or "update"
 datetime.settzdatabase(name)
 datetime.listtzdatabases() ->
   {"integrated": "version",
    "system": "???"
    "update": "version", # only if an update tz db is installed
   }
 PYTHON_TZDB = "integrated" or "system" or "update"

With this setup users get the full benefit of system updates but can use
the integrated or update database if they don't like the operating
system's data.

Christian


From brett at python.org  Mon Oct  1 01:50:12 2012
From: brett at python.org (Brett Cannon)
Date: Sun, 30 Sep 2012 19:50:12 -0400
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
In-Reply-To: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
Message-ID: <CAP1=2W4Wvr3dLAFh9aXNGuDOQkGV9F54XmMEinXgiHyB16jcYg@mail.gmail.com>

I accidentally left out the telco benchmark, which is bad since cdecimal
makes it just scream on Python 3.3 (and I verified with Python 3.2 that
this is an actual speedup and not some silly screw-up like I initially had
with spectral_norm):

### telco ###
Min: 0.897108 -> 0.016880: 53.15x faster
Avg: 0.899742 -> 0.017443: 51.58x faster
Significant (t=692.55)
Stddev: 0.00283 -> 0.00032: 8.8470x smaller

On Sun, Sep 30, 2012 at 7:12 PM, Brett Cannon <brett at python.org> wrote:

> I am presenting the talk "Python 3.3: Trust Me, It's Better Than 2.7" as
> PyCon Argentina and Brasil (and US if they accept the talk). As part of
> that talk I need to be able to benchmark Python 3.3 against 2.7 (both from
> tip) using the unladen benchmarks (which now include benchmarks from PyPy
> that can be relatively easily ported to Python 3).
>
> To make sure the unladen benchmarks run fine against Python 3.3, I did a
> fast run of the benchmarks. I figured people might be interested in the
> quick-and-dirty results on my 2 GHz Intel Core i7 MacBook Pro w/ 8 GB RAM
> and no attempt to control for performance beyond not actively browsing the
> web. As I said, quick-and-dirty and not authoritative; all done just to
> make sure all the benchmarks could run to completion (including the django,
> html5lib, and genshi benchmarks which are only on my laptop ATM until those
> projects cut a release with official Python 3 support).
>
> One thing to keep in mind is that many benchmarks use a raw str for
> things, so the benchmarks often compare Python 2.7 str vs. Python 3.3 str
> (i.e. str vs. unicode). While this might seem unfair, this is what
> real-world comparisons in performance will be from users so it's an
> (somewhat unfair) comparison that we just have to live with. I might take
> the time to try to make some tests run under both raw strings and unicode
> so both comparisons are available.
>
> If you care about helping out with the benchmarks (e.g. helping spot where
> the iteration counts should be higher, etc.) then head over to the speed at mailing list.
>
>
>
> > python3 perf.py -T --basedir ../benchmarks -f -b py3k
> ../cpython/builds/2.7-wide/bin/python ../cpython/builds/3.3/bin/python3.3
>
> ... output about the command line for the benchmarks ...
>
> ### 2to3 ###
> 0.785234 -> 0.722169: 1.09x faster
>
> ### call_method ###
> Min: 0.491433 -> 0.414841: 1.18x faster
> Avg: 0.493640 -> 0.416564: 1.19x faster
> Significant (t=127.21)
> Stddev: 0.00170 -> 0.00162: 1.0513x smaller
>
> ### call_method_slots ###
> Min: 0.492749 -> 0.416280: 1.18x faster
> Avg: 0.497888 -> 0.419275: 1.19x faster
> Significant (t=61.72)
> Stddev: 0.00433 -> 0.00237: 1.8304x smaller
>
> ### call_method_unknown ###
> Min: 0.575536 -> 0.427234: 1.35x faster
> Avg: 0.577286 -> 0.433428: 1.33x faster
> Significant (t=66.09)
> Stddev: 0.00117 -> 0.00835: 7.1621x larger
>
> ### call_simple ###
> Min: 0.413011 -> 0.338923: 1.22x faster
> Avg: 0.415862 -> 0.340699: 1.22x faster
> Significant (t=111.94)
> Stddev: 0.00223 -> 0.00134: 1.6616x smaller
>
> ### chaos ###
> Min: 0.375286 -> 0.435456: 1.16x slower
> Avg: 0.382798 -> 0.459515: 1.20x slower
> Significant (t=-5.01)
> Stddev: 0.01116 -> 0.03234: 2.8980x larger
>
> ### fastpickle ###
> Min: 0.853560 -> 0.770580: 1.11x faster
> Avg: 0.879498 -> 0.776249: 1.13x faster
> Significant (t=8.24)
> Stddev: 0.02771 -> 0.00407: 6.7995x smaller
>
> ### float ###
> Min: 0.476596 -> 0.391101: 1.22x faster
> Avg: 0.486164 -> 0.411553: 1.18x faster
> Significant (t=9.07)
> Stddev: 0.01049 -> 0.01511: 1.4411x larger
>
> ### formatted_logging ###
> Min: 0.346703 -> 0.451643: 1.30x slower
> Avg: 0.351218 -> 0.454626: 1.29x slower
> Significant (t=-51.50)
> Stddev: 0.00376 -> 0.00246: 1.5265x smaller
>
> ### genshi ###
> Min: 0.275107 -> 0.294309: 1.07x slower
> Avg: 0.287433 -> 0.299026: 1.04x slower
> Significant (t=-3.82)
> Stddev: 0.01077 -> 0.00467: 2.3044x smaller
>
> ### go ###
> Min: 0.719160 -> 0.781042: 1.09x slower
> Avg: 0.729322 -> 0.798135: 1.09x slower
> Significant (t=-8.54)
> Stddev: 0.01300 -> 0.01248: 1.0415x smaller
>
> ### hexiom2 ###
> 203.842661 -> 187.107363: 1.09x faster
>
> ### iterative_count ###
> Min: 0.145088 -> 0.153285: 1.06x slower
> Avg: 0.146369 -> 0.154425: 1.06x slower
> Significant (t=-9.21)
> Stddev: 0.00134 -> 0.00142: 1.0569x larger
>
> ### json_dump_v2 ###
> Min: 3.512367 -> 4.040813: 1.15x slower
> Avg: 3.521879 -> 4.057966: 1.15x slower
> Significant (t=-64.29)
> Stddev: 0.01071 -> 0.01526: 1.4247x larger
>
> ### json_load ###
> Min: 1.024560 -> 0.642353: 1.60x faster
> Avg: 1.025255 -> 0.644000: 1.59x faster
> Significant (t=426.59)
> Stddev: 0.00049 -> 0.00194: 3.9240x larger
>
> ### mako_v2 ###
> Min: 0.137584 -> 0.287701: 2.09x slower
> Avg: 0.140620 -> 0.293204: 2.09x slower
> Significant (t=-296.14)
> Stddev: 0.00243 -> 0.00272: 1.1195x larger
>
> ### meteor_contest ###
> Min: 0.284739 -> 0.254285: 1.12x faster
> Avg: 0.286174 -> 0.255323: 1.12x faster
> Significant (t=38.02)
> Stddev: 0.00124 -> 0.00133: 1.0725x larger
>
> ### nbody ###
> Min: 0.491416 -> 0.336127: 1.46x faster
> Avg: 0.493339 -> 0.337467: 1.46x faster
> Significant (t=185.50)
> Stddev: 0.00164 -> 0.00092: 1.7927x smaller
>
> ### normal_startup ###
> Min: 0.639285 -> 0.898157: 1.40x slower
> Avg: 0.645513 -> 0.901586: 1.40x slower
> Significant (t=-90.10)
> Stddev: 0.00575 -> 0.00270: 2.1309x smaller
>
> ### nqueens ###
> Min: 0.399351 -> 0.429575: 1.08x slower
> Avg: 0.403643 -> 0.430284: 1.07x slower
> Significant (t=-9.83)
> Stddev: 0.00603 -> 0.00053: 11.3092x smaller
>
> ### pathlib ###
> Min: 0.137462 -> 0.170506: 1.24x slower
> Avg: 0.145370 -> 0.172849: 1.19x slower
> Significant (t=-11.09)
> Stddev: 0.01232 -> 0.00128: 9.6403x smaller
>
> ### pidigits ###
> Min: 0.400265 -> 0.379307: 1.06x faster
> Avg: 0.401755 -> 0.381171: 1.05x faster
> Significant (t=14.65)
> Stddev: 0.00259 -> 0.00178: 1.4496x smaller
>
> ### raytrace ###
> Min: 1.770596 -> 1.958350: 1.11x slower
> Avg: 1.773719 -> 1.968401: 1.11x slower
> Significant (t=-44.19)
> Stddev: 0.00439 -> 0.00882: 2.0099x larger
>
> ### regex_effbot ###
> Min: 0.076566 -> 0.098124: 1.28x slower
> Avg: 0.077491 -> 0.098696: 1.27x slower
> Significant (t=-54.47)
> Stddev: 0.00052 -> 0.00069: 1.3227x larger
>
> ### regex_v8 ###
> Min: 0.091530 -> 0.109116: 1.19x slower
> Avg: 0.092308 -> 0.113627: 1.23x slower
> Significant (t=-5.72)
> Stddev: 0.00088 -> 0.00829: 9.4271x larger
>
> ### richards ###
> Min: 0.257974 -> 0.232134: 1.11x faster
> Avg: 0.259248 -> 0.234325: 1.11x faster
> Significant (t=23.80)
> Stddev: 0.00144 -> 0.00185: 1.2823x larger
>
> ### simple_logging ###
> Min: 0.326569 -> 0.416797: 1.28x slower
> Avg: 0.331694 -> 0.418844: 1.26x slower
> Significant (t=-36.32)
> Stddev: 0.00523 -> 0.00122: 4.3004x smaller
>
> ### spectral_norm ###
> Min: 0.483011 -> 0.741558: 1.54x slower
> Avg: 0.487128 -> 0.749741: 1.54x slower
> Significant (t=-57.40)
> Stddev: 0.00512 -> 0.00886: 1.7299x larger
>
> ### startup_nosite ###
> Min: 0.220444 -> 0.374521: 1.70x slower
> Avg: 0.222773 -> 0.376785: 1.69x slower
> Significant (t=-176.17)
> Stddev: 0.00166 -> 0.00221: 1.3331x larger
>
> ### threaded_count ###
> Min: 0.171352 -> 0.151892: 1.13x faster
> Avg: 0.183180 -> 0.153634: 1.19x faster
> Significant (t=8.12)
> Stddev: 0.00801 -> 0.00140: 5.7241x smaller
>
> ### unpack_sequence ###
> Min: 0.000075 -> 0.000061: 1.23x faster
> Avg: 0.000101 -> 0.000065: 1.54x faster
> Significant (t=206.90)
> Stddev: 0.00001 -> 0.00000: 3.2374x smaller
>
> The following not significant results are hidden, use -v to show them:
> chameleon, fannkuch, fastunpickle, regex_compile, silent_logging
>
> ### django ###
> Min: 0.868956 -> 0.894571: 1.03x slower
> Avg: 0.873620 -> 0.905274: 1.04x slower
> Significant (t=-6.97)
> Stddev: 0.00313 -> 0.00966: 3.0912x larger
>
> ### genshi ###
> Min: 0.269615 -> 0.286348: 1.06x slower
> Avg: 0.272206 -> 0.290708: 1.07x slower
> Significant (t=-12.29)
> Stddev: 0.00253 -> 0.00526: 2.0793x larger
>
> ### html5lib ###
> 12.279808 -> 11.862586: 1.04x faster
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/57821375/attachment.html>

From alexandre at peadrop.com  Mon Oct  1 02:07:10 2012
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Sun, 30 Sep 2012 17:07:10 -0700
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
In-Reply-To: <CAP1=2W4Wvr3dLAFh9aXNGuDOQkGV9F54XmMEinXgiHyB16jcYg@mail.gmail.com>
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
	<CAP1=2W4Wvr3dLAFh9aXNGuDOQkGV9F54XmMEinXgiHyB16jcYg@mail.gmail.com>
Message-ID: <CANcUUefNggQ48SuqcO=xrw_UCwz+GzPK=BXgtW=dYkLZA4yV5w@mail.gmail.com>

On Sun, Sep 30, 2012 at 4:50 PM, Brett Cannon <brett at python.org> wrote:

> I accidentally left out the telco benchmark, which is bad since cdecimal
> makes it just scream on Python 3.3 (and I verified with Python 3.2 that
> this is an actual speedup and not some silly screw-up like I initially had
> with spectral_norm):


You could also make the pickle benchmark use the C accelerator module by
passing the --use_cpickle flag. The Python 3 version should be a lot faster.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/f9bc711e/attachment.html>

From greg at krypto.org  Mon Oct  1 02:14:16 2012
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 30 Sep 2012 17:14:16 -0700
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
In-Reply-To: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
Message-ID: <CAGE7PNKS7cO_y94-1aJvc+XHG7nPhzi35ETM3SYThBBsk24BAQ@mail.gmail.com>

Interesting results!

Another data point for the benchmarks that would be interesting is memory
consumption of the python process during the runs.

In 3.3 a reasonable place to gather this would be to add a callback to the
new gc.callbacks and save a snapshot of the process's memory usage before
every collection to gather peak, average and median usage over the life of
the process.  2.7 doesn't have this feature but there is a backport of this
to 2.7 in the bugtracker.

I guess I should join speed@ :)

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/31c5acc0/attachment.html>

From solipsis at pitrou.net  Mon Oct  1 02:28:54 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 1 Oct 2012 02:28:54 +0200
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
Message-ID: <20121001022854.1e49615a@pitrou.net>

On Sun, 30 Sep 2012 19:12:47 -0400
Brett Cannon <brett at python.org> wrote:
> 
> ### mako_v2 ###
> Min: 0.137584 -> 0.287701: 2.09x slower
> Avg: 0.140620 -> 0.293204: 2.09x slower
> Significant (t=-296.14)
> Stddev: 0.00243 -> 0.00272: 1.1195x larger

Note that Mako can use the Markupsafe library for faster operation.
This will skew the result if one of your Pythons has Markupsafe
installed and the other does not.

Perhaps the benchmark runner should launch its subtests in a controlled
environment to avoid such issues?
(shipping a copy of Markupsafe would not be very practical, because it
has a C extension)

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From brett at python.org  Mon Oct  1 03:40:47 2012
From: brett at python.org (Brett Cannon)
Date: Sun, 30 Sep 2012 21:40:47 -0400
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
In-Reply-To: <CANcUUefNggQ48SuqcO=xrw_UCwz+GzPK=BXgtW=dYkLZA4yV5w@mail.gmail.com>
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
	<CAP1=2W4Wvr3dLAFh9aXNGuDOQkGV9F54XmMEinXgiHyB16jcYg@mail.gmail.com>
	<CANcUUefNggQ48SuqcO=xrw_UCwz+GzPK=BXgtW=dYkLZA4yV5w@mail.gmail.com>
Message-ID: <CAP1=2W7pFBh_=HS1TWf5JvJM3XOqNm41K+kvAx5tXN_V9R8ivw@mail.gmail.com>

On Sun, Sep 30, 2012 at 8:07 PM, Alexandre Vassalotti <alexandre at peadrop.com
> wrote:

>
>
> On Sun, Sep 30, 2012 at 4:50 PM, Brett Cannon <brett at python.org> wrote:
>
>> I accidentally left out the telco benchmark, which is bad since cdecimal
>> makes it just scream on Python 3.3 (and I verified with Python 3.2 that
>> this is an actual speedup and not some silly screw-up like I initially had
>> with spectral_norm):
>
>
> You could also make the pickle benchmark use the C accelerator module by
> passing the --use_cpickle flag. The Python 3 version should be a lot faster.
>

perf.py already uses --use_cpickle:

Running fastpickle...
INFO:root:Running ../cpython/builds/3.3/bin/python3.3
performance/bm_pickle.py -n 5 --use_cpickle pickle
INFO:root:Running ../cpython/builds/2.7-wide/bin/python
performance/bm_pickle.py -n 5 --use_cpickle pickle

One thing that might make a change is using -1 for the protocol instead of
2, but that means losing the perk of perf.py doing all of the calculations,
etc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/8217516e/attachment-0001.html>

From steve at pearwood.info  Mon Oct  1 03:35:58 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 1 Oct 2012 11:35:58 +1000
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
In-Reply-To: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
Message-ID: <20121001013558.GB8499@ando>

On Sun, Sep 30, 2012 at 07:12:47PM -0400, Brett Cannon wrote:

> > python3 perf.py -T --basedir ../benchmarks -f -b py3k
> ../cpython/builds/2.7-wide/bin/python ../cpython/builds/3.3/bin/python3.3

> ### call_method ###
> Min: 0.491433 -> 0.414841: 1.18x faster
> Avg: 0.493640 -> 0.416564: 1.19x faster
> Significant (t=127.21)
> Stddev: 0.00170 -> 0.00162: 1.0513x smaller

I'm not sure if this is the right place to discuss this, but what is the 
justification for recording the average and std deviation of the 
benchmarks?

If the benchmarks are based on timeit, the timeit docs warn against 
taking any statistic other than the minimum.



-- 
Steven

From brett at python.org  Mon Oct  1 03:41:30 2012
From: brett at python.org (Brett Cannon)
Date: Sun, 30 Sep 2012 21:41:30 -0400
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
In-Reply-To: <CAGE7PNKS7cO_y94-1aJvc+XHG7nPhzi35ETM3SYThBBsk24BAQ@mail.gmail.com>
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
	<CAGE7PNKS7cO_y94-1aJvc+XHG7nPhzi35ETM3SYThBBsk24BAQ@mail.gmail.com>
Message-ID: <CAP1=2W7qsW-VoTJeaNzfMoPWzgx8UUxHmUypioPdgxEWM4YmEg@mail.gmail.com>

On Sun, Sep 30, 2012 at 8:14 PM, Gregory P. Smith <greg at krypto.org> wrote:

> Interesting results!
>
> Another data point for the benchmarks that would be interesting is memory
> consumption of the python process during the runs.
>
> In 3.3 a reasonable place to gather this would be to add a callback to the
> new gc.callbacks and save a snapshot of the process's memory usage before
> every collection to gather peak, average and median usage over the life of
> the process.  2.7 doesn't have this feature but there is a backport of this
> to 2.7 in the bugtracker.
>
> I guess I should join speed@ :)
>
>
There is already support in perf.py to track memory:

-m, --track_memory    Track memory usage. This only works on Linux.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/9f55a876/attachment.html>

From brett at python.org  Mon Oct  1 03:49:20 2012
From: brett at python.org (Brett Cannon)
Date: Sun, 30 Sep 2012 21:49:20 -0400
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
In-Reply-To: <20121001022854.1e49615a@pitrou.net>
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
	<20121001022854.1e49615a@pitrou.net>
Message-ID: <CAP1=2W7c2U50=2xmYjDmhNhn_WBHK+X-u-ayZ41zOus0FP3m-g@mail.gmail.com>

On Sun, Sep 30, 2012 at 8:28 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sun, 30 Sep 2012 19:12:47 -0400
> Brett Cannon <brett at python.org> wrote:
> >
> > ### mako_v2 ###
> > Min: 0.137584 -> 0.287701: 2.09x slower
> > Avg: 0.140620 -> 0.293204: 2.09x slower
> > Significant (t=-296.14)
> > Stddev: 0.00243 -> 0.00272: 1.1195x larger
>
> Note that Mako can use the Markupsafe library for faster operation.
> This will skew the result if one of your Pythons has Markupsafe
> installed and the other does not.
>

Should probably have the benchmark print out a warning when markupsafe is
used. Turns out I have it installed in my user directory for Python 2.7 so
that probably came into play.


>
> Perhaps the benchmark runner should launch its subtests in a controlled
> environment to avoid such issues?
>

If we had venv in Python 2.7 that might be easy to do, but otherwise is
there an easy way without having to try to pull in virtualenv or something
crazy like a chroot or something?

-Brett


(shipping a copy of Markupsafe would not be very practical, because it
> has a C extension)
>
> Regards
>
> Antoine.
>
>
> --
> Software development and contracting: http://pro.pitrou.net
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/61e7745f/attachment.html>

From brett at python.org  Mon Oct  1 03:51:32 2012
From: brett at python.org (Brett Cannon)
Date: Sun, 30 Sep 2012 21:51:32 -0400
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
In-Reply-To: <20121001013558.GB8499@ando>
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
	<20121001013558.GB8499@ando>
Message-ID: <CAP1=2W5p-a0tKA3KkYGF27VZaruUx95Hn+TKfjASotkyeqbKYw@mail.gmail.com>

On Sun, Sep 30, 2012 at 9:35 PM, Steven D'Aprano <steve at pearwood.info>wrote:

> On Sun, Sep 30, 2012 at 07:12:47PM -0400, Brett Cannon wrote:
>
> > > python3 perf.py -T --basedir ../benchmarks -f -b py3k
> > ../cpython/builds/2.7-wide/bin/python ../cpython/builds/3.3/bin/python3.3
>
> > ### call_method ###
> > Min: 0.491433 -> 0.414841: 1.18x faster
> > Avg: 0.493640 -> 0.416564: 1.19x faster
> > Significant (t=127.21)
> > Stddev: 0.00170 -> 0.00162: 1.0513x smaller
>
> I'm not sure if this is the right place to discuss this,


The speed mailing list would be best.


> but what is the
> justification for recording the average and std deviation of the
> benchmarks?
>

Because the tests, when run in a more rigorous fashion, run many more
iterations so the average is used to even out bumps thanks to executing,
e.g. 50 times. And the stddev is there to know how variable the results
were in the end.


>
> If the benchmarks are based on timeit, the timeit docs warn against
> taking any statistic other than the minimum.
>

They don't use timeit.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/cd42b02e/attachment.html>

From robertc at robertcollins.net  Mon Oct  1 07:07:15 2012
From: robertc at robertcollins.net (Robert Collins)
Date: Mon, 1 Oct 2012 18:07:15 +1300
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
In-Reply-To: <20121001013558.GB8499@ando>
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
	<20121001013558.GB8499@ando>
Message-ID: <CAJ3HoZ2L9ETOry_6J+M4DDEZ0sr_0iErc6XmsSq65rGGbZpY8Q@mail.gmail.com>

On Mon, Oct 1, 2012 at 2:35 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sun, Sep 30, 2012 at 07:12:47PM -0400, Brett Cannon wrote:
>
>> > python3 perf.py -T --basedir ../benchmarks -f -b py3k
>> ../cpython/builds/2.7-wide/bin/python ../cpython/builds/3.3/bin/python3.3
>
>> ### call_method ###
>> Min: 0.491433 -> 0.414841: 1.18x faster
>> Avg: 0.493640 -> 0.416564: 1.19x faster
>> Significant (t=127.21)
>> Stddev: 0.00170 -> 0.00162: 1.0513x smaller
>
> I'm not sure if this is the right place to discuss this, but what is the
> justification for recording the average and std deviation of the
> benchmarks?
>
> If the benchmarks are based on timeit, the timeit docs warn against
> taking any statistic other than the minimum.

Also because timeit is wrong to give that recommendation.

There are factors - such as garbage collection - that affect
operations on average, even though they may not kick in in every run.
If you want to know how something will perform as part of a larger
system, taking the best possible and extrapolating from it is a
mistake. As a concrete example, consider an algorithm that generates
cycles with several hundred MB of memory in them. Best case the RAM is
available, nothing swaps, and gc doesn't kick in during the
algorithm's execution. However, the larger program has to deal with
those several hundred MB of memory sitting around until gc *does* kick
in, has to pay the price of a gc run over a large heap, and deal with
the impact on disk read cache. When you do enough runs to see those
effects *that will affect the whole program* kick in, then you can
extrapolate from that basis. e.g. the question timeit optimises itself
to answer isn't the question most folk need most of the time.

-Rob

From regebro at gmail.com  Mon Oct  1 09:02:08 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 09:02:08 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <5068D499.6000802@python.org>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
Message-ID: <CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>

On Mon, Oct 1, 2012 at 1:24 AM, Christian Heimes <christian at python.org>wrote:

> I like your basic approach but I'm suggesting a slightly different
> approach. Before I explain my proposal I like to get a naming convention
> straight.
>
> integrated tz database
> ----------------------
>
> A copy of the Olson database that is shipped with every version of
> Python (not just Windows). The integrated database is updated with every
> feature release of Python.
>
>
> system tz database
> ------------------
>
> That's an interface to the operating system's or platform's tz database.
> We probably have to provide multiple backends for different OSes, Java etc.
>
>
> update tz database
> ------------------
>
> A PyPI package that contains a current version of the Olson database. A
> new version of the update tz should be provided by the Python core
> developer team every time a new Olson database is available. The
> updatetz must never be distributed with the Python source code and shall
> not be installed by a distributor.
> Optionally we can include the code that creates an update tz package
> from a Olson database.
>
>
> By default Python tries to load the updatetz first, then systemtz (if
> available) and finally the integratedtz. A user can query the status and
> version of the databases, set and get the currently used database with
> three functions. The used database can also be set with an environment
> variable:
>
>  datetime.gettzdatabase() -> "integrated" or "system" or "update"
>  datetime.settzdatabase(name)
>  datetime.listtzdatabases() ->
>    {"integrated": "version",
>     "system": "???"
>     "update": "version", # only if an update tz db is installed
>    }
>  PYTHON_TZDB = "integrated" or "system" or "update"
>
> With this setup users get the full benefit of system updates but can use
> the integrated or update database if they don't like the operating
> system's data.
>

I don't want to default to a database that is guaranteed to be out of date.
I don't see the purpose. This is also why I'm skeptical at including the
data on Windows.

//Lennart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/0814726b/attachment.html>

From fijall at gmail.com  Mon Oct  1 09:43:07 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 1 Oct 2012 09:43:07 +0200
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
In-Reply-To: <CAJ3HoZ2L9ETOry_6J+M4DDEZ0sr_0iErc6XmsSq65rGGbZpY8Q@mail.gmail.com>
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
	<20121001013558.GB8499@ando>
	<CAJ3HoZ2L9ETOry_6J+M4DDEZ0sr_0iErc6XmsSq65rGGbZpY8Q@mail.gmail.com>
Message-ID: <CAK5idxQb_478RnxEV33uexHOiuSvK22Q5rcqgggs0Uev0X0dOg@mail.gmail.com>

On Mon, Oct 1, 2012 at 7:07 AM, Robert Collins
<robertc at robertcollins.net> wrote:
> On Mon, Oct 1, 2012 at 2:35 PM, Steven D'Aprano <steve at pearwood.info> wrote:
>> On Sun, Sep 30, 2012 at 07:12:47PM -0400, Brett Cannon wrote:
>>
>>> > python3 perf.py -T --basedir ../benchmarks -f -b py3k
>>> ../cpython/builds/2.7-wide/bin/python ../cpython/builds/3.3/bin/python3.3
>>
>>> ### call_method ###
>>> Min: 0.491433 -> 0.414841: 1.18x faster
>>> Avg: 0.493640 -> 0.416564: 1.19x faster
>>> Significant (t=127.21)
>>> Stddev: 0.00170 -> 0.00162: 1.0513x smaller
>>
>> I'm not sure if this is the right place to discuss this, but what is the
>> justification for recording the average and std deviation of the
>> benchmarks?
>>
>> If the benchmarks are based on timeit, the timeit docs warn against
>> taking any statistic other than the minimum.
>
> Also because timeit is wrong to give that recommendation.
>
> There are factors - such as garbage collection - that affect
> operations on average, even though they may not kick in in every run.
> If you want to know how something will perform as part of a larger
> system, taking the best possible and extrapolating from it is a
> mistake. As a concrete example, consider an algorithm that generates
> cycles with several hundred MB of memory in them. Best case the RAM is
> available, nothing swaps, and gc doesn't kick in during the
> algorithm's execution. However, the larger program has to deal with
> those several hundred MB of memory sitting around until gc *does* kick
> in, has to pay the price of a gc run over a large heap, and deal with
> the impact on disk read cache. When you do enough runs to see those
> effects *that will affect the whole program* kick in, then you can
> extrapolate from that basis. e.g. the question timeit optimises itself
> to answer isn't the question most folk need most of the time.
>
> -Rob
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com

Timeit disables the GC for good measure (which is very bad IMO, but it
was already discussed)

From fijall at gmail.com  Mon Oct  1 09:44:25 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 1 Oct 2012 09:44:25 +0200
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
In-Reply-To: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
Message-ID: <CAK5idxSMrr3+WafADoRf6v0XWfrRvVgMAqxQwjwFQNShBtuwFg@mail.gmail.com>

On Mon, Oct 1, 2012 at 1:12 AM, Brett Cannon <brett at python.org> wrote:
> I am presenting the talk "Python 3.3: Trust Me, It's Better Than 2.7" as
> PyCon Argentina and Brasil (and US if they accept the talk). As part of that
> talk I need to be able to benchmark Python 3.3 against 2.7 (both from tip)
> using the unladen benchmarks (which now include benchmarks from PyPy that
> can be relatively easily ported to Python 3).
>

Hi Brett.

*If* you're talking about benchmarks, would be cool if you mention
that pypy is actually much faster on most of them. Also a very sad
fact is that a lot of actually interesting benchmarks don't work on
py3k (although a growing number). Twisted and sympy are very
informative for example

From dirkjan at ochtman.nl  Mon Oct  1 10:28:11 2012
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Mon, 1 Oct 2012 10:28:11 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
Message-ID: <CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>

On Mon, Oct 1, 2012 at 9:02 AM, Lennart Regebro <regebro at gmail.com> wrote:
> I don't want to default to a database that is guaranteed to be out of date.
> I don't see the purpose. This is also why I'm skeptical at including the
> data on Windows.

I think that would be a little too pure to be practical. There are
many practical usages of tz data that would work fine with a year-old
timezone database.

Personally, I'd want to not ship any tzdata with non-Windows Python
packages on the assumption that they have good up-to-date OS tzdata
(or it should be easy to disable it in the configure phase). Also, I
wonder if it would be possible to select the copy of the data that is
the most recent (e.g. on Unix, pick the OS version if tzupdate is
installed but older than the OS version).

Cheers,

Dirkjan

From regebro at gmail.com  Mon Oct  1 10:47:25 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 10:47:25 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
Message-ID: <CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>

On Mon, Oct 1, 2012 at 10:28 AM, Dirkjan Ochtman <dirkjan at ochtman.nl> wrote:

> I think that would be a little too pure to be practical. There are
> many practical usages of tz data that would work fine with a year-old
> timezone database.
>

A year is no age for a Python installation. A customer of mine has one
website developed in 2003, still running on the same server. It runs Python
2.3, I don't remember which version, but I'd be surprised if it is 2.3.7
from 2008.


> Also, I
> wonder if it would be possible to select the copy of the data that is
> the most recent (e.g. on Unix, pick the OS version if tzupdate is
> installed but older than the OS version).


I don't think so. As far as I can tell, the data files contain no version
information (only information on the version of the file format, which
currently is 2).
It also contains no information on what the name of the timezone is. This
lack of information is unfortunate, but we'll have to live with it. The
format hasn't changed since 1989, it is unlikely that we'll get anyone to
change it anytime soon.

//Lennart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/ac2e5344/attachment-0001.html>

From dirkjan at ochtman.nl  Mon Oct  1 11:16:28 2012
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Mon, 1 Oct 2012 11:16:28 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
Message-ID: <CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>

On Mon, Oct 1, 2012 at 10:47 AM, Lennart Regebro <regebro at gmail.com> wrote:
> A year is no age for a Python installation. A customer of mine has one
> website developed in 2003, still running on the same server. It runs Python
> 2.3, I don't remember which version, but I'd be surprised if it is 2.3.7
> from 2008.

Right. If they don't keep their Python up-to-date, why would they
bother with their tzupdate?

My point is that there is not much of a difference in the incentive
for upgrading your timezone data whether an initial version of it came
with Python or not. Having to manually install it might make you
slightly more aware that it helps if you upgrade it once in a while,
but it seems more likely to be a fire and forget type of operation, in
which case it's basically the same as shipping a version of the
timezone data with Python (which is much easier, of course).

To put it crudely, you seem to think that most developers keep careful
track of what packages they need for their apps and actively assess
the risk for upgrading each of the packages involved. On the other
hand, I would assume more developers just get something working and
then fix any bugs that come up.

Cheers,

Dirkjan

From regebro at gmail.com  Mon Oct  1 12:11:41 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 12:11:41 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
Message-ID: <CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>

On Mon, Oct 1, 2012 at 11:16 AM, Dirkjan Ochtman <dirkjan at ochtman.nl> wrote:

> On Mon, Oct 1, 2012 at 10:47 AM, Lennart Regebro <regebro at gmail.com>
> wrote:
> > A year is no age for a Python installation. A customer of mine has one
> > website developed in 2003, still running on the same server. It runs
> Python
> > 2.3, I don't remember which version, but I'd be surprised if it is 2.3.7
> > from 2008.
>
> Right. If they don't keep their Python up-to-date, why would they
> bother with their tzupdate?
>

I'm sorry, is there a new releases of Python 2.3 made last year I don't
know about?

My point is that there is not much of a difference in the incentive
> for upgrading your timezone data whether an initial version of it came
> with Python or not.


Incentive, no. But there is a difference in awareness and likelyhood.


> Having to manually install it might make you
> slightly more aware that it helps if you upgrade it once in a while,
> but it seems more likely to be a fire and forget type of operation, in
> which case it's basically the same as shipping a version of the
> timezone data with Python (which is much easier, of course).
>

Well, me at least would include that package in the buildout or the
instructions, etc. It is therefore much more likely to be updated if it is
not included with Python.

To put it crudely, you seem to think that most developers keep careful
> track of what packages they need for their apps and actively assess
> the risk for upgrading each of the packages involved. On the other
> hand, I would assume more developers just get something working and
> then fix any bugs that come up.
>

No, I assume there are developers of both types, and in between. If
somebody just installed pytzdata and then doesn't upgrade it, fine, that's
his problem. He doesn't become *more* likely to upgrade it because it's
included in the standard library. But many developers are more likely to
keep it updated and upgrade it if it is *not* included, at least once in a
while. For example, if it's included in a buildout it could get updated
when the buildout is re-run because some of the custom modules have been
updated. While if it's included, it will never end up in the buildout and
never get updated.

//Lennart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/2ca8298e/attachment.html>

From solipsis at pitrou.net  Mon Oct  1 13:53:21 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 1 Oct 2012 13:53:21 +0200
Subject: [Python-Dev] Stdlib and timezones, again
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
Message-ID: <20121001135321.15002ee3@pitrou.net>

On Mon, 1 Oct 2012 12:11:41 +0200
Lennart Regebro <regebro at gmail.com> wrote:
> On Mon, Oct 1, 2012 at 11:16 AM, Dirkjan Ochtman <dirkjan at ochtman.nl> wrote:
> 
> > On Mon, Oct 1, 2012 at 10:47 AM, Lennart Regebro <regebro at gmail.com>
> > wrote:
> > > A year is no age for a Python installation. A customer of mine has one
> > > website developed in 2003, still running on the same server. It runs
> > Python
> > > 2.3, I don't remember which version, but I'd be surprised if it is 2.3.7
> > > from 2008.
> >
> > Right. If they don't keep their Python up-to-date, why would they
> > bother with their tzupdate?
> 
> I'm sorry, is there a new releases of Python 2.3 made last year I don't
> know about?

Python 2.3 has been EOL'ed for years. It definitely is not up-to-date,
for any reasonable definition of the term. For example, it will have
many unplugged security holes. So will that user's version of OpenSSL
and other libraries. If they don't want to apply security fixes, why
should we even care about their timezones' freshness?

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From solipsis at pitrou.net  Mon Oct  1 13:54:40 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 1 Oct 2012 13:54:40 +0200
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
In-Reply-To: <CAP1=2W7c2U50=2xmYjDmhNhn_WBHK+X-u-ayZ41zOus0FP3m-g@mail.gmail.com>
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
	<20121001022854.1e49615a@pitrou.net>
	<CAP1=2W7c2U50=2xmYjDmhNhn_WBHK+X-u-ayZ41zOus0FP3m-g@mail.gmail.com>
Message-ID: <20121001135440.7b17bf3c@pitrou.net>

On Sun, 30 Sep 2012 21:49:20 -0400
Brett Cannon <brett at python.org> wrote:
> > Note that Mako can use the Markupsafe library for faster operation.
> > This will skew the result if one of your Pythons has Markupsafe
> > installed and the other does not.
> >
> 
> Should probably have the benchmark print out a warning when markupsafe is
> used. Turns out I have it installed in my user directory for Python 2.7 so
> that probably came into play.
> 
> 
> >
> > Perhaps the benchmark runner should launch its subtests in a controlled
> > environment to avoid such issues?
> >
> 
> If we had venv in Python 2.7 that might be easy to do, but otherwise is
> there an easy way without having to try to pull in virtualenv or something
> crazy like a chroot or something?

The mako benchmark could manually exclude markupsafe from sys.modules.
That only addresses that specific benchmark, though.

Regards

Antoine.

From brett at python.org  Mon Oct  1 14:23:14 2012
From: brett at python.org (Brett Cannon)
Date: Mon, 1 Oct 2012 08:23:14 -0400
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
In-Reply-To: <CAK5idxSMrr3+WafADoRf6v0XWfrRvVgMAqxQwjwFQNShBtuwFg@mail.gmail.com>
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
	<CAK5idxSMrr3+WafADoRf6v0XWfrRvVgMAqxQwjwFQNShBtuwFg@mail.gmail.com>
Message-ID: <CAP1=2W4_61=_HEd4a1t_kxhAyqfAehXrE+-D-=avosXfVHAuLA@mail.gmail.com>

On Mon, Oct 1, 2012 at 3:44 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:

> On Mon, Oct 1, 2012 at 1:12 AM, Brett Cannon <brett at python.org> wrote:
> > I am presenting the talk "Python 3.3: Trust Me, It's Better Than 2.7" as
> > PyCon Argentina and Brasil (and US if they accept the talk). As part of
> that
> > talk I need to be able to benchmark Python 3.3 against 2.7 (both from
> tip)
> > using the unladen benchmarks (which now include benchmarks from PyPy that
> > can be relatively easily ported to Python 3).
> >
>
> Hi Brett.
>
> *If* you're talking about benchmarks, would be cool if you mention
> that pypy is actually much faster on most of them.


I will definitely mention that PyPy is actively working on Python 3 support
and people should help out where they can (whether it be technical or
financial) since PyPy will be faster than CPython in this regard and if you
needed a good chance to switch interpreters this would be it.

BTW, now that 3.3 is out is Antonio going to aim for 3.3 compatibility for
the initial release or stay back on 3.2?


> Also a very sad
> fact is that a lot of actually interesting benchmarks don't work on
> py3k (although a growing number). Twisted and sympy are very
> informative for example
>

As soon as those projects are ported we can obviously add those benchmarks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/bf6c80dc/attachment.html>

From brett at python.org  Mon Oct  1 14:23:59 2012
From: brett at python.org (Brett Cannon)
Date: Mon, 1 Oct 2012 08:23:59 -0400
Subject: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide
	build)
In-Reply-To: <20121001135440.7b17bf3c@pitrou.net>
References: <CAP1=2W7HXzUs+K1knT7Kn+4nUYuC-aoCpMb5AMXz2a_v+O6Tkw@mail.gmail.com>
	<20121001022854.1e49615a@pitrou.net>
	<CAP1=2W7c2U50=2xmYjDmhNhn_WBHK+X-u-ayZ41zOus0FP3m-g@mail.gmail.com>
	<20121001135440.7b17bf3c@pitrou.net>
Message-ID: <CAP1=2W63m5ESHmJY7fHzhRkEMjVsVRY_maK88hKYTcJt3fsZ+Q@mail.gmail.com>

On Mon, Oct 1, 2012 at 7:54 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sun, 30 Sep 2012 21:49:20 -0400
> Brett Cannon <brett at python.org> wrote:
> > > Note that Mako can use the Markupsafe library for faster operation.
> > > This will skew the result if one of your Pythons has Markupsafe
> > > installed and the other does not.
> > >
> >
> > Should probably have the benchmark print out a warning when markupsafe is
> > used. Turns out I have it installed in my user directory for Python 2.7
> so
> > that probably came into play.
> >
> >
> > >
> > > Perhaps the benchmark runner should launch its subtests in a controlled
> > > environment to avoid such issues?
> > >
> >
> > If we had venv in Python 2.7 that might be easy to do, but otherwise is
> > there an easy way without having to try to pull in virtualenv or
> something
> > crazy like a chroot or something?
>
> The mako benchmark could manually exclude markupsafe from sys.modules.
> That only addresses that specific benchmark, though)
>

Good point. Might be a good short term fix but it would be nice to have a
solution to prevent similar issues in the future.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/c3a13214/attachment.html>

From ncoghlan at gmail.com  Mon Oct  1 15:22:08 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 1 Oct 2012 18:52:08 +0530
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20121001135321.15002ee3@pitrou.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
Message-ID: <CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>

On Mon, Oct 1, 2012 at 5:23 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Python 2.3 has been EOL'ed for years. It definitely is not up-to-date,
> for any reasonable definition of the term. For example, it will have
> many unplugged security holes. So will that user's version of OpenSSL
> and other libraries. If they don't want to apply security fixes, why
> should we even care about their timezones' freshness?

Exactly. If a service provider has a bug in their application because
they're using an old timezone database, that is their problem, so long
as *we* ensure there is a clear and obvious mechanism for upgrading
*just* the TZ database, without upgrading Python itself. There's
nothing forcing people to keep their installed version of pytz up to
date, either.

If a timezone database is bundled into the standard library, there are
3 clear mechanisms for encouraging the use of fresh TZ data:

1. Consider TZ database updates to be bug fixes, and thus include them
in maintenance releases. This will keep the provided version
reasonably fresh for Python versions that are still in maintenance
mode.
2. Provide a mechanism to prefer the database from PyPI.
3. Provide a mechanism to prefer the OS database for platforms that
provide an Olson compatible interface (I briefly looked into that for
Windows a while back - it doesn't seem like a practical idea, since
Microsoft went off and did their own thing. It works for Linux and
other platforms that use the Olson database natively, though)

Since explicit is better than implicit, I *wouldn't* want to see
magical side affects where merely installing the database from PyPI,
or switching from Windows to Linux caused different behaviour.
However, it should be very easy for an application or environment to
*explicitly request* the use of the pytz database or the OS database
in preference to the bundled database.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From regebro at gmail.com  Mon Oct  1 15:31:46 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 15:31:46 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20121001135321.15002ee3@pitrou.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
Message-ID: <CAL0kPAXVAVaXPwxGdXFsLkQcEk-1niFnVH2tRHkVu_-u3f+HVg@mail.gmail.com>

On Mon, Oct 1, 2012 at 1:53 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Python 2.3 has been EOL'ed for years. It definitely is not up-to-date,
> for any reasonable definition of the term. For example, it will have
> many unplugged security holes. So will that user's version of OpenSSL
> and other libraries. If they don't want to apply security fixes, why
> should we even care about their timezones' freshness?
>

"Want to"?

//Lennart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/8d34a0ac/attachment.html>

From dirkjan at ochtman.nl  Mon Oct  1 15:33:29 2012
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Mon, 1 Oct 2012 15:33:29 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
Message-ID: <CAKmKYaCUrq1ikBmsrLfMDM3nRMz34Y8+jXn5gFSuz_LHxD7GyA@mail.gmail.com>

On Mon, Oct 1, 2012 at 3:22 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Since explicit is better than implicit, I *wouldn't* want to see
> magical side affects where merely installing the database from PyPI,
> or switching from Windows to Linux caused different behaviour.

I think that would be a pain. What you're proposing means that Linux
installations have to use the Python-installed copy by default
(because you want them to do the same thing as on Windows), even
though they have a perfectly good copy, often newer, of the database
installed on the system.

Cheers,

Dirkjan

From regebro at gmail.com  Mon Oct  1 15:34:47 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 15:34:47 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
Message-ID: <CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>

On Mon, Oct 1, 2012 at 3:22 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> If a timezone database is bundled into the standard library, there are
> 3 clear mechanisms for encouraging the use of fresh TZ data:
>
> 1. Consider TZ database updates to be bug fixes, and thus include them
> in maintenance releases. This will keep the provided version
> reasonably fresh for Python versions that are still in maintenance
> mode.
> 2. Provide a mechanism to prefer the database from PyPI.
> 3. Provide a mechanism to prefer the OS database for platforms that
> provide an Olson compatible interface (I briefly looked into that for
> Windows a while back - it doesn't seem like a practical idea, since
> Microsoft went off and did their own thing. It works for Linux and
> other platforms that use the Olson database natively, though)
>
> Since explicit is better than implicit, I *wouldn't* want to see
> magical side affects where merely installing the database from PyPI,
> or switching from Windows to Linux caused different behaviour.
> However, it should be very easy for an application or environment to
> *explicitly request* the use of the pytz database or the OS database
> in preference to the bundled database.
>

I proposed 2 and 3, and I don't really see much magical side-effects with
those.
As mentioned we can also include a database in the standardlib, but since
that will almost always be out of date, I don't really see the point.
It is of course only an issue on Windows, but still.

//Lennart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/9e910ba4/attachment-0001.html>

From zachary.ware+pydev at gmail.com  Mon Oct  1 15:57:10 2012
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Mon, 1 Oct 2012 08:57:10 -0500
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
Message-ID: <CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>

Re: to bundle or not to bundle

What about having an included database that issues a (silence-able) warning
any time it is used/imported/etc.? Have it say something to the effect of
"Warning, the Olson database included with Python is likely to be outdated,
please see <webpage> for information" where <webpage> (which may just be
the doc page for the module) spells out why it's outdated, why it's not
possible for it to be kept up to date, that this version may still work for
you depending on application and how to silence the warning, and how to get
the latest version via pypi or otherwise.

As far as preference of database, I would think the best route would be to
provide the ability to set the order you want to look in, with the default
being:
1) user specified source (usually not one of the below)
2) updated tzdb from pypi
3) OS's tzdb
4) included tzdb (with warning)

My $0.02USD, for what they're worth :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/fe3e7422/attachment.html>

From regebro at gmail.com  Mon Oct  1 16:06:18 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 16:06:18 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
Message-ID: <CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>

On Mon, Oct 1, 2012 at 3:57 PM, Zachary Ware
<zachary.ware+pydev at gmail.com>wrote:

> Re: to bundle or not to bundle
>
> What about having an included database that issues a (silence-able)
> warning any time it is used/imported/etc.? Have it say something to the
> effect of "Warning, the Olson database included with Python is likely to be
> outdated, please see <webpage> for information" where <webpage> (which may
> just be the doc page for the module) spells out why it's outdated, why it's
> not possible for it to be kept up to date, that this version may still work
> for you depending on application and how to silence the warning, and how to
> get the latest version via pypi or otherwise.
>
> As far as preference of database, I would think the best route would be to
> provide the ability to set the order you want to look in, with the default
> being:
> 1) user specified source (usually not one of the below)
> 2) updated tzdb from pypi
> 3) OS's tzdb
> 4) included tzdb (with warning)
>
> My $0.02USD, for what they're worth :)
>
Actually, that's not a bad idea. My original idea was to warn if it *was*
outdated, but since there is no way to check that, I scratched that idea.
But as I have pointed out several times, a database that is shipped with
Python is almost guaranteed to be outdated, so yeah, we could just warn
*all the time*. :-)

I like this idea. It gives an incentive to update: Get rid of the annoying
warning. It also will be silent on Unix, as we'll use the OS database, so
this will only happen on Windows or if you embed Python or similar.

//Lennart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/f76fb7d6/attachment.html>

From barry at python.org  Mon Oct  1 16:06:44 2012
From: barry at python.org (Barry Warsaw)
Date: Mon, 1 Oct 2012 10:06:44 -0400
Subject: [Python-Dev] [Python-checkins] peps: 3.2 and 3.3 release
 schedules: add information on bugfix releases and release
In-Reply-To: <CADiSq7cqREcQLvXG8pFCqBoDPbik3FdY=X+x9NW92wJx6_AV5A@mail.gmail.com>
References: <3XVjKS1J7BzQfD@mail.python.org>
	<CADiSq7cqREcQLvXG8pFCqBoDPbik3FdY=X+x9NW92wJx6_AV5A@mail.gmail.com>
Message-ID: <20121001100644.1edeb937@limelight.wooz.org>

On Oct 01, 2012, at 06:57 PM, Nick Coghlan wrote:

>On Mon, Oct 1, 2012 at 5:51 PM, georg.brandl <python-checkins at python.org> wrote:
>> +3.3.1 schedule
>> +--------------
>> +
>> +- 3.3.1 beta 1: planned for Oct/Nov 2012
>
>Copy and paste error from the 3.2 PEP?
>
>And thanks for adding these - very handy.

Agreed.  Perhaps the 2.7 RM would like to add something similar for that
release?

Cheers,
-Barry

From larry at hastings.org  Mon Oct  1 16:11:52 2012
From: larry at hastings.org (Larry Hastings)
Date: Mon, 01 Oct 2012 15:11:52 +0100
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAPTjJmr60O6Knar3CM3EgNezesHJExteppcgWOeQbRnaD7exGA@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068C4E5.9020606@ubuntu.com>
	<CAPTjJmr60O6Knar3CM3EgNezesHJExteppcgWOeQbRnaD7exGA@mail.gmail.com>
Message-ID: <5069A4A8.4020004@hastings.org>

On 10/01/2012 12:07 AM, Chris Angelico wrote:
> There's no guarantee that an individual sysadmin will have OS updates
> up-to-date.

Is there a way of asking the database its revision / date?  If so we 
could simply use the freshest data we can lay our hands on.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/c9267c75/attachment.html>

From solipsis at pitrou.net  Mon Oct  1 16:12:26 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 1 Oct 2012 16:12:26 +0200
Subject: [Python-Dev] Stdlib and timezones, again
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
Message-ID: <20121001161226.089ee544@pitrou.net>

On Mon, 1 Oct 2012 16:06:18 +0200
Lennart Regebro <regebro at gmail.com> wrote:
> Actually, that's not a bad idea. My original idea was to warn if it *was*
> outdated, but since there is no way to check that, I scratched that idea.
> But as I have pointed out several times, a database that is shipped with
> Python is almost guaranteed to be outdated, so yeah, we could just warn
> *all the time*. :-)
> 
> I like this idea. It gives an incentive to update: Get rid of the annoying
> warning.

Well, no, it is just silly. If we ship a database, that's because we
think it is good enough. A warning is just a nuisance here. We don't
display warnings when the installed Python version is too old.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From zachary.ware+pydev at gmail.com  Mon Oct  1 16:52:09 2012
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Mon, 1 Oct 2012 09:52:09 -0500
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20121001161226.089ee544@pitrou.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net>
Message-ID: <CAKJDb-PRup1TChur1C6tLi90ieBiGETO1_VYKqE_3A546iZZ-Q@mail.gmail.com>

On Mon, Oct 1, 2012 at 9:12 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 1 Oct 2012 16:06:18 +0200
> Lennart Regebro <regebro at gmail.com> wrote:
>> Actually, that's not a bad idea. My original idea was to warn if it *was*
>> outdated, but since there is no way to check that, I scratched that idea.
>> But as I have pointed out several times, a database that is shipped with
>> Python is almost guaranteed to be outdated, so yeah, we could just warn
>> *all the time*. :-)
>>
>> I like this idea. It gives an incentive to update: Get rid of the annoying
>> warning.
>
> Well, no, it is just silly. If we ship a database, that's because we
> think it is good enough. A warning is just a nuisance here. We don't
> display warnings when the installed Python version is too old.
>

My thought was that it's better to have *something* always available,
that has a decent chance of being "good enough" in a lot of cases (and
if it's good enough for you, just silence the warning), than to
noisily fail because we can't provide a perfect solution due to
political idiocy.  Or worse, to *silently* be wrong because someone
assumed we had provided a perfect solution without looking too hard. I
had no idea the Olson database was updated so often until Dirkjan
posted about there being 21 updates in 2009 alone.

For most of my uses, I would probably be a warning-silencer.  And that
wouldn't bother me; I would actually appreciate being reminded now and
then that things may have changed since the last time I did something
with timezones, and that I need to be careful of such changes.  But,
of course, that's just me, and it was my idea anyway ;)

From solipsis at pitrou.net  Mon Oct  1 17:03:25 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 1 Oct 2012 17:03:25 +0200
Subject: [Python-Dev] Stdlib and timezones, again
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net>
	<CAKJDb-PRup1TChur1C6tLi90ieBiGETO1_VYKqE_3A546iZZ-Q@mail.gmail.com>
Message-ID: <20121001170325.64a629e5@pitrou.net>

On Mon, 1 Oct 2012 09:52:09 -0500
Zachary Ware <zachary.ware+pydev at gmail.com> wrote:
> 
> My thought was that it's better to have *something* always available,
> that has a decent chance of being "good enough" in a lot of cases (and
> if it's good enough for you, just silence the warning), than to
> noisily fail because we can't provide a perfect solution due to
> political idiocy.  Or worse, to *silently* be wrong because someone
> assumed we had provided a perfect solution without looking too hard.

We can, and should, mention potential pitfalls in the documentation.

But I don't think a warning is warranted, anymore than for other known
issues (there are many of them at http://bugs.python.org/ :-)).

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From barry at python.org  Mon Oct  1 17:11:46 2012
From: barry at python.org (Barry Warsaw)
Date: Mon, 1 Oct 2012 11:11:46 -0400
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
Message-ID: <20121001111146.4970bda4@limelight.wooz.org>

On Oct 01, 2012, at 03:34 PM, Lennart Regebro wrote:

>On Mon, Oct 1, 2012 at 3:22 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> 1. Consider TZ database updates to be bug fixes, and thus include them
>> in maintenance releases. This will keep the provided version
>> reasonably fresh for Python versions that are still in maintenance
>> mode.
>> 2. Provide a mechanism to prefer the database from PyPI.
>> 3. Provide a mechanism to prefer the OS database for platforms that
>> provide an Olson compatible interface (I briefly looked into that for
>> Windows a while back - it doesn't seem like a practical idea, since
>> Microsoft went off and did their own thing. It works for Linux and
>> other platforms that use the Olson database natively, though)
>
>I proposed 2 and 3, and I don't really see much magical side-effects with
>those.  As mentioned we can also include a database in the standardlib, but
>since that will almost always be out of date, I don't really see the point.
>It is of course only an issue on Windows, but still.

I agree.  I don't think the Python RM should have to worry about tz updates,
given how frequent or unpredictable they can be.

For OSes that provide the database, I can't think of any reason not to prefer
that, except if your OS version is no longer being maintained, and then it
seems like updating your tz database is the least of your worries.

However, if someone wants to maintain a Cheeseshop package with updates, I
can't stop them.  My biggest concern there is that eventually the maintainers
will lose interest and this package will bitrot, and then we'll have obsolete
tz info out there that people will still rely on.  Oh well, I guess.

I completely agree that just installing the Cheeseshop tz package should *not*
be enough to prefer it over the system tz data.

Cheers,
-Barry

From regebro at gmail.com  Mon Oct  1 17:15:25 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 17:15:25 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20121001111146.4970bda4@limelight.wooz.org>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<20121001111146.4970bda4@limelight.wooz.org>
Message-ID: <CAL0kPAVhZAq2qHnQxgk7QReCQ-PZFY85Vwj0EvVR2iBzjbOpqg@mail.gmail.com>

On Mon, Oct 1, 2012 at 5:11 PM, Barry Warsaw <barry at python.org> wrote:

> I completely agree that just installing the Cheeseshop tz package should
> *not*
> be enough to prefer it over the system tz data.


Do you have another potential mechanism in mind?

//Lennart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/585fb1c7/attachment.html>

From zachary.ware at gmail.com  Mon Oct  1 17:16:16 2012
From: zachary.ware at gmail.com (Zachary Ware)
Date: Mon, 1 Oct 2012 10:16:16 -0500
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20121001170325.64a629e5@pitrou.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net>
	<CAKJDb-PRup1TChur1C6tLi90ieBiGETO1_VYKqE_3A546iZZ-Q@mail.gmail.com>
	<20121001170325.64a629e5@pitrou.net>
Message-ID: <CAKJDb-PCMmWKX4OWSCYge5NagZv7_iYjuX-gVDQ0TB_iPYs+KQ@mail.gmail.com>

On Oct 1, 2012 10:06 AM, "Antoine Pitrou" <solipsis at pitrou.net> wrote:
>
> On Mon, 1 Oct 2012 09:52:09 -0500
> Zachary Ware <zachary.ware+pydev at gmail.com> wrote:
> >
> > My thought was that it's better to have *something* always available,
> > that has a decent chance of being "good enough" in a lot of cases (and
> > if it's good enough for you, just silence the warning), than to
> > noisily fail because we can't provide a perfect solution due to
> > political idiocy.  Or worse, to *silently* be wrong because someone
> > assumed we had provided a perfect solution without looking too hard.
>
> We can, and should, mention potential pitfalls in the documentation.

With large red text and blink tags :-P

>
> But I don't think a warning is warranted, anymore than for other known
> issues (there are many of them at http://bugs.python.org/ :-)).
>

Just curious (and a bit off topic), what exactly does warrant a warning in
Python? I've been messing around with it for a couple years (since shortly
before 3.1, and always on 3.x) and I'm pretty sure I have yet to see a
warning for anything. Which I suppose could be counted as a good thing...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/0d977172/attachment-0001.html>

From solipsis at pitrou.net  Mon Oct  1 17:19:16 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 1 Oct 2012 17:19:16 +0200
Subject: [Python-Dev] Stdlib and timezones, again
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<20121001111146.4970bda4@limelight.wooz.org>
Message-ID: <20121001171916.0709607b@pitrou.net>

On Mon, 1 Oct 2012 11:11:46 -0400
Barry Warsaw <barry at python.org> wrote:
> 
> I agree.  I don't think the Python RM should have to worry about tz updates,
> given how frequent or unpredictable they can be.

I don't understand what makes them specifically "unpredictable".
We statically link OpenSSL and other libraries in the Windows builds. I
don't think they have very predictable updates either (especially
OpenSSL, actually).

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From barry at python.org  Mon Oct  1 17:25:08 2012
From: barry at python.org (Barry Warsaw)
Date: Mon, 1 Oct 2012 11:25:08 -0400
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
Message-ID: <20121001112508.396a6685@limelight.wooz.org>

On Sep 30, 2012, at 02:47 PM, Lennart Regebro wrote:

>The problem with including pytz in the stdlib is that it contains the
>tz/zoneinfo/Olson database, and it updates much more often than Python
>does.

Why include the database in Python at all?

If you have an OS that keeps the system tz data up-to-date, I can't think of
any reason why you wouldn't want to use it.

If you don't have the data, why not include information in the documentation
for how to download and install the database in a location that Python will
search for, along with information on how to enable that?  You could even
provide a nice script that would download, install, and optionally enable that
tz data's use.

I think that would cover all the bases:

* My OS keeps the tz data up-to-date.  I don't have to do nuthin', and Python
  gives me a nice API for using all the world's timezones on my superb OS.

* My OS keeps the tz data up-to-date, but I'm skeptical.  I run the update
  script whenever is appropriate, adding the --enable flag, and the tz data is
  grabbed from the intarwebs, installed, and Python starts using it instead of
  the system data.

* I am sad because I use an OS that has no tz data.  I run the update script
  once in a while, adding the --enable flag, and my Python is timezonally
  happy.

* I'm sad and lazy.  Oh well, Python throws an exception when I try to use a
  timezone that isn't UTC.

Cheers,
-Barry

From regebro at gmail.com  Mon Oct  1 17:29:11 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 17:29:11 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20121001112508.396a6685@limelight.wooz.org>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20121001112508.396a6685@limelight.wooz.org>
Message-ID: <CAL0kPAXUoZfjfqUVF4G1G+VrRLpuLkokGJiD3Ktm9QQJwXOxsA@mail.gmail.com>

On Mon, Oct 1, 2012 at 5:25 PM, Barry Warsaw <barry at python.org> wrote:

> On Sep 30, 2012, at 02:47 PM, Lennart Regebro wrote:
>
> >The problem with including pytz in the stdlib is that it contains the
> >tz/zoneinfo/Olson database, and it updates much more often than Python
> >does.
>
> Why include the database in Python at all?
>

Exactly my point.


> If you have an OS that keeps the system tz data up-to-date, I can't think
> of
> any reason why you wouldn't want to use it.
>
> If you don't have the data, why not include information in the
> documentation
> for how to download and install the database in a location that Python will
> search for, along with information on how to enable that?  You could even
> provide a nice script that would download, install, and optionally enable
> that
> tz data's use.
>

Right, I was going to do that by bundling the files in a package,
tentatively called pytzdata, available on PyPI, so it can be
easy_installed/pip installed, etc.


> I think that would cover all the bases:
>
> * My OS keeps the tz data up-to-date.  I don't have to do nuthin', and
> Python
>   gives me a nice API for using all the world's timezones on my superb OS.
>
> * My OS keeps the tz data up-to-date, but I'm skeptical.  I run the update
>   script whenever is appropriate, adding the --enable flag, and the tz
> data is
>   grabbed from the intarwebs, installed, and Python starts using it
> instead of
>   the system data.
>
> * I am sad because I use an OS that has no tz data.  I run the update
> script
>   once in a while, adding the --enable flag, and my Python is timezonally
>   happy.
>
> * I'm sad and lazy.  Oh well, Python throws an exception when I try to use
> a
>   timezone that isn't UTC.
>

We seem to be on the same page here.

//Lennart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/5bd7516b/attachment.html>

From solipsis at pitrou.net  Mon Oct  1 17:28:06 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 1 Oct 2012 17:28:06 +0200
Subject: [Python-Dev] Stdlib and timezones, again
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net>
	<CAKJDb-PRup1TChur1C6tLi90ieBiGETO1_VYKqE_3A546iZZ-Q@mail.gmail.com>
	<20121001170325.64a629e5@pitrou.net>
	<CAKJDb-PCMmWKX4OWSCYge5NagZv7_iYjuX-gVDQ0TB_iPYs+KQ@mail.gmail.com>
Message-ID: <20121001172806.24828d22@pitrou.net>

On Mon, 1 Oct 2012 10:16:16 -0500
Zachary Ware <zachary.ware at gmail.com> wrote:
> >
> > But I don't think a warning is warranted, anymore than for other known
> > issues (there are many of them at http://bugs.python.org/ :-)).
> >
> 
> Just curious (and a bit off topic), what exactly does warrant a warning in
> Python? I've been messing around with it for a couple years (since shortly
> before 3.1, and always on 3.x) and I'm pretty sure I have yet to see a
> warning for anything. Which I suppose could be counted as a good thing...

We don't really have a well-defined policy, except for one warning
category: DeprecationWarnings, whose purpose are quite clear. But they
are now silenced by default :-)

Generally, warnings are not easy to deal with for the end-user, so we
are reluctant to add any. We'd rather document annoyances and pitfalls;
and, moreover, we try to design our APIs so that they don't have
unexpected effects.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From barry at python.org  Mon Oct  1 17:29:55 2012
From: barry at python.org (Barry Warsaw)
Date: Mon, 1 Oct 2012 11:29:55 -0400
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAL0kPAVhZAq2qHnQxgk7QReCQ-PZFY85Vwj0EvVR2iBzjbOpqg@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<20121001111146.4970bda4@limelight.wooz.org>
	<CAL0kPAVhZAq2qHnQxgk7QReCQ-PZFY85Vwj0EvVR2iBzjbOpqg@mail.gmail.com>
Message-ID: <20121001112955.74f71902@limelight.wooz.org>

On Oct 01, 2012, at 05:15 PM, Lennart Regebro wrote:

>On Mon, Oct 1, 2012 at 5:11 PM, Barry Warsaw <barry at python.org> wrote:
>
>> I completely agree that just installing the Cheeseshop tz package should
>> *not* be enough to prefer it over the system tz data.
>
>Do you have another potential mechanism in mind?

Yes.

Using the script I mentioned in an different response, if someone installed
the database to some location (TBD), then there would probably be a config
file sitting next to it.  A simple .ini file with an 'enable' flag would be
needed to turn on the override.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/cfea0f2d/attachment.pgp>

From tjreedy at udel.edu  Mon Oct  1 17:32:03 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 01 Oct 2012 11:32:03 -0400
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
Message-ID: <k4cd1m$1on$1@ger.gmane.org>

On 10/1/2012 10:06 AM, Lennart Regebro wrote:

> Actually, that's not a bad idea. My original idea was to warn if it
> *was* outdated, but since there is no way to check that, I scratched
> that idea.

Is there really no way to get a 'last updated' time from the site where 
the database is kept? If not, perhaps we should provide one with a daily 
job (on pypi?) that downloads and compares.

-- 
Terry Jan Reedy


From regebro at gmail.com  Mon Oct  1 17:33:24 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 17:33:24 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20121001112955.74f71902@limelight.wooz.org>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<20121001111146.4970bda4@limelight.wooz.org>
	<CAL0kPAVhZAq2qHnQxgk7QReCQ-PZFY85Vwj0EvVR2iBzjbOpqg@mail.gmail.com>
	<20121001112955.74f71902@limelight.wooz.org>
Message-ID: <CAL0kPAVOKOfrzwKZUU-A9nWp7t8og5oZ9RJEHiPO50X7U79qNA@mail.gmail.com>

On Mon, Oct 1, 2012 at 5:29 PM, Barry Warsaw <barry at python.org> wrote:

> On Oct 01, 2012, at 05:15 PM, Lennart Regebro wrote:
>
> >On Mon, Oct 1, 2012 at 5:11 PM, Barry Warsaw <barry at python.org> wrote:
> >
> >> I completely agree that just installing the Cheeseshop tz package should
> >> *not* be enough to prefer it over the system tz data.
> >
> >Do you have another potential mechanism in mind?
>
> Yes.
>
> Using the script I mentioned in an different response, if someone installed
> the database to some location (TBD), then there would probably be a config
> file sitting next to it.  A simple .ini file with an 'enable' flag would be
> needed to turn on the override.
>

OK, I'll have that in mind.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/ab755ea6/attachment.html>

From barry at python.org  Mon Oct  1 17:38:01 2012
From: barry at python.org (Barry Warsaw)
Date: Mon, 1 Oct 2012 11:38:01 -0400
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAL0kPAXUoZfjfqUVF4G1G+VrRLpuLkokGJiD3Ktm9QQJwXOxsA@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20121001112508.396a6685@limelight.wooz.org>
	<CAL0kPAXUoZfjfqUVF4G1G+VrRLpuLkokGJiD3Ktm9QQJwXOxsA@mail.gmail.com>
Message-ID: <20121001113801.24debe64@limelight.wooz.org>

On Oct 01, 2012, at 05:29 PM, Lennart Regebro wrote:

>We seem to be on the same page here.

Well, that was easy!  Maybe I should be your PEP Czar :)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/a26a1094/attachment.pgp>

From solipsis at pitrou.net  Mon Oct  1 17:41:01 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 1 Oct 2012 17:41:01 +0200
Subject: [Python-Dev] Stdlib and timezones, again
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20121001112508.396a6685@limelight.wooz.org>
Message-ID: <20121001174101.728d81b2@pitrou.net>

On Mon, 1 Oct 2012 11:25:08 -0400
Barry Warsaw <barry at python.org> wrote:
> 
> If you have an OS that keeps the system tz data up-to-date, I can't think of
> any reason why you wouldn't want to use it.
> 
> If you don't have the data, why not include information in the documentation
> for how to download and install the database in a location that Python will
> search for, along with information on how to enable that?

Well, for one, that doesn't really fit in the "batteries included"
approach. Having to download separate data files is an annoyance.

> You could even
> provide a nice script that would download, install, and optionally enable that
> tz data's use.

This doesn't solve the update problem at all, since that tz data would
still be outdated after a couple of months. In other words, it has no
benefit for the user, but is more complicated to use.

Besides, that script would have to care about security issues. For
example you'd use a HTTPS-enabled source, and then bundle the
required CA certificates with Python. But wait, now Python has to ship
some data that will possibly become outdated in a year or two :-)

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From python-dev at masklinn.net  Mon Oct  1 18:00:40 2012
From: python-dev at masklinn.net (Xavier Morel)
Date: Mon, 1 Oct 2012 18:00:40 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <k4cd1m$1on$1@ger.gmane.org>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail
	.com> <k4cd1m$1on$1@ger.gmane.org>
Message-ID: <EF3A5E98-9519-46E4-BC87-3DDD6E6C4F7F@masklinn.net>

On 2012-10-01, at 17:32 , Terry Reedy wrote:

> On 10/1/2012 10:06 AM, Lennart Regebro wrote:
> 
>> Actually, that's not a bad idea. My original idea was to warn if it
>> *was* outdated, but since there is no way to check that, I scratched
>> that idea.
> 
> Is there really no way to get a 'last updated' time from the site where the database is kept? If not, perhaps we should provide one with a daily job (on pypi?) that downloads and compares.

There are several: there's a message on a dedicated mailing list and
there are HTTP, FTP and rsync repositories with both all releases and a
"latest" archive for both tzdata and tzcode. The HTTP repositories seems
to handle time-based conditional requests correctly (an
If-Modified-Since with the date of the latest release or later will
result in a 304 response, earlier will result in a 200)

The HTTP URIs are https://www.iana.org/time-zones/repository/tzdata-latest.tar.gz
and https://www.iana.org/time-zones/repository/tzcode-latest.tar.gz

For some reason, IANA does not seem to publish a feed.

From regebro at gmail.com  Mon Oct  1 18:01:43 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 18:01:43 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20121001113801.24debe64@limelight.wooz.org>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20121001112508.396a6685@limelight.wooz.org>
	<CAL0kPAXUoZfjfqUVF4G1G+VrRLpuLkokGJiD3Ktm9QQJwXOxsA@mail.gmail.com>
	<20121001113801.24debe64@limelight.wooz.org>
Message-ID: <CAL0kPAVfaBG0cfrh84KOATNAUvXB-WP+cr4wzGvXm3uhd5ismA@mail.gmail.com>

On Mon, Oct 1, 2012 at 5:38 PM, Barry Warsaw <barry at python.org> wrote:

> On Oct 01, 2012, at 05:29 PM, Lennart Regebro wrote:
>
> >We seem to be on the same page here.
>
> Well, that was easy!  Maybe I should be your PEP Czar :)
>

That sounds great!

What's a PEP Czar?

//My First PEP
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/beff1fec/attachment-0001.html>

From regebro at gmail.com  Mon Oct  1 18:04:01 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 18:04:01 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <k4cd1m$1on$1@ger.gmane.org>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<k4cd1m$1on$1@ger.gmane.org>
Message-ID: <CAL0kPAUvoMoUGoMaz9qHp3G4HhAZPsPmNj0SSQbWgJ0LjJ2+AQ@mail.gmail.com>

On Mon, Oct 1, 2012 at 5:32 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 10/1/2012 10:06 AM, Lennart Regebro wrote:
>
>  Actually, that's not a bad idea. My original idea was to warn if it
>> *was* outdated, but since there is no way to check that, I scratched
>> that idea.
>>
>
> Is there really no way to get a 'last updated' time from the site where
> the database is kept?


Sure, but there is no way to get a last updated time from the database
that's installed. And making an HTTP request when you call localize() to
check if the database is outdated or not doesn't seem to be a great idea.

//Lennart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/610a292f/attachment.html>

From larry at hastings.org  Mon Oct  1 18:21:26 2012
From: larry at hastings.org (Larry Hastings)
Date: Mon, 01 Oct 2012 17:21:26 +0100
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20121001112955.74f71902@limelight.wooz.org>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<20121001111146.4970bda4@limelight.wooz.org>
	<CAL0kPAVhZAq2qHnQxgk7QReCQ-PZFY85Vwj0EvVR2iBzjbOpqg@mail.gmail.com>
	<20121001112955.74f71902@limelight.wooz.org>
Message-ID: <5069C306.80804@hastings.org>

On 10/01/2012 04:29 PM, Barry Warsaw wrote:
> Using the script I mentioned in an different response, if someone installed
> the database to some location (TBD), then there would probably be a config
> file sitting next to it.  A simple .ini file with an 'enable' flag would be
> needed to turn on the override.

Perhaps said config file could also contain the timestamp of the tz 
database?  Then we could intelligently choose the most currentest one.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/6860dc69/attachment.html>

From regebro at gmail.com  Mon Oct  1 18:39:03 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 18:39:03 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <5069C306.80804@hastings.org>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<20121001111146.4970bda4@limelight.wooz.org>
	<CAL0kPAVhZAq2qHnQxgk7QReCQ-PZFY85Vwj0EvVR2iBzjbOpqg@mail.gmail.com>
	<20121001112955.74f71902@limelight.wooz.org>
	<5069C306.80804@hastings.org>
Message-ID: <CAL0kPAUWJK9Rd9sZ92yeUrVX1xeMXTpjc-szMtYpiz-e_mdT-g@mail.gmail.com>

On Mon, Oct 1, 2012 at 6:21 PM, Larry Hastings <larry at hastings.org> wrote:

>  On 10/01/2012 04:29 PM, Barry Warsaw wrote:
>
> Using the script I mentioned in an different response, if someone installed
> the database to some location (TBD), then there would probably be a config
> file sitting next to it.  A simple .ini file with an 'enable' flag would be
> needed to turn on the override.
>
>
> Perhaps said config file could also contain the timestamp of the tz
> database?  Then we could intelligently choose the most currentest one.
>

The most current one is likely to be the one provided by the operating
system, which does not contain any .ini file, nor, as far as I can tell,
any information about the database version or any timestamps.

//Lennart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/eb2aab87/attachment.html>

From g.brandl at gmx.net  Mon Oct  1 19:40:40 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 01 Oct 2012 19:40:40 +0200
Subject: [Python-Dev] peps: describe the current policy
In-Reply-To: <3XVrM00RfrzQRk@mail.python.org>
References: <3XVrM00RfrzQRk@mail.python.org>
Message-ID: <k4ckin$7u5$1@ger.gmane.org>

On 10/01/2012 07:38 PM, benjamin.peterson wrote:
> http://hg.python.org/peps/rev/641add9f9542
> changeset:   4530:641add9f9542
> user:        Benjamin Peterson <benjamin at python.org>
> date:        Mon Oct 01 13:38:31 2012 -0400
> summary:
>    describe the current policy
>
> files:
>    pep-0373.txt |  8 ++++++++
>    1 files changed, 8 insertions(+), 0 deletions(-)
>
>
> diff --git a/pep-0373.txt b/pep-0373.txt
> --- a/pep-0373.txt
> +++ b/pep-0373.txt
> @@ -49,6 +49,14 @@
>   Maintenance releases
>   ====================
>
> +Being the last of the 2.x series, 2.7 will have an extended period of
> +maintenance. The current plan is to support it for at least 5 years
> +from the initial 2.7 release. This means there will be bugfix releases
> +until 2015. Depending on Python 3 adoption, this may be modified up or
> +down.

TBH, I would guarantee it for at least 5 years: the current wording is not
an improvement over the current situation as it basically does not tell
anything definite.

Georg


From tjreedy at udel.edu  Mon Oct  1 19:54:54 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 01 Oct 2012 13:54:54 -0400
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAL0kPAUWJK9Rd9sZ92yeUrVX1xeMXTpjc-szMtYpiz-e_mdT-g@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<20121001111146.4970bda4@limelight.wooz.org>
	<CAL0kPAVhZAq2qHnQxgk7QReCQ-PZFY85Vwj0EvVR2iBzjbOpqg@mail.gmail.com>
	<20121001112955.74f71902@limelight.wooz.org>
	<5069C306.80804@hastings.org>
	<CAL0kPAUWJK9Rd9sZ92yeUrVX1xeMXTpjc-szMtYpiz-e_mdT-g@mail.gmail.com>
Message-ID: <k4cldj$hu5$1@ger.gmane.org>

On 10/1/2012 12:39 PM, Lennart Regebro wrote:
> On Mon, Oct 1, 2012 at 6:21 PM, Larry Hastings <larry at hastings.org
> <mailto:larry at hastings.org>> wrote:
>
>     On 10/01/2012 04:29 PM, Barry Warsaw wrote:
>>     Using the script I mentioned in an different response, if someone installed
>>     the database to some location (TBD), then there would probably be a config
>>     file sitting next to it.  A simple .ini file with an 'enable' flag would be
>>     needed to turn on the override.
>
>     Perhaps said config file could also contain the timestamp of the tz
>     database?  Then we could intelligently choose the most currentest one.
>
>
> The most current one is likely to be the one provided by the operating
> system, which does not contain any .ini file, nor, as far as I can tell,
> any information about the database version or any timestamps.

But Windows does not provide one, or at least, the proposal seems to be 
not use whatever it does have. I think your PEP should propose one api 
but conditional tz db access code for systems with and without the tz db 
already provided.

-- 
Terry Jan Reedy


From tseaver at palladion.com  Mon Oct  1 20:23:30 2012
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 01 Oct 2012 14:23:30 -0400
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20121001161226.089ee544@pitrou.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net>
Message-ID: <k4cn2k$2vs$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/01/2012 10:12 AM, Antoine Pitrou wrote:
> On Mon, 1 Oct 2012 16:06:18 +0200 Lennart Regebro <regebro at gmail.com>
> wrote:
>> Actually, that's not a bad idea. My original idea was to warn if it
>> *was* outdated, but since there is no way to check that, I scratched
>> that idea. But as I have pointed out several times, a database that
>> is shipped with Python is almost guaranteed to be outdated, so yeah,
>> we could just warn *all the time*. :-)
>> 
>> I like this idea. It gives an incentive to update: Get rid of the
>> annoying warning.
> 
> Well, no, it is just silly. If we ship a database, that's because we 
> think it is good enough. A warning is just a nuisance here. We don't 
> display warnings when the installed Python version is too old.

Using a bundled version would be the Wrong Thing (TM) for almost any
non-toy application (one where non-programmers actually care about the
timezones in which dates / times are entered / displayed):  the fact that
the errors would infrequent / subtle / hard to diagnose is an argument
*against* inclusion.

- -1 for including a copy in Python at all:  if your app needs it, and the
OS on which you deploy doesn't provide it, you have a straightforward way
of installing it.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEUEARECAAYFAlBp36IACgkQ+gerLs4ltQ4mbACY60cQ1SbUgRh0hNtUmxScRM68
ZwCg125QzeMmhFV0VYYZGmNVmEqpajo=
=ehg8
-----END PGP SIGNATURE-----


From ncoghlan at gmail.com  Mon Oct  1 20:48:10 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 2 Oct 2012 00:18:10 +0530
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <k4cn2k$2vs$1@ger.gmane.org>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net> <k4cn2k$2vs$1@ger.gmane.org>
Message-ID: <CADiSq7cdv=mUe_wfrtWop77bszeGzrB+6_FU78qjz=h7RoWMjA@mail.gmail.com>

Reminder to everyone: the current state of the art for getting up to
date tz info for Python is "pip install pytz".

If any proposal is more complicated than that, there's absolutely no
point in changing anything. The version I liked best so far is for
Python to just install a copy of pytz automatically (shipping it in
the installer rather than downloading it). OS packagers would then
take it out (replacing it with a dependency on a pytz emulator that
used the system database instead).

Cheers,
Nick.

From rdmurray at bitdance.com  Mon Oct  1 21:02:09 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 01 Oct 2012 15:02:09 -0400
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CADiSq7cdv=mUe_wfrtWop77bszeGzrB+6_FU78qjz=h7RoWMjA@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net> <k 4cn2k$2vs$1@ger.gmane.org>
	<CADiSq7cdv=mUe_wfrtWop77bszeGzrB+6_FU78qjz=h7RoWMjA@mail.gmail.com>
Message-ID: <20121001190209.D89622500FA@webabinitio.net>

On Tue, 02 Oct 2012 00:18:10 +0530, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Reminder to everyone: the current state of the art for getting up to
> date tz info for Python is "pip install pytz".
> 
> If any proposal is more complicated than that, there's absolutely no
> point in changing anything. The version I liked best so far is for
> Python to just install a copy of pytz automatically (shipping it in
> the installer rather than downloading it). OS packagers would then
> take it out (replacing it with a dependency on a pytz emulator that
> used the system database instead).

Emulator?  That makes no sense, I'm afraid.

I think we are talking here about incorporating pytz into the
stdlib.  The only question is how to manage the Olson database
on Windows, which has *always* been the question.

--David

From barry at python.org  Mon Oct  1 21:02:30 2012
From: barry at python.org (Barry Warsaw)
Date: Mon, 1 Oct 2012 15:02:30 -0400
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CADiSq7cdv=mUe_wfrtWop77bszeGzrB+6_FU78qjz=h7RoWMjA@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net> <k4cn2k$2vs$1@ger.gmane.org>
	<CADiSq7cdv=mUe_wfrtWop77bszeGzrB+6_FU78qjz=h7RoWMjA@mail.gmail.com>
Message-ID: <20121001150230.0dbc6e99@resist.wooz.org>

On Oct 02, 2012, at 12:18 AM, Nick Coghlan wrote:

>Reminder to everyone: the current state of the art for getting up to
>date tz info for Python is "pip install pytz".
>
>If any proposal is more complicated than that, there's absolutely no
>point in changing anything. The version I liked best so far is for
>Python to just install a copy of pytz automatically (shipping it in
>the installer rather than downloading it). OS packagers would then
>take it out (replacing it with a dependency on a pytz emulator that
>used the system database instead).

Why wouldn't the stdlib just ship with that emulator by default then?  If your
OS doesn't have a system database, then you `pip install pytz` ftw.

Cheers,
-Barry

From regebro at gmail.com  Mon Oct  1 21:02:52 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 21:02:52 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CADiSq7cdv=mUe_wfrtWop77bszeGzrB+6_FU78qjz=h7RoWMjA@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net> <k4cn2k$2vs$1@ger.gmane.org>
	<CADiSq7cdv=mUe_wfrtWop77bszeGzrB+6_FU78qjz=h7RoWMjA@mail.gmail.com>
Message-ID: <CAL0kPAWg1jqLY668WxQJv2__LQZ_Ou86+ePRueC_hyDKYJHQiQ@mail.gmail.com>

On Mon, Oct 1, 2012 at 8:48 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> If any proposal is more complicated than that, there's absolutely no
> point in changing anything. The version I liked best so far is for
> Python to just install a copy of pytz automatically (shipping it in
> the installer rather than downloading it). OS packagers would then
> take it out (replacing it with a dependency on a pytz emulator that
> used the system database instead).

But that would then mean that when you install from source, as I
typically do to avoid depending on Python, it would use another
database than the OS version. I don't want that, i want Python on Unix
to use the OS supplied database, unless another one is explicitly
installed/configured.

//Lennart

From tseaver at palladion.com  Mon Oct  1 21:03:05 2012
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 01 Oct 2012 15:03:05 -0400
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CADiSq7cdv=mUe_wfrtWop77bszeGzrB+6_FU78qjz=h7RoWMjA@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net> <k4cn2k$2vs$1@ger.gmane.org>
	<CADiSq7cdv=mUe_wfrtWop77bszeGzrB+6_FU78qjz=h7RoWMjA@mail.gmail.com>
Message-ID: <k4cpcp$o8b$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/01/2012 02:48 PM, Nick Coghlan wrote:

> Reminder to everyone: the current state of the art for getting up to 
> date tz info for Python is "pip install pytz".
> 
> If any proposal is more complicated than that, there's absolutely no 
> point in changing anything. The version I liked best so far is for 
> Python to just install a copy of pytz automatically (shipping it in 
> the installer rather than downloading it). OS packagers would then 
> take it out (replacing it with a dependency on a pytz emulator that 
> used the system database instead).

Lennart's original proposal was to land the machinery from the current
pytz package into the stdlib, leaving the data part as a
separately-installable package.  Lennart also proposed some kind of
(maybe configurable) policy for looking up the Olson-based timezonedb.

I'm fine with Lennart's proposal, but want to argue against including any
copy of the Olson db in Python itself:  I don't believe the set of folks
who need it but can't do the equivalent of 'pip / easy_install
<lennarts-new-name-for-the-data-part>' is big enough to outweigh the
downside of bundling immediately-stale data, and hoping that everybody
else remembers to configure it out.



Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlBp6OkACgkQ+gerLs4ltQ4fZwCfUtz9lnkjxM6rdTaO0io1yzIP
f2cAn0YfQt+n09Znuh6BZ/Culixpf7eE
=ecDk
-----END PGP SIGNATURE-----


From rdmurray at bitdance.com  Mon Oct  1 21:04:59 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 01 Oct 2012 15:04:59 -0400
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CADiSq7cdv=mUe_wfrtWop77bszeGzrB+6_FU78qjz=h7RoWMjA@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net> <k 4cn2k$2vs$1@ger.gmane.org>
	<CADiSq7cdv=mUe_wfrtWop77bszeGzrB+6_FU78qjz=h7RoWMjA@mail.gmail.com>
Message-ID: <20121001190459.E112D2500FA@webabinitio.net>

Gah, ignore half of that last post.  I think Lennart is talking
about incorporating the missing pytz *functionality* into
the stdlib.

--David

From guido at python.org  Mon Oct  1 21:09:16 2012
From: guido at python.org (Guido van Rossum)
Date: Mon, 1 Oct 2012 12:09:16 -0700
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20121001150230.0dbc6e99@resist.wooz.org>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net> <k4cn2k$2vs$1@ger.gmane.org>
	<CADiSq7cdv=mUe_wfrtWop77bszeGzrB+6_FU78qjz=h7RoWMjA@mail.gmail.com>
	<20121001150230.0dbc6e99@resist.wooz.org>
Message-ID: <CAP7+vJKY7yd-3XShAbMH2humFHD3n6ygKP5025Rqx1k30hE6rQ@mail.gmail.com>

On Mon, Oct 1, 2012 at 12:02 PM, Barry Warsaw <barry at python.org> wrote:
> On Oct 02, 2012, at 12:18 AM, Nick Coghlan wrote:
>
>>Reminder to everyone: the current state of the art for getting up to
>>date tz info for Python is "pip install pytz".
>>
>>If any proposal is more complicated than that, there's absolutely no
>>point in changing anything. The version I liked best so far is for
>>Python to just install a copy of pytz automatically (shipping it in
>>the installer rather than downloading it). OS packagers would then
>>take it out (replacing it with a dependency on a pytz emulator that
>>used the system database instead).
>
> Why wouldn't the stdlib just ship with that emulator by default then?  If your
> OS doesn't have a system database, then you `pip install pytz` ftw.

I think those are all suggestions included in Nick's intention. The
main points are that (a) pytz and some not-too-ancient Olson database
are included by default, (b) vendor distributions can tweak the
packaging so that the platform's copy of the Olson database is used
automatically; (c) users not benefitting from such vendor support can
update the database as easily as "pip install pytz" (though not
necessarily with that specific command).

-- 
--Guido van Rossum (python.org/~guido)

From solipsis at pitrou.net  Mon Oct  1 21:18:12 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 1 Oct 2012 21:18:12 +0200
Subject: [Python-Dev] Daily DMG builds
Message-ID: <20121001211812.4c40aa52@pitrou.net>


Hello,

It seems that the daily DMG builds have been failing for some time on
the default branch:
http://buildbot.python.org/daily-dmg/

Since there has been no report or complaint about that, should we just
stop producing those builds?

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From regebro at gmail.com  Mon Oct  1 21:59:03 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 1 Oct 2012 21:59:03 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20121001190209.D89622500FA@webabinitio.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net>
	<CADiSq7cdv=mUe_wfrtWop77bszeGzrB+6_FU78qjz=h7RoWMjA@mail.gmail.com>
	<20121001190209.D89622500FA@webabinitio.net>
Message-ID: <CAL0kPAVfW3GwBpr6-Sg5PE0-uzZQ_XcMH_gvf2AphQzB=cioJQ@mail.gmail.com>

On Mon, Oct 1, 2012 at 9:02 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Tue, 02 Oct 2012 00:18:10 +0530, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Reminder to everyone: the current state of the art for getting up to
>> date tz info for Python is "pip install pytz".
>>
>> If any proposal is more complicated than that, there's absolutely no
>> point in changing anything. The version I liked best so far is for
>> Python to just install a copy of pytz automatically (shipping it in
>> the installer rather than downloading it). OS packagers would then
>> take it out (replacing it with a dependency on a pytz emulator that
>> used the system database instead).
>
> Emulator?  That makes no sense, I'm afraid.

It wouldn't emulate anything, though. Ubuntu already does this, they
ship a version of pytz that includes no database, and just point it to
/usr/share/zoneinfo instead.

//Lennart

From nad at acm.org  Mon Oct  1 22:20:13 2012
From: nad at acm.org (Ned Deily)
Date: Mon, 01 Oct 2012 13:20:13 -0700
Subject: [Python-Dev] Daily DMG builds
References: <20121001211812.4c40aa52@pitrou.net>
Message-ID: <nad-FF6FDF.13201301102012@news.gmane.org>

In article <20121001211812.4c40aa52 at pitrou.net>,
 Antoine Pitrou <solipsis at pitrou.net> wrote:

> Hello,
> 
> It seems that the daily DMG builds have been failing for some time on
> the default branch:
> http://buildbot.python.org/daily-dmg/
> 
> Since there has been no report or complaint about that, should we just
> stop producing those builds?

It's failing on a checksum error of the cached download ncurses tarball 
which is kind of odd.  I've pinged David about it.

I must admit I don't pay attention to the dmg buildbot but I probably 
should.  I know at least one person (Raymond) has reported using the dmg 
builds regularly and the 2.7 builds are still going.  There is a bit of 
an issue now for 3.3 and default: the buildbot-slave is apparently 
running OS X 10.4 so it is no longer capable of building either of the 
two installer variants that we release on python.org (one now requires 
at least 10.5 and the other at least 10.6).  However, we don't want to 
break anything on 10.4, if possible, and the installers still work on 
later systems so there is some value in producing them - as long as 
someone is paying attention when they break and, hopefully, someone uses 
them.

-- 
 Ned Deily,
 nad at acm.org


From ethan at stoneleaf.us  Mon Oct  1 22:17:21 2012
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 01 Oct 2012 13:17:21 -0700
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <5068C4E5.9020606@ubuntu.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>	<20120930150345.4ad47c57@pitrou.net>	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>	<20120930151533.38fa85fd@pitrou.net>	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068C4E5.9020606@ubuntu.com>
Message-ID: <5069FA51.3010806@stoneleaf.us>

Matthias Klose wrote:
> On 30.09.2012 20:18, Gregory P. Smith wrote:
>> priority:
>>   1) api call supplying tz data to the process.
>>   2) pytzdata module if it exists
>>   3) tz data from the underlying operating system
>>   4) error.
> 
> I disagree on this order, at least for Linux systems. the tzdata database is
> well managed on major Linux distributions and should be used for this reason.

The order exists this way to allow for customization.  If I have 1) 
downloaded the most recent, or 2) made customizations (for whatever 
reason) then Python needs to use it.

Can I break stuff this way?  Sure.  Is it Python's responsibility to 
stop me?  Nope.  Consenting adults, and all that.

~Ethan~

From rdmurray at bitdance.com  Mon Oct  1 22:39:38 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 01 Oct 2012 16:39:38 -0400
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAL0kPAVfW3GwBpr6-Sg5PE0-uzZQ_XcMH_gvf2AphQzB=cioJQ@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net> <C
	ADiSq7cdv=mUe_wfrtWop77bszeGzrB+6_FU78qjz=h7RoWMjA@mail.gmail.com>
	<20121001190209.D89622500FA@webabinitio.net>
	<CAL0kPAVfW3GwBpr6-Sg5PE0-uzZQ_XcMH_gvf2AphQzB=cioJQ@mail.gmail.com>
Message-ID: <20121001203939.7313A250030@webabinitio.net>

On Mon, 01 Oct 2012 21:59:03 +0200, Lennart Regebro <regebro at gmail.com> wrote:
> On Mon, Oct 1, 2012 at 9:02 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> > On Tue, 02 Oct 2012 00:18:10 +0530, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >> Reminder to everyone: the current state of the art for getting up to
> >> date tz info for Python is "pip install pytz".
> >>
> >> If any proposal is more complicated than that, there's absolutely no
> >> point in changing anything. The version I liked best so far is for
> >> Python to just install a copy of pytz automatically (shipping it in
> >> the installer rather than downloading it). OS packagers would then
> >> take it out (replacing it with a dependency on a pytz emulator that
> >> used the system database instead).
> >
> > Emulator?  That makes no sense, I'm afraid.
> 
> It wouldn't emulate anything, though. Ubuntu already does this, they
> ship a version of pytz that includes no database, and just point it to
> /usr/share/zoneinfo instead.

That's what I meant.  It's not an emulator.

--David

From roundup-admin at psf.upfronthosting.co.za  Mon Oct  1 23:11:54 2012
From: roundup-admin at psf.upfronthosting.co.za (Python tracker)
Date: Mon, 01 Oct 2012 21:11:54 +0000
Subject: [Python-Dev] Failed issue tracker submission
Message-ID: <20121001211154.5987C1C892@psf.upfronthosting.co.za>



The node specified by the designator in the subject of your message
("16304") does not exist.

Subject was: "[issue16304]"



Mail Gateway Help
=================
Incoming messages are examined for multiple parts:
 . In a multipart/mixed message or part, each subpart is extracted and
   examined. The text/plain subparts are assembled to form the textual
   body of the message, to be stored in the file associated with a "msg"
   class node. Any parts of other types are each stored in separate files
   and given "file" class nodes that are linked to the "msg" node.
 . In a multipart/alternative message or part, we look for a text/plain
   subpart and ignore the other parts.
 . A message/rfc822 is treated similar tomultipart/mixed (except for
   special handling of the first text part) if unpack_rfc822 is set in
   the mailgw config section.

Summary
-------
The "summary" property on message nodes is taken from the first non-quoting
section in the message body. The message body is divided into sections by
blank lines. Sections where the second and all subsequent lines begin with
a ">" or "|" character are considered "quoting sections". The first line of
the first non-quoting section becomes the summary of the message.

Addresses
---------
All of the addresses in the To: and Cc: headers of the incoming message are
looked up among the user nodes, and the corresponding users are placed in
the "recipients" property on the new "msg" node. The address in the From:
header similarly determines the "author" property of the new "msg"
node. The default handling for addresses that don't have corresponding
users is to create new users with no passwords and a username equal to the
address. (The web interface does not permit logins for users with no
passwords.) If we prefer to reject mail from outside sources, we can simply
register an auditor on the "user" class that prevents the creation of user
nodes with no passwords.

Actions
-------
The subject line of the incoming message is examined to determine whether
the message is an attempt to create a new item or to discuss an existing
item. A designator enclosed in square brackets is sought as the first thing
on the subject line (after skipping any "Fwd:" or "Re:" prefixes).

If an item designator (class name and id number) is found there, the newly
created "msg" node is added to the "messages" property for that item, and
any new "file" nodes are added to the "files" property for the item.

If just an item class name is found there, we attempt to create a new item
of that class with its "messages" property initialized to contain the new
"msg" node and its "files" property initialized to contain any new "file"
nodes.

Triggers
--------
Both cases may trigger detectors (in the first case we are calling the
set() method to add the message to the item's spool; in the second case we
are calling the create() method to create a new node). If an auditor raises
an exception, the original message is bounced back to the sender with the
explanatory message given in the exception.
-------------- next part --------------
Return-Path: <python-dev at python.org>
X-Original-To: report at bugs.python.org
Delivered-To: roundup+tracker at psf.upfronthosting.co.za
Received: from mail.python.org (mail.python.org [IPv6:2001:888:2000:d::a6])
	by psf.upfronthosting.co.za (Postfix) with ESMTPS id 1F76E1C876
	for <report at bugs.python.org>; Mon,  1 Oct 2012 23:11:54 +0200 (CEST)
Received: from albatross.python.org (localhost [127.0.0.1])
	by mail.python.org (Postfix) with ESMTP id 3XVx5569b6zQsL
	for <report at bugs.python.org>; Mon,  1 Oct 2012 23:11:53 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=python.org; s=200901;
	t=1349125913; bh=bvONg+T9Lsep8hEPCu2kUnhY3MftIhWNUuY1WjZ7F58=;
	h=MIME-Version:Content-Type:Content-Transfer-Encoding:From:To:
	 Subject:Message-Id:Date;
	b=xmALJn1wwY3MQWnHpkjiY2RETSwhJQlyfvfzhqimSE5MnyqvmO9/FgTXdveEFWZWo
	 bryTanG+BI84fdYsRn37TuXNbUkcs/A7DjE1vWbpDUNdBkw3jdxQveGc8PWX/P/Ui2
	 aKOfPopoJgIywncnGIsrcycGG/hSfkkk/9tcb5YM=
Received: from localhost (HELO mail.python.org) (127.0.0.1)
  by albatross.python.org with SMTP; 01 Oct 2012 23:11:53 +0200
Received: from virt-7yvsjn.psf.osuosl.org (virt-7yvsjn.psf.osuosl.org [140.211.10.72])
	by mail.python.org (Postfix) with ESMTP
	for <report at bugs.python.org>; Mon,  1 Oct 2012 23:11:53 +0200 (CEST)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
From: python-dev at python.org
To: report at bugs.python.org
Subject: [issue16304]
Message-Id: <3XVx5569b6zQsL at mail.python.org>
Date: Mon,  1 Oct 2012 23:11:53 +0200 (CEST)

TmV3IGNoYW5nZXNldCBlNmQ4NzJiNjFjNTcgYnkgTmFkZWVtIFZhd2RhIGluIGJyYW5jaCAnMy4z
JzoKSXNzdWUgIzE2MzA0OiBBbm90aGVyIHBlcmZvcm1hbmNlIG9wdGltaXphdGlvbiBmb3IgQloy
RmlsZS4KaHR0cDovL2hnLnB5dGhvbi5vcmcvY3B5dGhvbi9yZXYvZTZkODcyYjYxYzU3CgoKTmV3
IGNoYW5nZXNldCA2ZDdiZjUxMmUwYzMgYnkgTmFkZWVtIFZhd2RhIGluIGJyYW5jaCAnMy4zJzoK
SXNzdWUgIzE2MzA0OiBGdXJ0aGVyIG9wdGltaXplIEJaMkZpbGUucmVhZGxpbmVzPygpLgpodHRw
Oi8vaGcucHl0aG9uLm9yZy9jcHl0aG9uL3Jldi82ZDdiZjUxMmUwYzMKCgpOZXcgY2hhbmdlc2V0
IGExOWY0N2QzODBkMiBieSBOYWRlZW0gVmF3ZGEgaW4gYnJhbmNoICdkZWZhdWx0JzoKTWVyZ2U6
ICMxNjMwNDogT3B0aW1pemF0aW9ucyBmb3IgQloyRmlsZSwgYW5kIG1pbm9yIGJ1Z2ZpeC4KaHR0
cDovL2hnLnB5dGhvbi5vcmcvY3B5dGhvbi9yZXYvYTE5ZjQ3ZDM4MGQyCg==

From nadeem.vawda at gmail.com  Mon Oct  1 23:21:48 2012
From: nadeem.vawda at gmail.com (Nadeem Vawda)
Date: Mon, 1 Oct 2012 23:21:48 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default):
 Merge: #16304: Optimizations for BZ2File, and minor bugfix.
In-Reply-To: <3XVx5C4T0BzQss@mail.python.org>
References: <3XVx5C4T0BzQss@mail.python.org>
Message-ID: <CANF4RMnY2O6+k+puTK6D9a9CxuMQRfDFLU5WDkz3KnDOJgt6_A@mail.gmail.com>

On Mon, Oct 1, 2012 at 11:11 PM, nadeem.vawda
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/a19f47d380d2
> changeset:   79382:a19f47d380d2
> parent:      79378:fb90e2ff95b7
> parent:      79381:6d7bf512e0c3
> user:        Nadeem Vawda <nadeem.vawda at gmail.com>
> date:        Mon Oct 01 23:11:35 2012 +0200
> summary:
>   Merge: #16304: Optimizations for BZ2File, and minor bugfix.

Correction: this is actually for issue *#16034*, as are all of my
other recent changesets referring to issue #16304.

From storchaka at gmail.com  Mon Oct  1 23:32:36 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 02 Oct 2012 00:32:36 +0300
Subject: [Python-Dev] cpython: utilize yield from
In-Reply-To: <3XVvMK75JzzQy7@mail.python.org>
References: <3XVvMK75JzzQy7@mail.python.org>
Message-ID: <k4d25k$8qr$1@ger.gmane.org>

On 01.10.12 22:54, philip.jenvey wrote:
> http://hg.python.org/cpython/rev/fb90e2ff95b7
> changeset:   79378:fb90e2ff95b7
> user:        Philip Jenvey <pjenvey at underboss.org>
> date:        Mon Oct 01 12:53:43 2012 -0700
> summary:
>    utilize yield from

This is not all.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: yield.patch
Type: text/x-patch
Size: 7261 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121002/56c3fa6b/attachment.bin>

From christian at python.org  Mon Oct  1 23:50:10 2012
From: christian at python.org (Christian Heimes)
Date: Mon, 01 Oct 2012 23:50:10 +0200
Subject: [Python-Dev] [RELEASED] Python 3.3.0
In-Reply-To: <20120930120104.GA6615@iskra.aviel.ru>
References: <5066E72E.2010100@python.org>
	<CAP7+vJJmQPfSi+0++RO6=AG400+OMCyKSe12paLf64Xi41Z8EA@mail.gmail.com>
	<k49a92$ltj$1@ger.gmane.org> <20120930120104.GA6615@iskra.aviel.ru>
Message-ID: <506A1012.6060808@python.org>

Am 30.09.2012 14:01, schrieb Oleg Broytman:
>    Many kudos to the team and to all contributors!
> 
>    Linux Weekly News regularly publishes tables "Who done what in Linux
> Kernel": http://lwn.net/Articles/507986/
> http://lwn.net/SubscriberLink/517564/bec11e6ace6ad699/
> 
>    It would be interesting to see tables like these for Python.

Ohloh has lots of statistics and graphics:

https://www.ohloh.net/p/python


From soltysh at gmail.com  Tue Oct  2 00:18:01 2012
From: soltysh at gmail.com (Maciej Szulik)
Date: Tue, 02 Oct 2012 00:18:01 +0200
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <5064D399.5050208@netwok.org>
References: <5064D399.5050208@netwok.org>
Message-ID: <506A1699.5080405@gmail.com>

On 09/28/2012 12:30 AM, ?ric Araujo wrote:
> Hi all,
>
> The Montreal-Python user group would like to host a bug day on October
> 27 (to be confirmed) at a partner university in Montreal.  It would be
> cool to do a bug day on IRC like we used to (and in other physical
> locations if people want to!) to get new contributors and close bugs.
> What do you think?
>
> Cheers


Eric,
How do you and Montreal-Pythoon user group see possibility to move this
event a little bit in time, to have enough time to organize everything
what's needed. In most cases peoples here in Poland. How about doing
this event whole month later, meaning end of November? If we all get
enough time to prepare and advertise it properly, we might attract
bigger group of people, which in long term means more hands to
work on bugs :)

Regards,
Maciej


From brian at python.org  Tue Oct  2 00:28:01 2012
From: brian at python.org (Brian Curtin)
Date: Mon, 1 Oct 2012 17:28:01 -0500
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <506A1699.5080405@gmail.com>
References: <5064D399.5050208@netwok.org>
	<506A1699.5080405@gmail.com>
Message-ID: <CAD+XWwoYk3VRk7osy-hq6Thjbemw+6gaX=FyMfGn-Ob-irz4Jw@mail.gmail.com>

On Mon, Oct 1, 2012 at 5:18 PM, Maciej Szulik <soltysh at gmail.com> wrote:
> On 09/28/2012 12:30 AM, ?ric Araujo wrote:
>>
>> Hi all,
>>
>> The Montreal-Python user group would like to host a bug day on October
>> 27 (to be confirmed) at a partner university in Montreal.  It would be
>> cool to do a bug day on IRC like we used to (and in other physical
>> locations if people want to!) to get new contributors and close bugs.
>> What do you think?
>>
>> Cheers
>
>
>
> Eric,
> How do you and Montreal-Pythoon user group see possibility to move this
> event a little bit in time, to have enough time to organize everything
> what's needed. In most cases peoples here in Poland. How about doing
> this event whole month later, meaning end of November? If we all get
> enough time to prepare and advertise it properly, we might attract
> bigger group of people, which in long term means more hands to
> work on bugs :)
>
> Regards,
> Maciej

You could also wait three months and get even more people, or wait 6
months and maybe get even *more* people. You could also lose everyone
else's interest during that time.

I think if people want to join up on the chosen date, it'd be
excellent. If they want to meet up on another date, this is also
excellent. It doesn't need to be a one time thing :)

From rdmurray at bitdance.com  Tue Oct  2 00:38:58 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 01 Oct 2012 18:38:58 -0400
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <CAD+XWwoYk3VRk7osy-hq6Thjbemw+6gaX=FyMfGn-Ob-irz4Jw@mail.gmail.com>
References: <5064D399.5050208@netwok.org> <506A1699.5080405@gmail.com>
	<CAD+XWwoYk3VRk7osy-hq6Thjbemw+6gaX=FyMfGn-Ob-irz4Jw@mail.gmail.com>
Message-ID: <20121001223859.417722500FB@webabinitio.net>

On Mon, 01 Oct 2012 17:28:01 -0500, Brian Curtin <brian at python.org> wrote:
> On Mon, Oct 1, 2012 at 5:18 PM, Maciej Szulik <soltysh at gmail.com> wrote:
> > On 09/28/2012 12:30 AM, ??ric Araujo wrote:
> >>
> >> Hi all,
> >>
> >> The Montreal-Python user group would like to host a bug day on October
> >> 27 (to be confirmed) at a partner university in Montreal.  It would be
> >> cool to do a bug day on IRC like we used to (and in other physical
> >> locations if people want to!) to get new contributors and close bugs.
> >> What do you think?
> >>
> >> Cheers
> >
> >
> >
> > Eric,
> > How do you and Montreal-Pythoon user group see possibility to move this
> > event a little bit in time, to have enough time to organize everything
> > what's needed. In most cases peoples here in Poland. How about doing
> > this event whole month later, meaning end of November? If we all get
> > enough time to prepare and advertise it properly, we might attract
> > bigger group of people, which in long term means more hands to
> > work on bugs :)
> >
> > Regards,
> > Maciej
> 
> You could also wait three months and get even more people, or wait 6
> months and maybe get even *more* people. You could also lose everyone
> else's interest during that time.
> 
> I think if people want to join up on the chosen date, it'd be
> excellent. If they want to meet up on another date, this is also
> excellent. It doesn't need to be a one time thing :)

Yes.  I think there are people already trying to to (or perhaps
even succeeding) in arranging spaces for that date, so changing
it would be a disruption to those plans.  On the other hand we
could also let those people speak up...

Having another one at the end of November would be great.

--David

From pjenvey at underboss.org  Tue Oct  2 00:43:14 2012
From: pjenvey at underboss.org (Philip Jenvey)
Date: Mon, 1 Oct 2012 15:43:14 -0700
Subject: [Python-Dev] cpython: utilize yield from
In-Reply-To: <k4d25k$8qr$1@ger.gmane.org>
References: <3XVvMK75JzzQy7@mail.python.org> <k4d25k$8qr$1@ger.gmane.org>
Message-ID: <B7F0E80C-1725-4985-B01C-C7BED265F846@underboss.org>


On Oct 1, 2012, at 2:32 PM, Serhiy Storchaka wrote:

> On 01.10.12 22:54, philip.jenvey wrote:
>> http://hg.python.org/cpython/rev/fb90e2ff95b7
>> changeset:   79378:fb90e2ff95b7
>> user:        Philip Jenvey <pjenvey at underboss.org>
>> date:        Mon Oct 01 12:53:43 2012 -0700
>> summary:
>>   utilize yield from
> 
> This is not all.

Applied, thanks Serhiy!

--
Philip Jenvey


From g.brandl at gmx.net  Tue Oct  2 01:00:16 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 02 Oct 2012 01:00:16 +0200
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <506A1699.5080405@gmail.com>
References: <5064D399.5050208@netwok.org> <506A1699.5080405@gmail.com>
Message-ID: <k4d79u$jft$1@ger.gmane.org>

On 10/02/2012 12:18 AM, Maciej Szulik wrote:
> On 09/28/2012 12:30 AM, ?ric Araujo wrote:
>> Hi all,
>>
>> The Montreal-Python user group would like to host a bug day on October
>> 27 (to be confirmed) at a partner university in Montreal.  It would be
>> cool to do a bug day on IRC like we used to (and in other physical
>> locations if people want to!) to get new contributors and close bugs.
>> What do you think?
>>
>> Cheers
>
>
> Eric,
> How do you and Montreal-Pythoon user group see possibility to move this
> event a little bit in time, to have enough time to organize everything
> what's needed. In most cases peoples here in Poland. How about doing
> this event whole month later, meaning end of November? If we all get
> enough time to prepare and advertise it properly, we might attract
> bigger group of people, which in long term means more hands to
> work on bugs :)

There's no reason there couldn't be another bug day in November...

Georg


From merwok at netwok.org  Tue Oct  2 01:08:44 2012
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Mon, 01 Oct 2012 19:08:44 -0400
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <20121001223859.417722500FB@webabinitio.net>
References: <5064D399.5050208@netwok.org> <506A1699.5080405@gmail.com>
	<CAD+XWwoYk3VRk7osy-hq6Thjbemw+6gaX=FyMfGn-Ob-irz4Jw@mail.gmail.com>
	<20121001223859.417722500FB@webabinitio.net>
Message-ID: <506A227C.3000007@netwok.org>

Le 01/10/2012 18:38, R. David Murray a ?crit :
> Yes.  I think there are people already trying to to (or perhaps
> even succeeding) in arranging spaces for that date, so changing
> it would be a disruption to those plans.
Yes, it?s always a bit painful to go through another round of email,
calls or meetings to reschedule.  Also it?s fun to kick off the idea and
see it take place, great for motivation!

Ned: I?m confident we?ll have a place for the planned date, you can make
arrangements and we?ll confirm as soon as possible (this week).

Maciej: If you don?t think you can let people know in time, by all means
lead another bug day some time after, and I?m sure there will be
participants on IRC.

Cheers

From db3l.net at gmail.com  Tue Oct  2 02:33:01 2012
From: db3l.net at gmail.com (David Bolen)
Date: Mon, 01 Oct 2012 20:33:01 -0400
Subject: [Python-Dev] Daily DMG builds
References: <20121001211812.4c40aa52@pitrou.net>
	<nad-FF6FDF.13201301102012@news.gmane.org>
Message-ID: <m2haqdlnlu.fsf@valheru.db3l.homeip.net>

Ned Deily <nad at acm.org> writes:

> In article <20121001211812.4c40aa52 at pitrou.net>,
>  Antoine Pitrou <solipsis at pitrou.net> wrote:
>
>> Hello,
>> 
>> It seems that the daily DMG builds have been failing for some time on
>> the default branch:
>> http://buildbot.python.org/daily-dmg/
>> 
>> Since there has been no report or complaint about that, should we just
>> stop producing those builds?
>
> It's failing on a checksum error of the cached download ncurses tarball 
> which is kind of odd.  I've pinged David about it.

Ned's email got to me before I got to the list today - the initial
issue does appear to be a bad ncurses external download from a while
back.  The download was Aug 6 so that seems to match when the 3.x dmg
stopped being produced.  I flushed that and let it grab a new copy.

The 3.3 branch then had a separate issue since it was brand new and
the Python-ast.[ch] rebuild process doesn't work right under 10.4, but
touching the files included in hg resolves that.  (If I recall it's
because it uses the system python which is ancient - I had done the
same fix a while back for the 3.x builder).

So I believe things should be ok at this point, or at least back to the
way they were previously.

> I must admit I don't pay attention to the dmg buildbot but I probably 
> should. 

I know I try to catch slave-related failures, but staying on top of
all of the builders can be a little tough.  I don't think there's a
good summary page for the slaves I want to follow, so it's tricky to
stay current (plus the trunk stuff tends to have more transient
failures).  Not sure if there's a way to have per-owner summary pages
or anything on the build master.  I do use the main build slaves
summary page for overall status, but scanning the individual builders
is 30+ other pages (the per-slave pages don't often have enough
history to cover all the builders), or trying to find each of my
builders in the overall waterfall page.  Neither approach tends to
survive very long in terms of my regularly doing it :-)

Anyway, I always welcome (and appreciate) any head's up for failures
that seem to be slave-related...

>          I know at least one person (Raymond) has reported using the dmg 
> builds regularly and the 2.7 builds are still going.  There is a bit of 
> an issue now for 3.3 and default: the buildbot-slave is apparently 
> running OS X 10.4 so it is no longer capable of building either of the 
> two installer variants that we release on python.org (one now requires 
> at least 10.5 and the other at least 10.6).  However, we don't want to 
> break anything on 10.4, if possible, and the installers still work on 
> later systems so there is some value in producing them - as long as 
> someone is paying attention when they break and, hopefully, someone uses 
> them.

It's been a few years since setting this up, but at least at one point
I think it was safer to build an installer on the older platform in
terms of compatibility, but it wouldn't surprise me if that process
has become dated, or no longer necessary.  In fact, if I recall
correctly, I think Ronald was already using a later platform for his
own building when the daily buildbot was set up.

At the time I needed a 10.4 platform for local application building (I
had an app supporting 10.4 clients and couldn't build it on a later
OSX and still have it install on older systems), but it was otherwise
idle so available as a buildbot.  The physical machine is unlikely to
get upgraded beyond 10.4 though.

It's never been entirely clear how useful the daily build slaves are.
We used to do daily Windows builds too but when they ran into some
problems the demand (or lack thereof) didn't seem to justify the time
to troubleshoot so they were discontinued.  My guess is the same will
probably happen to the OSX daily build slave once it hits something
difficult to troubleshoot/fix.  Although until it does I suppose
there's no need to explicitly turn it off.


It would still be interesting to know who may still be making use of
the images being produced.

-- David


From guido at python.org  Tue Oct  2 07:24:44 2012
From: guido at python.org (Guido van Rossum)
Date: Mon, 1 Oct 2012 22:24:44 -0700
Subject: [Python-Dev] Bad python 2.5 build on OSX 10.8 mountain lion
Message-ID: <CAP7+vJLQLLgDOB83XBg9-=+FyS7+1qEkr7inAL5igD4bJHpiiA@mail.gmail.com>

As discussed here, the python 2.5 binary distributed by Apple on mountain
lion is broken. Could someone file an official complaint? This is really
bad...

--Guido


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/e3c5f7a8/attachment.html>

From guido at python.org  Tue Oct  2 07:25:17 2012
From: guido at python.org (Guido van Rossum)
Date: Mon, 1 Oct 2012 22:25:17 -0700
Subject: [Python-Dev] Bad python 2.5 build on OSX 10.8 mountain lion
In-Reply-To: <CAP7+vJLQLLgDOB83XBg9-=+FyS7+1qEkr7inAL5igD4bJHpiiA@mail.gmail.com>
References: <CAP7+vJLQLLgDOB83XBg9-=+FyS7+1qEkr7inAL5igD4bJHpiiA@mail.gmail.com>
Message-ID: <CAP7+vJLYBgPCmKNT5UVzQ9Na3qBdumE9CmJ5PL6yZqR75AyAnw@mail.gmail.com>

Forgot the link...
http://code.google.com/p/googleappengine/issues/detail?id=7885

On Monday, October 1, 2012, Guido van Rossum wrote:

> As discussed here, the python 2.5 binary distributed by Apple on mountain
> lion is broken. Could someone file an official complaint? This is really
> bad...
>
> --Guido
>
>
> --
> --Guido van Rossum (python.org/~guido)
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121001/11257586/attachment.html>

From nad at acm.org  Tue Oct  2 09:18:20 2012
From: nad at acm.org (Ned Deily)
Date: Tue, 02 Oct 2012 00:18:20 -0700
Subject: [Python-Dev] Bad python 2.5 build on OSX 10.8 mountain lion
References: <CAP7+vJLQLLgDOB83XBg9-=+FyS7+1qEkr7inAL5igD4bJHpiiA@mail.gmail.com>
	<CAP7+vJLYBgPCmKNT5UVzQ9Na3qBdumE9CmJ5PL6yZqR75AyAnw@mail.gmail.com>
Message-ID: <nad-AD48B0.00182002102012@news.gmane.org>

In article 
<CAP7+vJLYBgPCmKNT5UVzQ9Na3qBdumE9CmJ5PL6yZqR75AyAnw at mail.gmail.com>,
 Guido van Rossum <guido at python.org> wrote:

> Forgot the link...
> http://code.google.com/p/googleappengine/issues/detail?id=7885
> 
> On Monday, October 1, 2012, Guido van Rossum wrote:
> 
> > As discussed here, the python 2.5 binary distributed by Apple on mountain
> > lion is broken. Could someone file an official complaint? This is really
> > bad...

I've filed a bug against 10.8 python2.5.   The 10.8 versions of Apple's 
pythons are compile with clang and we did see some sign extension issues 
with ctypes.  The 10.7 version of Apple's python2.5 is compiled with 
llvm-gcc and handles 2**31 correctly.

Unfortunately, AFAIK, only the submitter (i.e. me) can see the bug 
report (problem id 12411585).  I'll report back when I hear something.

-- 
 Ned Deily,
 nad at acm.org


From stefan at bytereef.org  Tue Oct  2 09:31:35 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Tue, 2 Oct 2012 09:31:35 +0200
Subject: [Python-Dev] Bad python 2.5 build on OSX 10.8 mountain lion
In-Reply-To: <nad-AD48B0.00182002102012@news.gmane.org>
References: <CAP7+vJLQLLgDOB83XBg9-=+FyS7+1qEkr7inAL5igD4bJHpiiA@mail.gmail.com>
	<CAP7+vJLYBgPCmKNT5UVzQ9Na3qBdumE9CmJ5PL6yZqR75AyAnw@mail.gmail.com>
	<nad-AD48B0.00182002102012@news.gmane.org>
Message-ID: <20121002073135.GA26567@sleipnir.bytereef.org>

Ned Deily <nad at acm.org> wrote:
> > Forgot the link...
> > http://code.google.com/p/googleappengine/issues/detail?id=7885
> > 
> > On Monday, October 1, 2012, Guido van Rossum wrote:
> > 
> > > As discussed here, the python 2.5 binary distributed by Apple on mountain
> > > lion is broken. Could someone file an official complaint? This is really
> > > bad...
> 
> I've filed a bug against 10.8 python2.5.   The 10.8 versions of Apple's 
> pythons are compile with clang and we did see some sign extension issues 
> with ctypes.  The 10.7 version of Apple's python2.5 is compiled with 
> llvm-gcc and handles 2**31 correctly.

Yes, this looks like http://bugs.python.org/issue11149 .


Stefan Krah



From nad at acm.org  Tue Oct  2 09:47:59 2012
From: nad at acm.org (Ned Deily)
Date: Tue, 02 Oct 2012 00:47:59 -0700
Subject: [Python-Dev] Bad python 2.5 build on OSX 10.8 mountain lion
References: <CAP7+vJLQLLgDOB83XBg9-=+FyS7+1qEkr7inAL5igD4bJHpiiA@mail.gmail.com>
	<CAP7+vJLYBgPCmKNT5UVzQ9Na3qBdumE9CmJ5PL6yZqR75AyAnw@mail.gmail.com>
	<nad-AD48B0.00182002102012@news.gmane.org>
	<20121002073135.GA26567@sleipnir.bytereef.org>
Message-ID: <nad-B4E67A.00475902102012@news.gmane.org>

In article <20121002073135.GA26567 at sleipnir.bytereef.org>,
 Stefan Krah <stefan at bytereef.org> wrote:
> Ned Deily <nad at acm.org> wrote:
> > > Forgot the link...
> > > http://code.google.com/p/googleappengine/issues/detail?id=7885
> > > 
> > > On Monday, October 1, 2012, Guido van Rossum wrote:
> > > 
> > > > As discussed here, the python 2.5 binary distributed by Apple on 
> > > > mountain
> > > > lion is broken. Could someone file an official complaint? This is 
> > > > really
> > > > bad...
> > 
> > I've filed a bug against 10.8 python2.5.   The 10.8 versions of Apple's 
> > pythons are compile with clang and we did see some sign extension issues 
> > with ctypes.  The 10.7 version of Apple's python2.5 is compiled with 
> > llvm-gcc and handles 2**31 correctly.
> Yes, this looks like http://bugs.python.org/issue11149 .

Ah, right, thanks.  I've updated the Apple issue accordingly.

-- 
 Ned Deily,
 nad at acm.org


From phd at phdru.name  Tue Oct  2 15:45:24 2012
From: phd at phdru.name (Oleg Broytman)
Date: Tue, 2 Oct 2012 17:45:24 +0400
Subject: [Python-Dev] [RELEASED] Python 3.3.0
In-Reply-To: <506A1012.6060808@python.org>
References: <5066E72E.2010100@python.org>
	<CAP7+vJJmQPfSi+0++RO6=AG400+OMCyKSe12paLf64Xi41Z8EA@mail.gmail.com>
	<k49a92$ltj$1@ger.gmane.org> <20120930120104.GA6615@iskra.aviel.ru>
	<506A1012.6060808@python.org>
Message-ID: <20121002134524.GA12984@iskra.aviel.ru>

On Mon, Oct 01, 2012 at 11:50:10PM +0200, Christian Heimes <christian at python.org> wrote:
> Am 30.09.2012 14:01, schrieb Oleg Broytman:
> >    Many kudos to the team and to all contributors!
> > 
> >    Linux Weekly News regularly publishes tables "Who done what in Linux
> > Kernel": http://lwn.net/Articles/507986/
> > http://lwn.net/SubscriberLink/517564/bec11e6ace6ad699/
> > 
> >    It would be interesting to see tables like these for Python.
> 
> Ohloh has lots of statistics and graphics:
> 
> https://www.ohloh.net/p/python

   Good enough, thank you!

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From g.brandl at gmx.net  Tue Oct  2 17:16:15 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 02 Oct 2012 17:16:15 +0200
Subject: [Python-Dev] experimental: Misc/NEWS included in docs
In-Reply-To: <20120930223120.2c2e22c2@pitrou.net>
References: <k49gpe$2dt$1@ger.gmane.org>
	<CAOTb1wd364Yc00JHVSkB9ahK1LOa4J8N+9Ve76_i6HMc7yFLOA@mail.gmail.com>
	<20120930223120.2c2e22c2@pitrou.net>
Message-ID: <k4f0ft$ngq$2@ger.gmane.org>

On 09/30/2012 10:31 PM, Antoine Pitrou wrote:
> On Sun, 30 Sep 2012 12:14:37 -0700
> Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
>
>> On Sun, Sep 30, 2012 at 6:17 AM, Georg Brandl <g.brandl at gmx.net> wrote:
>> > at http://docs.python.org/3.3/whatsnew/news.html, there is now
>> > a rendering of Misc/NEWS with tracker links and a crude filtering
>> > capability.  I thought that this will complement the "whatsnew"
>> > documents nicely for people looking for more detail.
>> >
>> > Please let me know if it's useful, or what could be done to make
>> > it *more* useful.
>>
>> Good idea.  Some suggestions:
>>
>> * A note at the top explaining what this page is and how it differs
>> from the other What's New page (including perhaps a link to the other
>> What's New page).  It might also be good if the two had more
>> distinctive titles.  They are currently "What?s New in Python" and
>> "Python News."
>
> Agreed. I think the titles are currently very confusing. The page
> you've just added should be called something like "Changelog", to
> stress its more bare'n'technical natural compared to our official
> "What's New" pages.
>
> Also it would be extra nice if the "What's new in Python X.Y" section
> headers could be renamed to "Python X.Y changelog", or similar.

Please have a look again (it's renamed to .../changelog.html); Ezio and I
tried to incorporate all the suggestions.

Georg


From ncoghlan at gmail.com  Tue Oct  2 18:01:40 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 2 Oct 2012 21:31:40 +0530
Subject: [Python-Dev] experimental: Misc/NEWS included in docs
In-Reply-To: <k4f0ft$ngq$2@ger.gmane.org>
References: <k49gpe$2dt$1@ger.gmane.org>
	<CAOTb1wd364Yc00JHVSkB9ahK1LOa4J8N+9Ve76_i6HMc7yFLOA@mail.gmail.com>
	<20120930223120.2c2e22c2@pitrou.net> <k4f0ft$ngq$2@ger.gmane.org>
Message-ID: <CADiSq7fgO2U_C5nftRnzrh7ZBFwvo1xrWSk4ZoOUp8QTgKe3eQ@mail.gmail.com>

On Tue, Oct 2, 2012 at 8:46 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>
> Please have a look again (it's renamed to .../changelog.html); Ezio and I
> tried to incorporate all the suggestions.

Very nice. I tweaked the wording in the 3.3.1 section to be more
explicit that it hasn't been released yet and people should look
further down for the 3.3.0 changes.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From merwok at netwok.org  Tue Oct  2 20:47:55 2012
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 02 Oct 2012 14:47:55 -0400
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <5064D399.5050208@netwok.org>
References: <5064D399.5050208@netwok.org>
Message-ID: <506B36DB.1090001@netwok.org>

Hello,

We confirm October 27!  User groups and devs on IRC, join us if you can.
 I?ll send an announcement to the mentoring list to let subscribers know
about the event.

Cheers

From soltysh at gmail.com  Tue Oct  2 23:39:43 2012
From: soltysh at gmail.com (Maciej Szulik)
Date: Tue, 02 Oct 2012 23:39:43 +0200
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <506B36DB.1090001@netwok.org>
References: <5064D399.5050208@netwok.org> <506B36DB.1090001@netwok.org>
Message-ID: <506B5F1F.7040300@gmail.com>

On 10/02/2012 08:47 PM, ?ric Araujo wrote:
> Hello,
>
> We confirm October 27!  User groups and devs on IRC, join us if you can.
>   I?ll send an announcement to the mentoring list to let subscribers know
> about the event.
>
> Cheers

I'll be joining this particular event for sure, through IRC, I was
planing to get more Poles interested in topic ;)
Thanks for clearing things, I'll see what can be done with another Bug 
Day :)

Regards,
Maciej


From rosuav at gmail.com  Wed Oct  3 00:14:06 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 3 Oct 2012 08:14:06 +1000
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <506B36DB.1090001@netwok.org>
References: <5064D399.5050208@netwok.org>
	<506B36DB.1090001@netwok.org>
Message-ID: <CAPTjJmqGWfoCKKBHuJwJGpg-roNoQQ0NMt_HVDCAaau4U+4MQA@mail.gmail.com>

On Wed, Oct 3, 2012 at 4:47 AM, ?ric Araujo <merwok at netwok.org> wrote:
> Hello,
>
> We confirm October 27!  User groups and devs on IRC, join us if you can.
>  I?ll send an announcement to the mentoring list to let subscribers know
> about the event.

Apologies if this is a stupid question (or just completely
misdirected), but does this mean that if I'm interested in
participating in the bug day, the first step should be to join
core-mentorship@?

Thanks!

ChrisA

From merwok at netwok.org  Wed Oct  3 00:19:08 2012
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 02 Oct 2012 18:19:08 -0400
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <CAPTjJmqGWfoCKKBHuJwJGpg-roNoQQ0NMt_HVDCAaau4U+4MQA@mail.gmail.com>
References: <5064D399.5050208@netwok.org> <506B36DB.1090001@netwok.org>
	<CAPTjJmqGWfoCKKBHuJwJGpg-roNoQQ0NMt_HVDCAaau4U+4MQA@mail.gmail.com>
Message-ID: <506B685C.3050608@netwok.org>

Hi Chris,

Le 02/10/2012 18:14, Chris Angelico a ?crit :
> Apologies if this is a stupid question (or just completely
> misdirected), but does this mean that if I'm interested in
> participating in the bug day, the first step should be to join
> core-mentorship@?

It?s not required, but you are welcome to do it.  I?ll send an
announcement with details about participating soon?.

Regards

From rosuav at gmail.com  Wed Oct  3 00:30:23 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 3 Oct 2012 08:30:23 +1000
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <506B685C.3050608@netwok.org>
References: <5064D399.5050208@netwok.org> <506B36DB.1090001@netwok.org>
	<CAPTjJmqGWfoCKKBHuJwJGpg-roNoQQ0NMt_HVDCAaau4U+4MQA@mail.gmail.com>
	<506B685C.3050608@netwok.org>
Message-ID: <CAPTjJmoC6+fyBB_bzOAJXyGbtdG61K5VNJ02aK1pYjHdujtxkw@mail.gmail.com>

On Wed, Oct 3, 2012 at 8:19 AM, ?ric Araujo <merwok at netwok.org> wrote:
> Hi Chris,
>
> Le 02/10/2012 18:14, Chris Angelico a ?crit :
>> Apologies if this is a stupid question (or just completely
>> misdirected), but does this mean that if I'm interested in
>> participating in the bug day, the first step should be to join
>> core-mentorship@?
>
> It?s not required, but you are welcome to do it.  I?ll send an
> announcement with details about participating soon?.

Thanks. I've joined the list (as if I needed *still more* email in my
inbox... heh), as it seems the best way to get those critical details
of start/end times, IRC channel, etc, etc.

Thanks for organizing this. It's a good excuse/opportunity to get started!

ChrisA

From storchaka at gmail.com  Wed Oct  3 10:54:28 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Wed, 03 Oct 2012 11:54:28 +0300
Subject: [Python-Dev] Should vars() return modifiable dict?
Message-ID: <k4gug7$e1q$1@ger.gmane.org>

For objects with __dict__ vars() returns modifiable dict:

 >>> class A():
...     pass
...
 >>> a = A()
 >>> a.x = 42
 >>> vars(a)
{'x': 42}
 >>> vars(a)['x'] = 43
 >>> vars(a)['y'] = 44
 >>> a.x, a.y, vars(a)
(43, 44, {'y': 44, 'x': 43})

For globals vars() returns modifiable dict:

 >>> x = 42
 >>> vars()['x']
42
 >>> vars()['x'] = 43
 >>> vars()['y'] = 44
 >>> vars()['x'], vars()['y']
(43, 44)

For locals vars() returns... hmm, partially modifiable dict:

 >>> def f():
...     x = 42
...     print(vars())
...     vars()['x'] = 43
...     vars()['y'] = 44
...     print(x, vars())
...
 >>> f()
{'x': 42}
42 {'y': 44, 'x': 42}

Should behavior of vars() be corrected for locals?

Should vars() for objects with __slots__ [1] returns modifiable or 
non-modifiable dict?

[1] http://bugs.python.org/issue13290


From larry at hastings.org  Wed Oct  3 13:27:37 2012
From: larry at hastings.org (Larry Hastings)
Date: Wed, 03 Oct 2012 13:27:37 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
Message-ID: <506C2129.5010908@hastings.org>


Howdy howdy.  Unless someone has a better idea, I'm the release manager 
for Python 3.4.  I've roughed out a release schedule, assuming a 
16-month period between 3.3 and 3.4.  It works out to having 3.4 ship 
about seven weeks before the PyCon 2014 core dev sprint, so even if we 
slip some we should still have a fresh 3.5 "default" branch open for 
some hacking.

Here are the dates, note that they're all Saturdays:

    - 3.4.0 alpha 1: August 3, 2013
    - 3.4.0 alpha 2: August 31, 2013
    - 3.4.0 alpha 3: September 28, 2013
    - 3.4.0 alpha 4: October 19, 2013
    - 3.4.0 beta 1: November 23, 2013

    (Beta 1 is also "feature freeze"--no new features beyond this point.)

    - 3.4.0 beta 2: January 4, 2014
    - 3.4.0 candidate 1: January 18, 2014
    - 3.4.0 candidate 2: February 1, 2014
    - 3.4.0 final: February 22, 2014


I welcome your feedback--without any further input I'll go ahead and 
post this as a PEP in a week or so.

Happy hacking,


//arry/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121003/b118ba75/attachment.html>

From ncoghlan at gmail.com  Wed Oct  3 13:40:10 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 3 Oct 2012 17:10:10 +0530
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C2129.5010908@hastings.org>
References: <506C2129.5010908@hastings.org>
Message-ID: <CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>

On Wed, Oct 3, 2012 at 4:57 PM, Larry Hastings <larry at hastings.org> wrote:
>
> Howdy howdy.  Unless someone has a better idea, I'm the release manager for
> Python 3.4.  I've roughed out a release schedule, assuming a 16-month period
> between 3.3 and 3.4.  It works out to having 3.4 ship about seven weeks
> before the PyCon 2014 core dev sprint, so even if we slip some we should
> still have a fresh 3.5 "default" branch open for some hacking.
>
> Here are the dates, note that they're all Saturdays:
>
> - 3.4.0 alpha 1: August 3, 2013

Looks pretty good to me, but I'd still like to experiment with
bringing this one up a few months (say to April, a few weeks after
PyCon US 2013). I'm sure we can land enough interesting features
between now and then for people to be willing to tinker with it and
give us feedback :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From skip at pobox.com  Wed Oct  3 13:45:48 2012
From: skip at pobox.com (Skip Montanaro)
Date: Wed, 3 Oct 2012 06:45:48 -0500
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C2129.5010908@hastings.org>
References: <506C2129.5010908@hastings.org>
Message-ID: <CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>

> I've roughed out a release schedule....

Is there a rough list of changes for 3.4 written down somewhere, or is
that only to be inferred based on PEPs whose Python-Version header
reads "3.4"?  How confident are you that the schedule you've proposed
gives enough time for proposed changes to be implemented and to
stablize?

Skip

From larry at hastings.org  Wed Oct  3 13:46:44 2012
From: larry at hastings.org (Larry Hastings)
Date: Wed, 03 Oct 2012 13:46:44 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
Message-ID: <506C25A4.5090900@hastings.org>

On 10/03/2012 01:40 PM, Nick Coghlan wrote:
> On Wed, Oct 3, 2012 at 4:57 PM, Larry Hastings <larry at hastings.org> wrote:
>> - 3.4.0 alpha 1: August 3, 2013
> Looks pretty good to me, but I'd still like to experiment with
> bringing this one up a few months (say to April, a few weeks after
> PyCon US 2013). I'm sure we can land enough interesting features
> between now and then for people to be willing to tinker with it and
> give us feedback :)

I don't agree.  It's my understanding that the alphas are largely 
ignored, and having them earlier would hardly make them more relevant.  
I'm not saying "no"--but I'd definitely want to see more people than 
just you clamoring for the early alphas before I agree to anything.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121003/1062ad75/attachment.html>

From larry at hastings.org  Wed Oct  3 13:54:16 2012
From: larry at hastings.org (Larry Hastings)
Date: Wed, 03 Oct 2012 13:54:16 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>
Message-ID: <506C2768.4010408@hastings.org>


On 10/03/2012 01:45 PM, Skip Montanaro wrote:
> Is there a rough list of changes for 3.4 written down somewhere, or is 
> that only to be inferred based on PEPs whose Python-Version header 
> reads "3.4"? How confident are you that the schedule you've proposed 
> gives enough time for proposed changes to be implemented and to stablize? 

Here's a short list of proposed changes for 3.4--basically all the stuff 
that didn't make it in to 3.3:

    Candidate PEPs:

    * PEP 395: Qualified Names for Modules
    * PEP 3143: Standard daemon process library
    * PEP 3154: Pickle protocol version 4

    Other proposed large-scale changes:

    * Breaking out standard library and docs in separate repos
    * Addition of the "packaging" module, deprecating "distutils"
    * Addition of the "regex" module
    * Email version 6
    * A standard event-loop interface (PEP by Jim Fulton pending)


As for the rest of it, my understanding was that there is no longer any 
great plan written in the stars for Python releases.  Python releases 
are comprised of whatever features people propose, implement, and are 
willing to support, that they can get done in time for the beta cutoff.  
>From that perspective, the schedule drives the features more than the 
other-way around.

The schedule proposes six weeks between feature-freeze and release. If 
that isn't enough we'll slip a little.  But given that we have little 
idea what features will make it in to 3.4, it's hard to say for certain 
whether or not that's enough time.  At this point all I can do is 
propose a schedule and hope for the best.

Here's hoping,


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121003/0fd5019a/attachment.html>

From g.brandl at gmx.net  Wed Oct  3 14:00:12 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 03 Oct 2012 14:00:12 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C2768.4010408@hastings.org>
References: <506C2129.5010908@hastings.org>
	<CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>
	<506C2768.4010408@hastings.org>
Message-ID: <k4h9ca$8bt$1@ger.gmane.org>

On 10/03/2012 01:54 PM, Larry Hastings wrote:
>
> On 10/03/2012 01:45 PM, Skip Montanaro wrote:
>> Is there a rough list of changes for 3.4 written down somewhere, or is that
>> only to be inferred based on PEPs whose Python-Version header reads "3.4"? How
>> confident are you that the schedule you've proposed gives enough time for
>> proposed changes to be implemented and to stablize?
>
> Here's a short list of proposed changes for 3.4--basically all the stuff that
> didn't make it in to 3.3:
>
>     Candidate PEPs:
>
>     * PEP 395: Qualified Names for Modules
>     * PEP 3143: Standard daemon process library
>     * PEP 3154: Pickle protocol version 4
>
>     Other proposed large-scale changes:
>
>     * Breaking out standard library and docs in separate repos
>     * Addition of the "packaging" module, deprecating "distutils"
>     * Addition of the "regex" module
>     * Email version 6
>     * A standard event-loop interface (PEP by Jim Fulton pending)
>
>
> As for the rest of it, my understanding was that there is no longer any great
> plan written in the stars for Python releases.  Python releases are comprised of
> whatever features people propose, implement, and are willing to support, that
> they can get done in time for the beta cutoff.  From that perspective, the
> schedule drives the features more than the other-way around.
>
> The schedule proposes six weeks between feature-freeze and release.

Looks more like 3 months, which is fine.

Georg


From barry at python.org  Wed Oct  3 15:46:31 2012
From: barry at python.org (Barry Warsaw)
Date: Wed, 3 Oct 2012 09:46:31 -0400
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>
Message-ID: <20121003094631.7d470105@limelight.wooz.org>

On Oct 03, 2012, at 06:45 AM, Skip Montanaro wrote:

>Is there a rough list of changes for 3.4 written down somewhere

Let the wild rumpus begin!

-Barry


From ncoghlan at gmail.com  Wed Oct  3 16:55:17 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 3 Oct 2012 20:25:17 +0530
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C25A4.5090900@hastings.org>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
Message-ID: <CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>

On Wed, Oct 3, 2012 at 5:16 PM, Larry Hastings <larry at hastings.org> wrote:
> I don't agree.  It's my understanding that the alphas are largely ignored,
> and having them earlier would hardly make them more relevant.

I would appreciate it you stopped promoting this myth. Each step in
the release process widens the pool of people providing feedback. Some
are providing feedback (and patches!) right the way through by
building their own copy of Python from source, a few start poking
around with the first alpha, more wait for feature freeze, we get a
whole slew of people that wait until the release candidates come out,
and then we get even more that don't check for backwards incompatible
changes until after the final release (so they have to wait until the
x.y.1 release before they can upgrade).

Yes, the pool is substantially smaller in the early phases, but
phrases like "largely ignored" do a grave disservice to our alpha
testers that provide early feedback when we have plenty of time to fix
problems, rather than leaving their checks to the last minute and
forcing us to choose between delaying the release and shipping with
known defects.

>  I'm not
> saying "no"--but I'd definitely want to see more people than just you
> clamoring for the early alphas before I agree to anything.

Python 3.4 will almost certainly include significant changes to main
module and sys.path initialisation as well as the way import failures
are reported at the command line (and perhaps in the interactive
interpreter), along with some adjustments to the Unicode handling
feature set and the disassembly support. I *can't* effectively trial
those changes on PyPI (except perhaps some of the disassembly
changes), and I don't have the resources to create and distribute
Windows and Mac OS X installers on my own. That means, before the
release of 3.4a1, any feedback on most of these changes will be
limited to those developers with the wherewithal to build Python from
source.

Regardless of when the first alpha happens, I'll be promoting the hell
out of it, begging for feedback on any of these changes that are
available by then (which should be quite a few, given the preceding
PyCon US sprints). However, I would *like* to have months rather than
weeks to act on any feedback we do receive. I'm not asking the release
team to do any more work - I'm just asking for a chunk of it to be
brought forward a few months. If I was asking for an *extra* release,
I could understand resistance to the idea, but what's the concrete
benefit of *delaying* the first alpha release by 4 months from when
I'm hoping to see it happen?

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From larry at hastings.org  Wed Oct  3 17:13:18 2012
From: larry at hastings.org (Larry Hastings)
Date: Wed, 03 Oct 2012 17:13:18 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
Message-ID: <506C560E.80306@hastings.org>

On 10/03/2012 04:55 PM, Nick Coghlan wrote:
> Regardless of when the first alpha happens, I'll be promoting the hell
> out of it, begging for feedback on any of these changes that are
> available by then (which should be quite a few, given the preceding
> PyCon US sprints).

If you can show me people who use the alphas who want them earlier, I'll 
consider it.  So far the only person who's said they want them is you, 
and IIUC you won't be a consumer of the alpha per se.

Begging for feedback doesn't mean you'll get any,


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121003/ff98aaf5/attachment.html>

From skip at pobox.com  Wed Oct  3 17:21:47 2012
From: skip at pobox.com (Skip Montanaro)
Date: Wed, 3 Oct 2012 10:21:47 -0500
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C560E.80306@hastings.org>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
Message-ID: <CANc-5Uzsxh2Coysjr73ZKxAYR-Lqy7MJNEx=TRyOrciLRbvznw@mail.gmail.com>

> If you can show me people who use the alphas who want them earlier, I'll
> consider it.  So far the only person who's said they want them is you, and
> IIUC you won't be a consumer of the alpha per se.
>
> Begging for feedback doesn't mean you'll get any,

I haven't done any Python core development work in quite awhile, but
BITD, I always just used the current tip of whatever version control
system we happened to be using.  That would automatically get me all
the pre-release versions.  No need to beg. :-)

Skip

From rdmurray at bitdance.com  Wed Oct  3 17:22:10 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 03 Oct 2012 11:22:10 -0400
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
Message-ID: <20121003152210.5A4A025010D@webabinitio.net>

On Wed, 03 Oct 2012 20:25:17 +0530, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Wed, Oct 3, 2012 at 5:16 PM, Larry Hastings <larry at hastings.org> wrote:
> > I don't agree.  It's my understanding that the alphas are largely ignored,
> > and having them earlier would hardly make them more relevant.
> 
> I would appreciate it you stopped promoting this myth. Each step in
> the release process widens the pool of people providing feedback. Some
> are providing feedback (and patches!) right the way through by
> building their own copy of Python from source, a few start poking
> around with the first alpha, more wait for feature freeze, we get a
> whole slew of people that wait until the release candidates come out,
> and then we get even more that don't check for backwards incompatible
> changes until after the final release (so they have to wait until the
> x.y.1 release before they can upgrade).
> 
> Yes, the pool is substantially smaller in the early phases, but
> phrases like "largely ignored" do a grave disservice to our alpha
> testers that provide early feedback when we have plenty of time to fix
> problems, rather than leaving their checks to the last minute and
> forcing us to choose between delaying the release and shipping with
> known defects.

I don't have any data to back this up, but it is my impression that more
distributions are providing access to alpha releases in their "testing"
package trees.  As Python continues to grow in importance[1], the number
of people interacting with Python on the distribution development teams[2]
increases, and therefor the number of people likely to run alphas for
testing increases.  So even if Larry were right *now*, he isn't right
for the future, and we should do all we can to nurture an increasing
culture of alpha-testing.

--David

[1] in case anyone hasn't noticed, we *are* a growing community, regardless
    of where various analytics put us relative to other languages :) 

[2] I include things like macports in this category, though I have
    no experience myself with their culture

From ncoghlan at gmail.com  Wed Oct  3 17:27:40 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 3 Oct 2012 20:57:40 +0530
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C560E.80306@hastings.org>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
Message-ID: <CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>

On Wed, Oct 3, 2012 at 8:43 PM, Larry Hastings <larry at hastings.org> wrote:
> Begging for feedback doesn't mean you'll get any,

I received a fair number of complaints from people that wanted to
experiment with yield from, but couldn't, because the first alpha
wasn't out yet and they weren't sufficiently interested to go to the
effort of building their own copy of Python.

*People like to try out the new versions*, so I have often received
useful feedback during alpha periods - all it takes is one interested
person from outside the core development team to get interested in a
new feature and we can have a useful conversation. By *refusing* to
release the alpha early you are *guaranteeing* I won't get the early
feedback I want on the new features that are likely to come in 3.4.

That's your prerogative as RM of course, but you haven't given any
reason beyond the circular "I don't care about enabling feedback from
people that can't or won't build from source, because people that
can't or won't build from source don't provide useful feedback". I'm
*not* happy about that attitude, because it's based on a blatantly
false premise. If it was true, we wouldn't bother with alpha releases
*at all*, and instead just skip straight to feature freeze and beta
releases.

Give me a reason like "I don't want to because I want to concentrate
the release management work into a 6 month period", and I'll accept
it, but don't try to rationalise it with statements about user
feedback that I know from experience to be untrue.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From brian at python.org  Wed Oct  3 17:28:13 2012
From: brian at python.org (Brian Curtin)
Date: Wed, 3 Oct 2012 10:28:13 -0500
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C560E.80306@hastings.org>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
Message-ID: <CAD+XWwpF91QvUUAd7XPwuN5d0YQEZHWg9WPF5o2tZWggaGP3WA@mail.gmail.com>

On Wed, Oct 3, 2012 at 10:13 AM, Larry Hastings <larry at hastings.org> wrote:
> On 10/03/2012 04:55 PM, Nick Coghlan wrote:
>
> Regardless of when the first alpha happens, I'll be promoting the hell
> out of it, begging for feedback on any of these changes that are
> available by then (which should be quite a few, given the preceding
> PyCon US sprints).
>
>
> If you can show me people who use the alphas who want them earlier, I'll
> consider it.  So far the only person who's said they want them is you, and
> IIUC you won't be a consumer of the alpha per se.
>
> Begging for feedback doesn't mean you'll get any,

This doesn't answer the question of the users wanting the alphas
earlier, but they're certainly more than largely ignored...

The webstats in April 2012 show 5628 downloads of 3.3a1 and 4946
downloads of 3.3a2 Windows installers.

From fijall at gmail.com  Wed Oct  3 17:29:06 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 3 Oct 2012 17:29:06 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
Message-ID: <CAK5idxTt9_BMyU8t9QaXW7eF8x2d9wbP-66a8vOeYuaFVPECwg@mail.gmail.com>

On Wed, Oct 3, 2012 at 5:27 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Wed, Oct 3, 2012 at 8:43 PM, Larry Hastings <larry at hastings.org> wrote:
>> Begging for feedback doesn't mean you'll get any,
>
> I received a fair number of complaints from people that wanted to
> experiment with yield from, but couldn't, because the first alpha
> wasn't out yet and they weren't sufficiently interested to go to the
> effort of building their own copy of Python.
>

How about having nightly builds then?

From ncoghlan at gmail.com  Wed Oct  3 17:29:38 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 3 Oct 2012 20:59:38 +0530
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CAD+XWwpF91QvUUAd7XPwuN5d0YQEZHWg9WPF5o2tZWggaGP3WA@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CAD+XWwpF91QvUUAd7XPwuN5d0YQEZHWg9WPF5o2tZWggaGP3WA@mail.gmail.com>
Message-ID: <CADiSq7eoBBv+nxOboZ-hi7qqgUX1mn_5c2dNFuB3OE53Ma1aTQ@mail.gmail.com>

On Wed, Oct 3, 2012 at 8:58 PM, Brian Curtin <brian at python.org> wrote:
> This doesn't answer the question of the users wanting the alphas
> earlier, but they're certainly more than largely ignored...
>
> The webstats in April 2012 show 5628 downloads of 3.3a1 and 4946
> downloads of 3.3a2 Windows installers.

So, there's 5000 people that could provide advance feedback that I
would then have a few months to act on? Sounds useful to me.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ericsnowcurrently at gmail.com  Wed Oct  3 17:31:23 2012
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Wed, 3 Oct 2012 09:31:23 -0600
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
Message-ID: <CALFfu7DNQpWNPbH6mWPxHRH+bxp+WgFUz02j=1UdEZsZz3KGDw@mail.gmail.com>

On Wed, Oct 3, 2012 at 8:55 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> I *can't* effectively trial
> those changes on PyPI (except perhaps some of the disassembly
> changes), and I don't have the resources to create and distribute
> Windows and Mac OS X installers on my own. That means, before the
> release of 3.4a1, any feedback on most of these changes will be
> limited to those developers with the wherewithal to build Python from
> source.

Would it help to have automated nightly builds that include those
installers (if that's even feasible)?  I'm not opposed to an earlier
alpha as well, but some of the concern you have would be additionally
addressed by such nightly builds.

-eric

From ncoghlan at gmail.com  Wed Oct  3 17:31:32 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 3 Oct 2012 21:01:32 +0530
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CAK5idxTt9_BMyU8t9QaXW7eF8x2d9wbP-66a8vOeYuaFVPECwg@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
	<CAK5idxTt9_BMyU8t9QaXW7eF8x2d9wbP-66a8vOeYuaFVPECwg@mail.gmail.com>
Message-ID: <CADiSq7eK3qqQ7YM6u3RpduN=2z3hHqsfh_Nv5qJi6x2VxAOCmg@mail.gmail.com>

On Wed, Oct 3, 2012 at 8:59 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Wed, Oct 3, 2012 at 5:27 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Wed, Oct 3, 2012 at 8:43 PM, Larry Hastings <larry at hastings.org> wrote:
>>> Begging for feedback doesn't mean you'll get any,
>>
>> I received a fair number of complaints from people that wanted to
>> experiment with yield from, but couldn't, because the first alpha
>> wasn't out yet and they weren't sufficiently interested to go to the
>> effort of building their own copy of Python.
>>
>
> How about having nightly builds then?

Nightly builds are a lot of work to setup and keep running smoothly,
and the rate of change isn't high enough to justify them. I proposed
to Larry originally that we start the alphas 6 months after 3.3.0, but
he didn't like that idea, so I since reduced it to simply doing alpha
1 early so that it din't mean any more work for the release team.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From steve at pearwood.info  Wed Oct  3 17:34:33 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 04 Oct 2012 01:34:33 +1000
Subject: [Python-Dev] Should vars() return modifiable dict?
In-Reply-To: <k4gug7$e1q$1@ger.gmane.org>
References: <k4gug7$e1q$1@ger.gmane.org>
Message-ID: <506C5B09.60005@pearwood.info>

On 03/10/12 18:54, Serhiy Storchaka wrote:

> For locals vars() returns... hmm, partially modifiable dict:
>
>> >> def f():
> ... x = 42
> ... print(vars())
> ... vars()['x'] = 43
> ... vars()['y'] = 44
> ... print(x, vars())
> ...
>> >> f()
> {'x': 42}
> 42 {'y': 44, 'x': 42}
>
> Should behavior of vars() be corrected for locals?

I believe you are misinterpreting what you are seeing. In this case,
vars() simply returns locals(), which is an ordinary dict, but changes
to that dict are not guaranteed to propagate to the actual local
variables themselves. You make changes to that dict, then call vars()
again, which returns a fresh locals() dict. So what you are seeing is
simply a side-effect of the fact that changes to locals() may not
actually effect the local variables.

Note that in IronPython, the behaviour of your code is different:


steve at runes:~$ ipy
IronPython 2.6 Beta 2 DEBUG (2.6.0.20) on .NET 2.0.50727.1433
Type "help", "copyright", "credits" or "license" for more information.
>>> def f():
...     x = 42
...     print(vars())
...     vars()['x'] = 43
...     vars()['y'] = 44
...     print(x, vars())
...
>>> f()
{'x': 42}
(43, {'y': 44, 'x': 43})



> Should vars() for objects with __slots__ [1] returns modifiable or non-modifiable dict?
>
> [1] http://bugs.python.org/issue13290


You are assuming that the behaviour of vars(obj) should change. I don't think
that is necessarily the case.

vars(obj) is defined as returning the object __dict__ attribute. If an object
has no __dict__, vars() should (pick one):

1) Keep the current behaviour and raise an exception.

2) Return a regular dict containing {slot: value} for each of the slots.
    Since the dict is a copy, changes to the dict will not effect the
    original object.

3) Return a dictproxy containing {slot: value} for each of the slots.
    Since the dictproxy does not support item assignment, you can't
    modify it.

4) Return some other proxy object such that changes to the dict will
    also change the object's slot attributes.


I find myself unable to choose between 2) and 4), which suggests that
the status quo wins and we keep the current behaviour.



-- 
Steven

From p.f.moore at gmail.com  Wed Oct  3 17:40:09 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 3 Oct 2012 16:40:09 +0100
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C560E.80306@hastings.org>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
Message-ID: <CACac1F8DQxpEe49p1yLdG_kMQ43251JDLxmRmBbPdxyzwmKNJg@mail.gmail.com>

On 3 October 2012 16:13, Larry Hastings <larry at hastings.org> wrote:
> On 10/03/2012 04:55 PM, Nick Coghlan wrote:
>
> Regardless of when the first alpha happens, I'll be promoting the hell
> out of it, begging for feedback on any of these changes that are
> available by then (which should be quite a few, given the preceding
> PyCon US sprints).
>
>
> If you can show me people who use the alphas who want them earlier, I'll
> consider it.  So far the only person who's said they want them is you, and
> IIUC you won't be a consumer of the alpha per se.
>
> Begging for feedback doesn't mean you'll get any,

FWIW, I *will* be downloading and installing the 3.4 alphas as soon as
they come out. How much I'll *use* them depends on what issues I find
with them - but that's the whole point, I'll feed back my experiences.

I will not build from source, or download nightly builds routinely.
Doing so implies a commitment to track the latest build that I can't
offer.

Paul.

From barry at python.org  Wed Oct  3 18:01:37 2012
From: barry at python.org (Barry Warsaw)
Date: Wed, 3 Oct 2012 12:01:37 -0400
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <20121003152210.5A4A025010D@webabinitio.net>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<20121003152210.5A4A025010D@webabinitio.net>
Message-ID: <20121003120137.48a3eb0d@limelight.wooz.org>

On Oct 03, 2012, at 11:22 AM, R. David Murray wrote:

>I don't have any data to back this up, but it is my impression that more
>distributions are providing access to alpha releases in their "testing"
>package trees.

Ubuntu and Debian generally does, thanks to Matthias's great work.  Python
3.3's been available (though obviously not the default) in Ubuntu 12.10 for a
while now, and in Debian experimental, and we have started to get package
build failure reports related to it:

http://bugs.debian.org/cgi-bin/pkgreport.cgi?users=debian-python at lists.debian.org;tag=python3.3

It's not that much yet though, and I'm not sure if having the alphas available
really helps with that, but it *is* nice to be able to make alphas available
to our users via experimental, PPAs, etc. so that folks can play with it much
more easily.

>As Python continues to grow in importance[1], the number of people
>interacting with Python on the distribution development teams[2] increases,
>and therefor the number of people likely to run alphas for testing increases.
>So even if Larry were right *now*, he isn't right for the future, and we
>should do all we can to nurture an increasing culture of alpha-testing.

Definitely.

>[1] in case anyone hasn't noticed, we *are* a growing community, regardless
>    of where various analytics put us relative to other languages :) 

Which is *fantastic*!

Cheers,
-Barry

From larry at hastings.org  Wed Oct  3 18:02:03 2012
From: larry at hastings.org (Larry Hastings)
Date: Wed, 03 Oct 2012 18:02:03 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
Message-ID: <506C617B.3020801@hastings.org>


On 10/03/2012 05:27 PM, Nick Coghlan wrote:
> That's your prerogative as RM of course, but you haven't given any
> reason beyond the circular "I don't care about enabling feedback from
> people that can't or won't build from source, because people that
> can't or won't build from source don't provide useful feedback".

That's not quite what I said.  I simply said that the alphas are largely 
ignored.

My perspective is that most people ignore the alphas and only get 
interested when the software "solidifies".  Having more alphas, or 
having earlier alphas, doesn't mean we'll have more people interested in 
the alphas.

Adding more alphas (as you have previously suggested) adds additional 
workload on myself and the rest of the release team--I don't know what 
the Mac build is like, but I know Martin has to intercede manually to 
coax a Windows installer out of the build. I'm loathe to burn their time 
for what I perceive as little return.

Changing an existing alpha to be earlier doesn't alter the workload, but 
I fear it makes the alpha less relevant.  Evaluating alphas / betas 
takes an investment of time, and whether or not a potential alpha user 
makes that investment depends on what they expect to get out of testing 
the alpha.  If they're doing it out of the goodness of their hearts, 
just to help Python--well, that's fabulous, and more / earlier alphas 
might actually interest them.  But my suspicion is that most people who 
try the alphas are doing early integration testing with their own 
stuff.  For those people, the earlier the alpha, the less interesting it 
probably is to them. Earlier means that the software will be less 
finished.  It will be buggier, it won't have as many features as the 
beta will.  As a result it won't be as revealing--or as relevant--as a 
later alpha or even a beta.  If that's their perspective, I suspect 
they'll be less likely to try an earlier alpha.

In short, I see the scheduling of alphas as a tradeoff between "early 
enough that we'll have time to fix things" and "late enough that the 
software is reasonably complete".

What it really comes down to: I'm a first-time RM, and I lack the 
courage/wisdom to overrule what appears to be a reasonable status quo.  
I feel I don't have to defend the decision to maintain the status quo; I 
feel instead you have to make a case for changing it. So far all I 
recall seeing from you are assertions.  I'd like to see some harder data.


On 10/03/2012 05:28 PM, Brian Curtin wrote:
> The webstats in April 2012 show 5628 downloads of 3.3a1 and 4946
> downloads of 3.3a2 Windows installers.

I'd love to know how much feedback we got as a result of these 
downloads.  Do we have any way of telling?

And out of curiosity, how many 3.3.0 final Windows installers have been 
downloaded so far?


On 10/03/2012 05:29 PM, Maciej Fijalkowski wrote:
> How about having nightly builds then?

We simply don't have a fully-automated process to produce installers for 
all platforms.  In fact, I fear we don't have a fully-automated process 
to produce installers for /any/ platforms.


On 10/03/2012 05:40 PM, Paul Moore wrote:
> FWIW, I *will* be downloading and installing the 3.4 alphas as soon as
> they come out. How much I'll *use* them depends on what issues I find
> with them - but that's the whole point, I'll feed back my experiences.

Excellent, thanks for the data!  As long as you're volunteering 
interesting data ;-), what platform will you be testing on?  And what is 
your purpose in downloading the alphas--just Good Samaritan-ism, or 
integration testing with software you care about, or what?


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121003/5a47fe1b/attachment.html>

From brian at python.org  Wed Oct  3 18:19:57 2012
From: brian at python.org (Brian Curtin)
Date: Wed, 3 Oct 2012 11:19:57 -0500
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C617B.3020801@hastings.org>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
	<506C617B.3020801@hastings.org>
Message-ID: <CAD+XWwqJmA_pOPThUApd-wWNTLUE3tHp8kKkwT1e5Y3RiYGG6Q@mail.gmail.com>

On Wed, Oct 3, 2012 at 11:02 AM, Larry Hastings <larry at hastings.org> wrote:
>
> On 10/03/2012 05:28 PM, Brian Curtin wrote:
>
> The webstats in April 2012 show 5628 downloads of 3.3a1 and 4946
> downloads of 3.3a2 Windows installers.
>
>
> I'd love to know how much feedback we got as a result of these downloads.
> Do we have any way of telling?

Not really, but I guess we could query the tracker for the time frame
the alphas were fresh and poke around.

> And out of curiosity, how many 3.3.0 final Windows installers have been
> downloaded so far?

27,947 32-bit and 15,816 64-bit installers downloaded in
September...which is fun considering that was across the last two days
of the month, on a weekend.

So far in October we're at 32,449 32-bit and 22,221 64-bit downloads.

From stephen at xemacs.org  Wed Oct  3 18:23:09 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 04 Oct 2012 01:23:09 +0900
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CADiSq7eK3qqQ7YM6u3RpduN=2z3hHqsfh_Nv5qJi6x2VxAOCmg@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
	<CAK5idxTt9_BMyU8t9QaXW7eF8x2d9wbP-66a8vOeYuaFVPECwg@mail.gmail.com>
	<CADiSq7eK3qqQ7YM6u3RpduN=2z3hHqsfh_Nv5qJi6x2VxAOCmg@mail.gmail.com>
Message-ID: <878vbnle36.fsf@uwakimon.sk.tsukuba.ac.jp>

Nick Coghlan writes:

 > I proposed to Larry originally that we start the alphas 6 months
 > after 3.3.0,

I like this idea better, even though to keep the total workload
constant you'd need to more than double the interval between alphas.
Still, a 10- or 12-month release schedule feels like a much bigger
commitment to me (and I have managed a release that took 6 months, so
I do know what that feels like for real).

 > but he didn't like that idea, so I since reduced it to simply doing
 > alpha 1 early so that it din't mean any more work for the release
 > team.

Heck, if you're only going to have *one* early alpha, you could do it
yourself, and see if you can persuade the installer builders to
volunteer one extra build.

From rdmurray at bitdance.com  Wed Oct  3 18:26:49 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 03 Oct 2012 12:26:49 -0400
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C617B.3020801@hastings.org>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
	<506C617B.3020801@hastings.org>
Message-ID: <20121003162650.257B025016D@webabinitio.net>

On Wed, 03 Oct 2012 18:02:03 +0200, Larry Hastings <larry at hastings.org> wrote:
> Changing an existing alpha to be earlier doesn't alter the workload, but 
> I fear it makes the alpha less relevant.  Evaluating alphas / betas 
> takes an investment of time, and whether or not a potential alpha user 
> makes that investment depends on what they expect to get out of testing 
> the alpha.  If they're doing it out of the goodness of their hearts, 
> just to help Python--well, that's fabulous, and more / earlier alphas 
> might actually interest them.  But my suspicion is that most people who 
> try the alphas are doing early integration testing with their own 
> stuff.  For those people, the earlier the alpha, the less interesting it 
> probably is to them. Earlier means that the software will be less 
> finished.  It will be buggier, it won't have as many features as the 
> beta will.  As a result it won't be as revealing--or as relevant--as a 
> later alpha or even a beta.  If that's their perspective, I suspect 
> they'll be less likely to try an earlier alpha.

In my perception (again, I can't point to any raw data) there is an
opposite dynamic that happens in stable projects that produce alphas
throughout the release cycle.  In those projects, people will often
run the alphas, even in production[1], in order to get the new features
sooner.

Python is a stable project.  Even our alphas do not as a rule brown bag,
though their suitability for specific projects depends on the bugs found.

I think Python *could* be in this camp, if we wanted to be.

(I'm not addressing release-team load here, I'm just making an
observation.)

--David

[1] I am *not* advocating running an alpha in production, but for some
    projects "production" is a small enough audience that they can get
    away with it.)

From rdmurray at bitdance.com  Wed Oct  3 18:34:27 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 03 Oct 2012 12:34:27 -0400
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CAD+XWwqJmA_pOPThUApd-wWNTLUE3tHp8kKkwT1e5Y3RiYGG6Q@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
	<506C617B.3020801@hastings.org>
	<CAD+XWwqJmA_pOPThUApd-wWNTLUE3tHp8kKkwT1e5Y3RiYGG6Q@mail.gmail.com>
Message-ID: <20121003163428.16DA325016D@webabinitio.net>

On Wed, 03 Oct 2012 11:19:57 -0500, Brian Curtin <brian at python.org> wrote:
> On Wed, Oct 3, 2012 at 11:02 AM, Larry Hastings <larry at hastings.org> wrote:
> >
> > On 10/03/2012 05:28 PM, Brian Curtin wrote:
> >
> > The webstats in April 2012 show 5628 downloads of 3.3a1 and 4946
> > downloads of 3.3a2 Windows installers.
> >
> >
> > I'd love to know how much feedback we got as a result of these downloads.
> > Do we have any way of telling?
> 
> Not really, but I guess we could query the tracker for the time frame
> the alphas were fresh and poke around.

There *were* bug reports during the alpha phase.  A number of regressions
were caught.  Also, there were more alpha-phase bug reports than
I remember getting for 3.2.  I remember thinking, "wow, cool, we're
actually getting regression bug reports during the alpha phase, people
must actually be testing this time".

--David

From ethan at stoneleaf.us  Wed Oct  3 17:38:57 2012
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 03 Oct 2012 08:38:57 -0700
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CADiSq7eoBBv+nxOboZ-hi7qqgUX1mn_5c2dNFuB3OE53Ma1aTQ@mail.gmail.com>
References: <506C2129.5010908@hastings.org>	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>	<506C25A4.5090900@hastings.org>	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>	<506C560E.80306@hastings.org>	<CAD+XWwpF91QvUUAd7XPwuN5d0YQEZHWg9WPF5o2tZWggaGP3WA@mail.gmail.com>
	<CADiSq7eoBBv+nxOboZ-hi7qqgUX1mn_5c2dNFuB3OE53Ma1aTQ@mail.gmail.com>
Message-ID: <506C5C11.1040502@stoneleaf.us>

Nick Coghlan wrote:
> On Wed, Oct 3, 2012 at 8:58 PM, Brian Curtin <brian at python.org> wrote:
>> This doesn't answer the question of the users wanting the alphas
>> earlier, but they're certainly more than largely ignored...
>>
>> The webstats in April 2012 show 5628 downloads of 3.3a1 and 4946
>> downloads of 3.3a2 Windows installers.
> 
> So, there's 5000 people that could provide advance feedback that I
> would then have a few months to act on? Sounds useful to me.

I have to agree.

+1 for an early alpha.  Call it alpha 0 if you like.  ;)

~Ethan~

From p.f.moore at gmail.com  Wed Oct  3 19:02:07 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 3 Oct 2012 18:02:07 +0100
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <20121003163428.16DA325016D@webabinitio.net>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
	<506C617B.3020801@hastings.org>
	<CAD+XWwqJmA_pOPThUApd-wWNTLUE3tHp8kKkwT1e5Y3RiYGG6Q@mail.gmail.com>
	<20121003163428.16DA325016D@webabinitio.net>
Message-ID: <CACac1F8FczyFp6eZaqnu8Rg3tBOj+XFLY6+Dg0fJf4qjN2PAvg@mail.gmail.com>

On 3 October 2012 17:34, R. David Murray <rdmurray at bitdance.com> wrote:
> There *were* bug reports during the alpha phase.  A number of regressions
> were caught.  Also, there were more alpha-phase bug reports than
> I remember getting for 3.2.  I remember thinking, "wow, cool, we're
> actually getting regression bug reports during the alpha phase, people
> must actually be testing this time".

One possibly-relevant point is that I caught some bugs in packaging
that could only be properly investigated from an installer-based
build. The source build layout on Windows is different, and it
mattered to packaging. (Arguably, that itself was a bug in packaging,
and it shouldn't have mattered, but nevertheless). As far as I can
tell, it is *not* possible (in any practical sense) to create a
production-layout install on Windows without building the MSIs (which
definitely takes installing extra tools and things to complete - more
than I ever managed to do successfully).

Paul.

From storchaka at gmail.com  Wed Oct  3 19:00:17 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Wed, 03 Oct 2012 20:00:17 +0300
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C617B.3020801@hastings.org>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
	<506C617B.3020801@hastings.org>
Message-ID: <k4hqv8$fv8$1@ger.gmane.org>

On 03.10.12 19:02, Larry Hastings wrote:
> But my suspicion is that most people who
> try the alphas are doing early integration testing with their own
> stuff.  For those people, the earlier the alpha, the less interesting it
> probably is to them. Earlier means that the software will be less
> finished.  It will be buggier, it won't have as many features as the
> beta will.  As a result it won't be as revealing--or as relevant--as a
> later alpha or even a beta.  If that's their perspective, I suspect
> they'll be less likely to try an earlier alpha.

I wholeheartedly agree with Larry. Personally I looked for the first 
time near Python 3.3 after release of the first alpha (before a lot of 
years followed the development from a distance), but if alpha came out 
much earlier and would be less mature, I would have probably ignored it. 
Of course, this is only my personal case, I can't speak for other people.



From g.brandl at gmx.net  Wed Oct  3 19:02:22 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 03 Oct 2012 19:02:22 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <20121003162650.257B025016D@webabinitio.net>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
	<506C617B.3020801@hastings.org>
	<20121003162650.257B025016D@webabinitio.net>
Message-ID: <k4hr2t$gvc$1@ger.gmane.org>

On 10/03/2012 06:26 PM, R. David Murray wrote:
> On Wed, 03 Oct 2012 18:02:03 +0200, Larry Hastings <larry at hastings.org> wrote:
>> Changing an existing alpha to be earlier doesn't alter the workload, but
>> I fear it makes the alpha less relevant.  Evaluating alphas / betas
>> takes an investment of time, and whether or not a potential alpha user
>> makes that investment depends on what they expect to get out of testing
>> the alpha.  If they're doing it out of the goodness of their hearts,
>> just to help Python--well, that's fabulous, and more / earlier alphas
>> might actually interest them.  But my suspicion is that most people who
>> try the alphas are doing early integration testing with their own
>> stuff.  For those people, the earlier the alpha, the less interesting it
>> probably is to them. Earlier means that the software will be less
>> finished.  It will be buggier, it won't have as many features as the
>> beta will.  As a result it won't be as revealing--or as relevant--as a
>> later alpha or even a beta.  If that's their perspective, I suspect
>> they'll be less likely to try an earlier alpha.
>
> In my perception (again, I can't point to any raw data) there is an
> opposite dynamic that happens in stable projects that produce alphas
> throughout the release cycle.  In those projects, people will often
> run the alphas, even in production[1], in order to get the new features
> sooner.
>
> Python is a stable project.  Even our alphas do not as a rule brown bag,
> though their suitability for specific projects depends on the bugs found.
>
> I think Python *could* be in this camp, if we wanted to be.

I agree.

> (I'm not addressing release-team load here, I'm just making an
> observation.)

As for release-team load: after the first alpha is done the subsequent ones
are quite lightweight from the main RM perspective.  Regarding the Mac/Windows
builds, there's no law saying that only one person is allowed to make binaries
for a given version.  We should by now have at least two for each platform;
Ned and Ronald for Mac, and AFAIK Brian Curtin has volunteered to fill in for
Martin in emergencies anyway.

As soon as betas arrive, of course it makes sense for the release team to stay
the same to ensure a smooth road to final.

Georg


From python at mrabarnett.plus.com  Wed Oct  3 19:10:48 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 03 Oct 2012 18:10:48 +0100
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CAD+XWwpF91QvUUAd7XPwuN5d0YQEZHWg9WPF5o2tZWggaGP3WA@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CAD+XWwpF91QvUUAd7XPwuN5d0YQEZHWg9WPF5o2tZWggaGP3WA@mail.gmail.com>
Message-ID: <506C7198.40600@mrabarnett.plus.com>

On 2012-10-03 16:28, Brian Curtin wrote:
> On Wed, Oct 3, 2012 at 10:13 AM, Larry Hastings <larry at hastings.org> wrote:
>> On 10/03/2012 04:55 PM, Nick Coghlan wrote:
>>
>> Regardless of when the first alpha happens, I'll be promoting the hell
>> out of it, begging for feedback on any of these changes that are
>> available by then (which should be quite a few, given the preceding
>> PyCon US sprints).
>>
>>
>> If you can show me people who use the alphas who want them earlier, I'll
>> consider it.  So far the only person who's said they want them is you, and
>> IIUC you won't be a consumer of the alpha per se.
>>
>> Begging for feedback doesn't mean you'll get any,
>
> This doesn't answer the question of the users wanting the alphas
> earlier, but they're certainly more than largely ignored...
>
> The webstats in April 2012 show 5628 downloads of 3.3a1 and 4946
> downloads of 3.3a2 Windows installers.
>
I downloaded the alphas to test the support for PEP 393 I'd added to
the regex module.

Not having to build Python 3.3 from source certainly helped _me_! :-)

From solipsis at pitrou.net  Wed Oct  3 20:05:24 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 3 Oct 2012 20:05:24 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
Message-ID: <20121003200524.52b5245d@pitrou.net>

On Wed, 3 Oct 2012 20:25:17 +0530
Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Wed, Oct 3, 2012 at 5:16 PM, Larry Hastings <larry at hastings.org> wrote:
> > I don't agree.  It's my understanding that the alphas are largely ignored,
> > and having them earlier would hardly make them more relevant.
> 
> I would appreciate it you stopped promoting this myth. Each step in
> the release process widens the pool of people providing feedback.

Well, if you look at the download statistics at
http://www.python.org/webstats/ or even at the bug influx when
publishing an alpha, it seems clear to me that alphas have almost no
impact.

Only rcs seem to gather significant interest from the broader community.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From solipsis at pitrou.net  Wed Oct  3 20:10:53 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 3 Oct 2012 20:10:53 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<20121003200524.52b5245d@pitrou.net>
Message-ID: <20121003201053.3c986fb6@pitrou.net>

On Wed, 3 Oct 2012 20:05:24 +0200
Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Wed, 3 Oct 2012 20:25:17 +0530
> Nick Coghlan <ncoghlan at gmail.com> wrote:
> > On Wed, Oct 3, 2012 at 5:16 PM, Larry Hastings <larry at hastings.org> wrote:
> > > I don't agree.  It's my understanding that the alphas are largely ignored,
> > > and having them earlier would hardly make them more relevant.
> > 
> > I would appreciate it you stopped promoting this myth. Each step in
> > the release process widens the pool of people providing feedback.
> 
> Well, if you look at the download statistics at
> http://www.python.org/webstats/ or even at the bug influx when
> publishing an alpha, it seems clear to me that alphas have almost no
> impact.

Erm, I hadn't seen Brian's reply before sending this. Apparently there
*are* people who download the alphas, which for some reason I had
overlooked in the stats pages. My mistake, sorry.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From martin at v.loewis.de  Wed Oct  3 20:23:00 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Wed, 03 Oct 2012 20:23:00 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C2129.5010908@hastings.org>
References: <506C2129.5010908@hastings.org>
Message-ID: <20121003202300.Horde.phYXIKGZi1VQbIKE23BhX5A@webmail.df.eu>


Zitat von Larry Hastings <larry at hastings.org>:

> I welcome your feedback--without any further input I'll go ahead and  
> post this as a PEP in a week or so.

I personally feel that the release period is too long. Is it really necessary
to start with the release more than 6 month before the scheduled release date?
Starting with alpha 3 (i.e. September 2013) seems sufficient to me.

Regards,
Martin



From martin at v.loewis.de  Wed Oct  3 20:28:16 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Wed, 03 Oct 2012 20:28:16 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>
Message-ID: <20121003202816.Horde.DgNRbqGZi1VQbIPA6nzxbcA@webmail.df.eu>


Zitat von Skip Montanaro <skip at pobox.com>:

>> I've roughed out a release schedule....
>
> Is there a rough list of changes for 3.4 written down somewhere, or is
> that only to be inferred based on PEPs whose Python-Version header
> reads "3.4"?  How confident are you that the schedule you've proposed
> gives enough time for proposed changes to be implemented and to
> stablize?

I think it's really vice-versa. We traditionally follow a timed
process: we release when enough time has passed, and include features
which are ready by the time we release.

Regards,
Martin




From martin at v.loewis.de  Wed Oct  3 20:34:58 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Wed, 03 Oct 2012 20:34:58 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CAK5idxTt9_BMyU8t9QaXW7eF8x2d9wbP-66a8vOeYuaFVPECwg@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
	<CAK5idxTt9_BMyU8t9QaXW7eF8x2d9wbP-66a8vOeYuaFVPECwg@mail.gmail.com>
Message-ID: <20121003203458.Horde.spLiTaGZi1VQbIVSFPYxhgA@webmail.df.eu>


Zitat von Maciej Fijalkowski <fijall at gmail.com>:

> On Wed, Oct 3, 2012 at 5:27 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Wed, Oct 3, 2012 at 8:43 PM, Larry Hastings <larry at hastings.org> wrote:
>>> Begging for feedback doesn't mean you'll get any,
>>
>> I received a fair number of complaints from people that wanted to
>> experiment with yield from, but couldn't, because the first alpha
>> wasn't out yet and they weren't sufficiently interested to go to the
>> effort of building their own copy of Python.
>>
>
> How about having nightly builds then?

We actually do have nightly builds.

Regards,
Martin



From martin at v.loewis.de  Wed Oct  3 20:38:49 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Wed, 03 Oct 2012 20:38:49 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CALFfu7DNQpWNPbH6mWPxHRH+bxp+WgFUz02j=1UdEZsZz3KGDw@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<CALFfu7DNQpWNPbH6mWPxHRH+bxp+WgFUz02j=1UdEZsZz3KGDw@mail.gmail.com>
Message-ID: <20121003203849.Horde.AwjFOqGZi1VQbIY5blExjXA@webmail.df.eu>


Zitat von Eric Snow <ericsnowcurrently at gmail.com>:

> On Wed, Oct 3, 2012 at 8:55 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> I *can't* effectively trial
>> those changes on PyPI (except perhaps some of the disassembly
>> changes), and I don't have the resources to create and distribute
>> Windows and Mac OS X installers on my own. That means, before the
>> release of 3.4a1, any feedback on most of these changes will be
>> limited to those developers with the wherewithal to build Python from
>> source.
>
> Would it help to have automated nightly builds that include those
> installers (if that's even feasible)?

We once had nightly builds of the Windows installers. It required a
dedicated buildbot operator, since the process tended to break.

We still do have OSX nightly installers:

http://buildbot.python.org/daily-dmg/

Regards,
Martin



From martin at v.loewis.de  Wed Oct  3 20:33:28 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Wed, 03 Oct 2012 20:33:28 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
Message-ID: <20121003203328.Horde.Za5_TqGZi1VQbIT43MHBgoA@webmail.df.eu>


Zitat von Nick Coghlan <ncoghlan at gmail.com>:

> Regardless of when the first alpha happens, I'll be promoting the hell
> out of it, begging for feedback on any of these changes that are
> available by then (which should be quite a few, given the preceding
> PyCon US sprints). However, I would *like* to have months rather than
> weeks to act on any feedback we do receive. I'm not asking the release
> team to do any more work - I'm just asking for a chunk of it to be
> brought forward a few months. If I was asking for an *extra* release,
> I could understand resistance to the idea, but what's the concrete
> benefit of *delaying* the first alpha release by 4 months from when
> I'm hoping to see it happen?

I wouldn't mind having alpha 1 in April 2013, and alpha 2 in October 2013.
I share Larry's skepticism, and actually fear that it may confuse users
(which find that they test something completely different from what gets
released). However, I don't mind supporting this strategy anyway as an
experiment.

Regards,
Martin



From g.rodola at gmail.com  Thu Oct  4 03:14:11 2012
From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=)
Date: Thu, 4 Oct 2012 03:14:11 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C2768.4010408@hastings.org>
References: <506C2129.5010908@hastings.org>
	<CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>
	<506C2768.4010408@hastings.org>
Message-ID: <CAFYqXL8x=bsRoEBOitLz-=tMO-Tf9Juq075GTP8XYP9ZB1jhDw@mail.gmail.com>

2012/10/3 Larry Hastings <larry at hastings.org>:

> Other proposed large-scale changes:
> [...]
> * A standard event-loop interface (PEP by Jim Fulton pending)

Really? Was this discussed somewhere? I'd like to know more about it.

--- Giampaolo
http://code.google.com/p/pyftpdlib/
http://code.google.com/p/psutil/
http://code.google.com/p/pysendfile/

From rdmurray at bitdance.com  Thu Oct  4 03:38:27 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 03 Oct 2012 21:38:27 -0400
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CAFYqXL8x=bsRoEBOitLz-=tMO-Tf9Juq075GTP8XYP9ZB1jhDw@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>
	<506C2768.4010408@hastings.org>
	<CAFYqXL8x=bsRoEBOitLz-=tMO-Tf9Juq075GTP8XYP9ZB1jhDw@mail.gmail.com>
Message-ID: <20121004013827.F07742500FB@webabinitio.net>


On Thu, 04 Oct 2012 03:14:11 +0200, =?ISO-8859-1?Q?Giampaolo_Rodol=E0?= <g.rodola at gmail.com> wrote:
> 2012/10/3 Larry Hastings <larry at hastings.org>:
> 
> > Other proposed large-scale changes:
> > [...]
> > * A standard event-loop interface (PEP by Jim Fulton pending)
> 
> Really? Was this discussed somewhere? I'd like to know more about it.

I believe it was discussed at the Language Summit at the last Pycon.
As I recall there was at least one other person interested in helping
with it, but I don't remember who.

--David

From stephen at xemacs.org  Thu Oct  4 03:52:39 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 04 Oct 2012 10:52:39 +0900
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <20121003203328.Horde.Za5_TqGZi1VQbIT43MHBgoA@webmail.df.eu>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<20121003203328.Horde.Za5_TqGZi1VQbIT43MHBgoA@webmail.df.eu>
Message-ID: <87626rknq0.fsf@uwakimon.sk.tsukuba.ac.jp>

martin at v.loewis.de writes:

 > I wouldn't mind having alpha 1 in April 2013, and alpha 2 in October 2013.
 > I share Larry's skepticism, and actually fear that it may confuse users
 > (which find that they test something completely different from what gets
 > released).

I don't really think you need to worry.  The Mallory-style testers[1]
who download early alphas generally don't have specific expectations
that it will correspond to the eventual release in my experience.

I have little experience with testers who download alphas to get a
head start on integration work, but I suppose Larry is right, you'll
lose a few of those if an alpha is "too early".  I suspect that those
folks mostly listen to an internal clock, not so much to release
announcements, and download the freshest release when they are ready
to start that kind of work.  Those who decide *not* to download alpha
1 because "it's too old/incomplete, I'll wait for alpha 2" will be a
subset, I suspect a minority, of those ready to get an alpha for
integration testing at any given time.  OTOH, is it really that big a
loss?  If I'm right, they *will* get alpha 2.

In fact, you might find a larger than usual response from those folks
for alpha 1 because Nick will be out advertising "these changes are
big, guys, both you and we want testing early!"  Except that unlike
most advertising, Nick's pretty plausible.<wink/>  At least, that's
Nick's main point for doing this experiment *now* as I understand it.
(I gather he agrees with David about a trend to "continuous alpha",
but that's a separate though related issue.)

Footnotes: 
[1]  "Why did you download the alpha?"  "Because it's there."


From ideasman42 at gmail.com  Thu Oct  4 05:35:25 2012
From: ideasman42 at gmail.com (Campbell Barton)
Date: Thu, 4 Oct 2012 13:35:25 +1000
Subject: [Python-Dev] Using environment variables problematic for embedded
	python.
Message-ID: <CAEcf3Nyeqg85e3Gebg6+O72=zsFRGTw_CJtU9=rO3ZD-qEa01Q@mail.gmail.com>

Hi,

We've run into an issue recently with blender3d on ms-windows where we
want to enforce the encoding is UTF-8 with the embedded python
interpreter.
(the encoding defaults to cp437).

I naively thought setting the environment variable before calling
Py_Initialize() would work, but the way python DLL loads, it gets its
own environment variables that cant be modified directly [1].
eg, _putenv("PYTHONIOENCODING=utf-8:surrogateescape");

We had bug reports by windows users not able to export files because
the stdout errors on printing paths with unsupported encoding. [2],[3]

---

Of course we could distribute blender with a bat file launcher that
sets env variables, or ask the user to set their env variable - but I
dont think this is really a good option.

I tried overriding the stderr & stdout, but this caused another bug in
a part of out code that catches exception messages from the stderr.
[4]
import sys, io
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8',
errors='surrogateescape', line_buffering=True)
sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8',
errors='surrogateescape', line_buffering=True)



IMHO either of these solutions would be fine.

* have a PyOS_PutEnv() function, gettext has gettext_putenv() to
workaround this problem.

* manage this the same as Py_GetPythonHome(), which can be defined by
the embedding application to override the default.


Id like to know if there is some known solution to workaround this
issue, if not - would either of these would be acceptable in python
(can write up a patch if it helps)

Regards,
Campbell

---

[1] http://stackoverflow.com/questions/5153547/environment-variables-are-different-for-dll-than-exe
[2] http://projects.blender.org/tracker/index.php?func=detail&aid=32750
[3] http://projects.blender.org/tracker/index.php?func=detail&aid=31555
[4] http://projects.blender.org/tracker/?func=detail&aid=32720

From ideasman42 at gmail.com  Thu Oct  4 06:35:02 2012
From: ideasman42 at gmail.com (Campbell Barton)
Date: Thu, 4 Oct 2012 14:35:02 +1000
Subject: [Python-Dev] Using environment variables problematic for
	embedded python.
In-Reply-To: <CAEcf3Nyeqg85e3Gebg6+O72=zsFRGTw_CJtU9=rO3ZD-qEa01Q@mail.gmail.com>
References: <CAEcf3Nyeqg85e3Gebg6+O72=zsFRGTw_CJtU9=rO3ZD-qEa01Q@mail.gmail.com>
Message-ID: <CAEcf3NxNyXZ89KqgkeqiKGBRMOCa52Zuy13dPez5UynSNcduVQ@mail.gmail.com>

On Thu, Oct 4, 2012 at 1:35 PM, Campbell Barton <ideasman42 at gmail.com> wrote:
> Hi,
>
> We've run into an issue recently with blender3d on ms-windows where we
> want to enforce the encoding is UTF-8 with the embedded python
> interpreter.
> (the encoding defaults to cp437).
>
> I naively thought setting the environment variable before calling
> Py_Initialize() would work, but the way python DLL loads, it gets its
> own environment variables that cant be modified directly [1].
> eg, _putenv("PYTHONIOENCODING=utf-8:surrogateescape");
>
> We had bug reports by windows users not able to export files because
> the stdout errors on printing paths with unsupported encoding. [2],[3]
>
> ---
>
> Of course we could distribute blender with a bat file launcher that
> sets env variables, or ask the user to set their env variable - but I
> dont think this is really a good option.
>
> I tried overriding the stderr & stdout, but this caused another bug in
> a part of out code that catches exception messages from the stderr.
> [4]
> import sys, io
> sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8',
> errors='surrogateescape', line_buffering=True)
> sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8',
> errors='surrogateescape', line_buffering=True)
>
>
>
> IMHO either of these solutions would be fine.
>
> * have a PyOS_PutEnv() function, gettext has gettext_putenv() to
> workaround this problem.
>
> * manage this the same as Py_GetPythonHome(), which can be defined by
> the embedding application to override the default.
>
>
> Id like to know if there is some known solution to workaround this
> issue, if not - would either of these would be acceptable in python
> (can write up a patch if it helps)
>
> Regards,
> Campbell
>
> ---
>
> [1] http://stackoverflow.com/questions/5153547/environment-variables-are-different-for-dll-than-exe
> [2] http://projects.blender.org/tracker/index.php?func=detail&aid=32750
> [3] http://projects.blender.org/tracker/index.php?func=detail&aid=31555
> [4] http://projects.blender.org/tracker/?func=detail&aid=32720

To follow up and give a correction to overwriting sys.stdout/stderr,
The issue seemed to be that __stderr__/__stdout__ was later
overwritten, loosing the original reference to the buffer (maybe
refcount issue here?), either way it would silence the output.


import sys, io
sys.__stdout__ = sys.stdout =
io.TextIOWrapper(io.open(sys.stdout.fileno(), "wb", -1),
encoding='utf-8', errors='surrogateescape', newline="\n",
line_buffering=True)
sys.__stderr__ = sys.stderr =
io.TextIOWrapper(io.open(sys.stderr.fileno(), "wb", -1),
encoding='utf-8', errors='surrogateescape', newline="\n",
line_buffering=True)

This all works as expected without bug [4] (above), however on exit I
get an assert in MSVCR90.DLL's write.c (called from python32_d.dll):
_VALIDATE_CLEAR_OSSERR_RETURN((_osfile(fh) & FOPEN), EBADF, -1);

I'd rather not loose more time debugging why this assert happens,
IMHO this is too low-level a way to change the encoing of stdio/stderr
and error-prone too, so some way to reliably set PYTHONIOENCODING from
a program embedding python is still needed.

-- 
- Campbell

From guido at python.org  Thu Oct  4 06:51:18 2012
From: guido at python.org (Guido van Rossum)
Date: Wed, 3 Oct 2012 21:51:18 -0700
Subject: [Python-Dev] Using environment variables problematic for
	embedded python.
In-Reply-To: <CAEcf3NxNyXZ89KqgkeqiKGBRMOCa52Zuy13dPez5UynSNcduVQ@mail.gmail.com>
References: <CAEcf3Nyeqg85e3Gebg6+O72=zsFRGTw_CJtU9=rO3ZD-qEa01Q@mail.gmail.com>
	<CAEcf3NxNyXZ89KqgkeqiKGBRMOCa52Zuy13dPez5UynSNcduVQ@mail.gmail.com>
Message-ID: <CAP7+vJKH_Q_aCCSn99QPtr0eC=DGYOMf4OB_nOBobEuZLz915A@mail.gmail.com>

This seems more fit for the tracker; can you file there? (Then post the
issue link here.) I do think you have a legitimate use case to set the
default encoding to utf-8. (Though there may be a way already.) Does Python
3.3 have te same bug?

On Wednesday, October 3, 2012, Campbell Barton wrote:

> On Thu, Oct 4, 2012 at 1:35 PM, Campbell Barton <ideasman42 at gmail.com<javascript:;>>
> wrote:
> > Hi,
> >
> > We've run into an issue recently with blender3d on ms-windows where we
> > want to enforce the encoding is UTF-8 with the embedded python
> > interpreter.
> > (the encoding defaults to cp437).
> >
> > I naively thought setting the environment variable before calling
> > Py_Initialize() would work, but the way python DLL loads, it gets its
> > own environment variables that cant be modified directly [1].
> > eg, _putenv("PYTHONIOENCODING=utf-8:surrogateescape");
> >
> > We had bug reports by windows users not able to export files because
> > the stdout errors on printing paths with unsupported encoding. [2],[3]
> >
> > ---
> >
> > Of course we could distribute blender with a bat file launcher that
> > sets env variables, or ask the user to set their env variable - but I
> > dont think this is really a good option.
> >
> > I tried overriding the stderr & stdout, but this caused another bug in
> > a part of out code that catches exception messages from the stderr.
> > [4]
> > import sys, io
> > sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8',
> > errors='surrogateescape', line_buffering=True)
> > sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8',
> > errors='surrogateescape', line_buffering=True)
> >
> >
> >
> > IMHO either of these solutions would be fine.
> >
> > * have a PyOS_PutEnv() function, gettext has gettext_putenv() to
> > workaround this problem.
> >
> > * manage this the same as Py_GetPythonHome(), which can be defined by
> > the embedding application to override the default.
> >
> >
> > Id like to know if there is some known solution to workaround this
> > issue, if not - would either of these would be acceptable in python
> > (can write up a patch if it helps)
> >
> > Regards,
> > Campbell
> >
> > ---
> >
> > [1]
> http://stackoverflow.com/questions/5153547/environment-variables-are-different-for-dll-than-exe
> > [2] http://projects.blender.org/tracker/index.php?func=detail&aid=32750
> > [3] http://projects.blender.org/tracker/index.php?func=detail&aid=31555
> > [4] http://projects.blender.org/tracker/?func=detail&aid=32720
>
> To follow up and give a correction to overwriting sys.stdout/stderr,
> The issue seemed to be that __stderr__/__stdout__ was later
> overwritten, loosing the original reference to the buffer (maybe
> refcount issue here?), either way it would silence the output.
>
>
> import sys, io
> sys.__stdout__ = sys.stdout =
> io.TextIOWrapper(io.open(sys.stdout.fileno(), "wb", -1),
> encoding='utf-8', errors='surrogateescape', newline="\n",
> line_buffering=True)
> sys.__stderr__ = sys.stderr =
> io.TextIOWrapper(io.open(sys.stderr.fileno(), "wb", -1),
> encoding='utf-8', errors='surrogateescape', newline="\n",
> line_buffering=True)
>
> This all works as expected without bug [4] (above), however on exit I
> get an assert in MSVCR90.DLL's write.c (called from python32_d.dll):
> _VALIDATE_CLEAR_OSSERR_RETURN((_osfile(fh) & FOPEN), EBADF, -1);
>
> I'd rather not loose more time debugging why this assert happens,
> IMHO this is too low-level a way to change the encoing of stdio/stderr
> and error-prone too, so some way to reliably set PYTHONIOENCODING from
> a program embedding python is still needed.
>
> --
> - Campbell
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org <javascript:;>
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121003/9532a95e/attachment.html>

From hs at ox.cx  Thu Oct  4 08:00:40 2012
From: hs at ox.cx (Hynek Schlawack)
Date: Thu, 4 Oct 2012 08:00:40 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <20121004013827.F07742500FB@webabinitio.net>
References: <506C2129.5010908@hastings.org>
	<CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>
	<506C2768.4010408@hastings.org>
	<CAFYqXL8x=bsRoEBOitLz-=tMO-Tf9Juq075GTP8XYP9ZB1jhDw@mail.gmail.com>
	<20121004013827.F07742500FB@webabinitio.net>
Message-ID: <3E5416D3-10CA-4C2B-A0D1-EED1D49F6C47@ox.cx>


Am 04.10.2012 um 03:38 schrieb R. David Murray <rdmurray at bitdance.com>:

>>> Other proposed large-scale changes:
>>> [...]
>>> * A standard event-loop interface (PEP by Jim Fulton pending)
>> 
>> Really? Was this discussed somewhere? I'd like to know more about it.
> 
> I believe it was discussed at the Language Summit at the last Pycon.
> As I recall there was at least one other person interested in helping
> with it, but I don't remember who.

T?was lvh and I remember doing some cheap spelling checking on it back at EuroPython 20_11_ but the work seems to has stalled 9 months ago.

Adding him to the loop.

From christian at python.org  Thu Oct  4 12:32:39 2012
From: christian at python.org (Christian Heimes)
Date: Thu, 04 Oct 2012 12:32:39 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C2768.4010408@hastings.org>
References: <506C2129.5010908@hastings.org>
	<CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>
	<506C2768.4010408@hastings.org>
Message-ID: <506D65C7.8000306@python.org>

Am 03.10.2012 13:54, schrieb Larry Hastings:
> As for the rest of it, my understanding was that there is no longer any
> great plan written in the stars for Python releases.  Python releases
> are comprised of whatever features people propose, implement, and are
> willing to support, that they can get done in time for the beta cutoff. 
> From that perspective, the schedule drives the features more than the
> other-way around.

I have some crypto and security related features planed for Python 3.4.
I'm planing a PEP on password hashing algorithms (password based key
derivation functions) like pbkdf2, bcrypt and scrypt. The topic was
discussed on python-ideas about half a year ago. We agree that it's
worth to have accelerated C implementations in the core.

Two days ago NIST announced the SHA-3 contest winner. My wrapper of
keccak https://bitbucket.org/tiran/pykeccak/ is almost ready and just
needs some cleanup and more tests. Once it's done I'll remove the Python
3.2 and 2.x compatibility code and integrate it into 3.4.

Christian


From dirkjan at ochtman.nl  Thu Oct  4 12:49:35 2012
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Thu, 4 Oct 2012 12:49:35 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506D65C7.8000306@python.org>
References: <506C2129.5010908@hastings.org>
	<CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>
	<506C2768.4010408@hastings.org> <506D65C7.8000306@python.org>
Message-ID: <CAKmKYaD51Z1dBioeFawpU0MuK_GV2XjsdR=XQ=y1tBhwY3Uxvw@mail.gmail.com>

On Thu, Oct 4, 2012 at 12:32 PM, Christian Heimes <christian at python.org> wrote:
> Two days ago NIST announced the SHA-3 contest winner. My wrapper of
> keccak https://bitbucket.org/tiran/pykeccak/ is almost ready and just
> needs some cleanup and more tests. Once it's done I'll remove the Python
> 3.2 and 2.x compatibility code and integrate it into 3.4.

See also https://github.com/bjornedstrom/python-sha3.

Cheers,

Dirkjan

From christian at python.org  Thu Oct  4 13:12:30 2012
From: christian at python.org (Christian Heimes)
Date: Thu, 04 Oct 2012 13:12:30 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CAKmKYaD51Z1dBioeFawpU0MuK_GV2XjsdR=XQ=y1tBhwY3Uxvw@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>
	<506C2768.4010408@hastings.org> <506D65C7.8000306@python.org>
	<CAKmKYaD51Z1dBioeFawpU0MuK_GV2XjsdR=XQ=y1tBhwY3Uxvw@mail.gmail.com>
Message-ID: <506D6F1E.80005@python.org>

Am 04.10.2012 12:49, schrieb Dirkjan Ochtman:
> On Thu, Oct 4, 2012 at 12:32 PM, Christian Heimes <christian at python.org> wrote:
>> Two days ago NIST announced the SHA-3 contest winner. My wrapper of
>> keccak https://bitbucket.org/tiran/pykeccak/ is almost ready and just
>> needs some cleanup and more tests. Once it's done I'll remove the Python
>> 3.2 and 2.x compatibility code and integrate it into 3.4.
> 
> See also https://github.com/bjornedstrom/python-sha3.

Bj?rn began his implementation about the same time I made my first
experiments. He even ran into the same issue with block_size. I've
compared his code with my code in ticket http://bugs.python.org/issue16113 .

Summary: My code works with Python 3.x (tested with 3.2 and 3.3) and
doesn't require additional Python subclasses. Bj?rns code has better
unit tests as I haven't committed the test suite yet.



From jcea at jcea.es  Thu Oct  4 14:28:47 2012
From: jcea at jcea.es (Jesus Cea)
Date: Thu, 04 Oct 2012 14:28:47 +0200
Subject: [Python-Dev] Bootstrap issue: "select" is compiled "too late"
Message-ID: <506D80FF.2060301@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

<http://buildbot.python.org/all/builders/x86%20OpenIndiana%202.7/builds/1287/steps/compile/logs/stdio>

The relevant part:

"""
ranlib libpython2.7.a
gcc   -o python \
			Modules/python.o \
			libpython2.7.a -lsocket -lnsl -ldl    -lm
Traceback (most recent call last):
  File "./setup.py", line 8, in <module>
    from platform import machine as platform_machine
  File
"/export/home/buildbot/32bits/2.7.cea-indiana-x86/build/Lib/platform.py",
line 116, in <module>
    import sys,string,os,re,subprocess
  File
"/export/home/buildbot/32bits/2.7.cea-indiana-x86/build/Lib/subprocess.py",
line 429, in <module>
    import select
ImportError: No module named select
"""

My patch adds a "select" dependency, and "select" is not compiled yet
at this stage.

Any suggestion about how to solve this?

Thanks!.

- -- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQCVAwUBUG2A/5lgi5GaxT1NAQL0KAQAgTFJc8owtoCAAAQGhhhTqGgrUuUegrM+
PCVZr/IVdnv+tNzAZWl84ng4ZQbfXBzU+6y5XIga0dpifqee9VOVNnLcHL4xv4fP
EWV4jZ9AWN8fPiAbgAe6MGQp3NYlRWTDRoPZTz/zXLqHxzMYzr9LpAOURqXr2/eB
VbYTRAhkGdM=
=m3j8
-----END PGP SIGNATURE-----

From benjamin at python.org  Thu Oct  4 14:34:43 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 4 Oct 2012 08:34:43 -0400
Subject: [Python-Dev] Bootstrap issue: "select" is compiled "too late"
In-Reply-To: <506D80FF.2060301@jcea.es>
References: <506D80FF.2060301@jcea.es>
Message-ID: <CAPZV6o9v_0wU4SRFj_xCfi5fiFWWk-o0MMKYCbadvTJbnOtNaQ@mail.gmail.com>

2012/10/4 Jesus Cea <jcea at jcea.es>:
> Any suggestion about how to solve this?

Easy solutions include somehow removing the dependence on subprocess
or moving the import of subprocess into the function that uses it.



-- 
Regards,
Benjamin

From ideasman42 at gmail.com  Thu Oct  4 15:01:49 2012
From: ideasman42 at gmail.com (Campbell Barton)
Date: Thu, 4 Oct 2012 23:01:49 +1000
Subject: [Python-Dev] Using environment variables problematic for
	embedded python.
In-Reply-To: <CAP7+vJKH_Q_aCCSn99QPtr0eC=DGYOMf4OB_nOBobEuZLz915A@mail.gmail.com>
References: <CAEcf3Nyeqg85e3Gebg6+O72=zsFRGTw_CJtU9=rO3ZD-qEa01Q@mail.gmail.com>
	<CAEcf3NxNyXZ89KqgkeqiKGBRMOCa52Zuy13dPez5UynSNcduVQ@mail.gmail.com>
	<CAP7+vJKH_Q_aCCSn99QPtr0eC=DGYOMf4OB_nOBobEuZLz915A@mail.gmail.com>
Message-ID: <CAEcf3NxYN5u+QdwKAL5Us_ncP3nA9vfq03EEhdLqqu1Nmn9qeA@mail.gmail.com>

Checked Python 3.4 (latest hg checkout) and this area of the code
hasn't changed: pythonrun,c initstdio()

reported the issue:
http://bugs.python.org/issue16129

On Thu, Oct 4, 2012 at 2:51 PM, Guido van Rossum <guido at python.org> wrote:
> This seems more fit for the tracker; can you file there? (Then post the
> issue link here.) I do think you have a legitimate use case to set the
> default encoding to utf-8. (Though there may be a way already.) Does Python
> 3.3 have te same bug?
>
>
> On Wednesday, October 3, 2012, Campbell Barton wrote:
>>
>> On Thu, Oct 4, 2012 at 1:35 PM, Campbell Barton <ideasman42 at gmail.com>
>> wrote:
>> > Hi,
>> >
>> > We've run into an issue recently with blender3d on ms-windows where we
>> > want to enforce the encoding is UTF-8 with the embedded python
>> > interpreter.
>> > (the encoding defaults to cp437).
>> >
>> > I naively thought setting the environment variable before calling
>> > Py_Initialize() would work, but the way python DLL loads, it gets its
>> > own environment variables that cant be modified directly [1].
>> > eg, _putenv("PYTHONIOENCODING=utf-8:surrogateescape");
>> >
>> > We had bug reports by windows users not able to export files because
>> > the stdout errors on printing paths with unsupported encoding. [2],[3]
>> >
>> > ---
>> >
>> > Of course we could distribute blender with a bat file launcher that
>> > sets env variables, or ask the user to set their env variable - but I
>> > dont think this is really a good option.
>> >
>> > I tried overriding the stderr & stdout, but this caused another bug in
>> > a part of out code that catches exception messages from the stderr.
>> > [4]
>> > import sys, io
>> > sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8',
>> > errors='surrogateescape', line_buffering=True)
>> > sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8',
>> > errors='surrogateescape', line_buffering=True)
>> >
>> >
>> >
>> > IMHO either of these solutions would be fine.
>> >
>> > * have a PyOS_PutEnv() function, gettext has gettext_putenv() to
>> > workaround this problem.
>> >
>> > * manage this the same as Py_GetPythonHome(), which can be defined by
>> > the embedding application to override the default.
>> >
>> >
>> > Id like to know if there is some known solution to workaround this
>> > issue, if not - would either of these would be acceptable in python
>> > (can write up a patch if it helps)
>> >
>> > Regards,
>> > Campbell
>> >
>> > ---
>> >
>> > [1]
>> > http://stackoverflow.com/questions/5153547/environment-variables-are-different-for-dll-than-exe
>> > [2] http://projects.blender.org/tracker/index.php?func=detail&aid=32750
>> > [3] http://projects.blender.org/tracker/index.php?func=detail&aid=31555
>> > [4] http://projects.blender.org/tracker/?func=detail&aid=32720
>>
>> To follow up and give a correction to overwriting sys.stdout/stderr,
>> The issue seemed to be that __stderr__/__stdout__ was later
>> overwritten, loosing the original reference to the buffer (maybe
>> refcount issue here?), either way it would silence the output.
>>
>>
>> import sys, io
>> sys.__stdout__ = sys.stdout =
>> io.TextIOWrapper(io.open(sys.stdout.fileno(), "wb", -1),
>> encoding='utf-8', errors='surrogateescape', newline="\n",
>> line_buffering=True)
>> sys.__stderr__ = sys.stderr =
>> io.TextIOWrapper(io.open(sys.stderr.fileno(), "wb", -1),
>> encoding='utf-8', errors='surrogateescape', newline="\n",
>> line_buffering=True)
>>
>> This all works as expected without bug [4] (above), however on exit I
>> get an assert in MSVCR90.DLL's write.c (called from python32_d.dll):
>> _VALIDATE_CLEAR_OSSERR_RETURN((_osfile(fh) & FOPEN), EBADF, -1);
>>
>> I'd rather not loose more time debugging why this assert happens,
>> IMHO this is too low-level a way to change the encoing of stdio/stderr
>> and error-prone too, so some way to reliably set PYTHONIOENCODING from
>> a program embedding python is still needed.
>>
>> --
>> - Campbell
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>
> --
> --Guido van Rossum (python.org/~guido)



-- 
- Campbell

From jcea at jcea.es  Thu Oct  4 15:17:56 2012
From: jcea at jcea.es (Jesus Cea)
Date: Thu, 04 Oct 2012 15:17:56 +0200
Subject: [Python-Dev] Bootstrap issue: "select" is compiled "too late"
In-Reply-To: <CAPZV6o9v_0wU4SRFj_xCfi5fiFWWk-o0MMKYCbadvTJbnOtNaQ@mail.gmail.com>
References: <506D80FF.2060301@jcea.es>
	<CAPZV6o9v_0wU4SRFj_xCfi5fiFWWk-o0MMKYCbadvTJbnOtNaQ@mail.gmail.com>
Message-ID: <506D8C84.1060900@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/10/12 14:34, Benjamin Peterson wrote:
> 2012/10/4 Jesus Cea <jcea at jcea.es>:
>> Any suggestion about how to solve this?
> 
> Easy solutions include somehow removing the dependence on
> subprocess or moving the import of subprocess into the function
> that uses it.

Moving the import into the function that needs it solves the issue.

See <http://hg.python.org/cpython/rev/d6d908dc11f2>.

Thanks for the suggestion, Benjamin.

- -- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQCVAwUBUG2MhJlgi5GaxT1NAQLiuAP+OkMW7wVO3rDvbpBGwes5hf6SfixrOyWu
E/ZMcnhoJ3q8VCdpgIR0GKgzBobc4NuNFotakXe10JH6tQKJe/lnZfgzlJwvj5Pc
G85anv08789oYrxzijmqcdrBJUrSoiqO1WEJBhELUFLjT9gtNnQ2RlEID81LMEzq
Su3+VieZcBk=
=flZu
-----END PGP SIGNATURE-----

From jcea at jcea.es  Thu Oct  4 15:32:25 2012
From: jcea at jcea.es (Jesus Cea)
Date: Thu, 04 Oct 2012 15:32:25 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
Message-ID: <506D8FE9.7060907@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/10/12 17:27, Nick Coghlan wrote:
> I received a fair number of complaints from people that wanted to 
> experiment with yield from, but couldn't, because the first alpha 
> wasn't out yet and they weren't sufficiently interested to go to
> the effort of building their own copy of Python.

In my job, we do "alpha" releases quite frequently. The promises for
alpha is: a) no feature complete, b) the API, etc , can change before
release, c) features included are useable. The point is to use them
and experiment with them.

We don't call them "alphas", but "engineering previews". And they
expires after six months.

Delivering useable alphas requires to do a "minimal" of release
engineering, like disabling features half done, documenting known
bugs, etc. So yes, it it a bit of extra work.

In Python context, the idea of somebody using a "alpha" in production
is scary, and I have a few bad experiences doing a "make install" of a
alpha and when installing the final release keeping some alpha files
around causing erratic behaviour until I did a "\rm -r
/usr/local/SOME" and reinstall again. I think it happened to me with
2.6 :). There are some consequences here, like bytecode (pyc) changes,
modules renaming or vanishing before the final release, etc.

- -- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQCVAwUBUG2P6Jlgi5GaxT1NAQJLBgP9GRnurLjdx2IIcrnbjGuRC0EoQF5ISh5p
nVQldPRefb3c3MEpsVK0pmTDiPY0Nelepg8j8YMjZeri87AKHojMQ6Teu6NGvx/5
ZMRFsBve8YKfkqAr0ygVl8RlscOMXZsCq2iUHhc8I4L2Nqx7H7yoUDlMcak8JB4t
VxYAhugWOmE=
=/2aa
-----END PGP SIGNATURE-----

From brett at python.org  Thu Oct  4 16:24:31 2012
From: brett at python.org (Brett Cannon)
Date: Thu, 4 Oct 2012 10:24:31 -0400
Subject: [Python-Dev] Bootstrap issue: "select" is compiled "too late"
In-Reply-To: <506D8C84.1060900@jcea.es>
References: <506D80FF.2060301@jcea.es>
	<CAPZV6o9v_0wU4SRFj_xCfi5fiFWWk-o0MMKYCbadvTJbnOtNaQ@mail.gmail.com>
	<506D8C84.1060900@jcea.es>
Message-ID: <CAP1=2W5CVkT8rGp-+_XSGSiF090XNcQSQoETjAH0KLOKV_Bj=g@mail.gmail.com>

On Thu, Oct 4, 2012 at 9:17 AM, Jesus Cea <jcea at jcea.es> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 04/10/12 14:34, Benjamin Peterson wrote:
> > 2012/10/4 Jesus Cea <jcea at jcea.es>:
> >> Any suggestion about how to solve this?
> >
> > Easy solutions include somehow removing the dependence on
> > subprocess or moving the import of subprocess into the function
> > that uses it.
>
> Moving the import into the function that needs it solves the issue.
>
> See <http://hg.python.org/cpython/rev/d6d908dc11f2>.
>
> Thanks for the suggestion, Benjamin.


This used to be a no-no, though, because if someone called that function in
a thread during an import it would deadlock. Now Antoine reworked the locks
so I don't know if this is still true or not.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121004/aaa6f114/attachment.html>

From ncoghlan at gmail.com  Thu Oct  4 18:38:40 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 4 Oct 2012 22:08:40 +0530
Subject: [Python-Dev] Bootstrap issue: "select" is compiled "too late"
In-Reply-To: <CAP1=2W5CVkT8rGp-+_XSGSiF090XNcQSQoETjAH0KLOKV_Bj=g@mail.gmail.com>
References: <506D80FF.2060301@jcea.es>
	<CAPZV6o9v_0wU4SRFj_xCfi5fiFWWk-o0MMKYCbadvTJbnOtNaQ@mail.gmail.com>
	<506D8C84.1060900@jcea.es>
	<CAP1=2W5CVkT8rGp-+_XSGSiF090XNcQSQoETjAH0KLOKV_Bj=g@mail.gmail.com>
Message-ID: <CADiSq7fZONfWT7vfXFZ2CTGPfFCA6RRy9EYdm0hryoeow1X=4Q@mail.gmail.com>

On Thu, Oct 4, 2012 at 7:54 PM, Brett Cannon <brett at python.org> wrote:
> This used to be a no-no, though, because if someone called that function in
> a thread during an import it would deadlock. Now Antoine reworked the locks
> so I don't know if this is still true or not.

You can still technically deadlock, but you need a circular import
*and* two different threads that start at different parts of the
cycle.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From zachary.ware+pydev at gmail.com  Thu Oct  4 19:01:43 2012
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Thu, 4 Oct 2012 12:01:43 -0500
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20121001203939.7313A250030@webabinitio.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net>
	<20121001190209.D89622500FA@webabinitio.net>
	<CAL0kPAVfW3GwBpr6-Sg5PE0-uzZQ_XcMH_gvf2AphQzB=cioJQ@mail.gmail.com>
	<20121001203939.7313A250030@webabinitio.net>
Message-ID: <CAKJDb-N598YXabHXt1mBH3UTOoKDd1CrFqnoxErTMCeNO3W4PQ@mail.gmail.com>

It occurred to me this morning that Python already ships a set of
timezone data with the Windows installer: Tcl/Tk's.  Is there any way
we could use that as the default on Windows?

From brian at python.org  Thu Oct  4 19:06:59 2012
From: brian at python.org (Brian Curtin)
Date: Thu, 4 Oct 2012 12:06:59 -0500
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAKJDb-N598YXabHXt1mBH3UTOoKDd1CrFqnoxErTMCeNO3W4PQ@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net>
	<20121001190209.D89622500FA@webabinitio.net>
	<CAL0kPAVfW3GwBpr6-Sg5PE0-uzZQ_XcMH_gvf2AphQzB=cioJQ@mail.gmail.com>
	<20121001203939.7313A250030@webabinitio.net>
	<CAKJDb-N598YXabHXt1mBH3UTOoKDd1CrFqnoxErTMCeNO3W4PQ@mail.gmail.com>
Message-ID: <CAD+XWwq50iav8d++N0rTFFFDuxswu6rNY+t7HXP8nbn3b7vJog@mail.gmail.com>

On Thu, Oct 4, 2012 at 12:01 PM, Zachary Ware
<zachary.ware+pydev at gmail.com> wrote:
> It occurred to me this morning that Python already ships a set of
> timezone data with the Windows installer: Tcl/Tk's.  Is there any way
> we could use that as the default on Windows?

I would say no. You could choose not to include Tcl/Tk in your
installation, or you could build your own Python and not include those
sources. In either case, the lack of a GUI toolkit is now affecting
your use of timezone data - weird/bad.

Plus we only pull updated Tcl/Tk every few years.

From _ at lvh.cc  Thu Oct  4 08:09:01 2012
From: _ at lvh.cc (Laurens Van Houtven)
Date: Thu, 4 Oct 2012 08:09:01 +0200
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <3E5416D3-10CA-4C2B-A0D1-EED1D49F6C47@ox.cx>
References: <506C2129.5010908@hastings.org>
	<CANc-5UzRzVi-ntCGj6vnchLp-1gTHq_igQTycD6-3ir0-F2qiA@mail.gmail.com>
	<506C2768.4010408@hastings.org>
	<CAFYqXL8x=bsRoEBOitLz-=tMO-Tf9Juq075GTP8XYP9ZB1jhDw@mail.gmail.com>
	<20121004013827.F07742500FB@webabinitio.net>
	<3E5416D3-10CA-4C2B-A0D1-EED1D49F6C47@ox.cx>
Message-ID: <CAE_Hg6bAJd7PhQLkxeUNKnS3HZXH+51qiuDoHy5UDqr_9fcuXw@mail.gmail.com>

Hello :)

That PEP is indeed pining to the fjords. I may be able to contribute (and
hopefully I can with the preliminary research and
talking-to-people-about-what-they-want that I've already done that maybe
hasn't made it into the draft PEP versions), but I can not currently spare
the cycles to carry it forth from its current state to a completed
implemented in 3.4 PEP.

cheers
lvh

On Thu, Oct 4, 2012 at 8:00 AM, Hynek Schlawack <hs at ox.cx> wrote:

>
> Am 04.10.2012 um 03:38 schrieb R. David Murray <rdmurray at bitdance.com>:
>
> >>> Other proposed large-scale changes:
> >>> [...]
> >>> * A standard event-loop interface (PEP by Jim Fulton pending)
> >>
> >> Really? Was this discussed somewhere? I'd like to know more about it.
> >
> > I believe it was discussed at the Language Summit at the last Pycon.
> > As I recall there was at least one other person interested in helping
> > with it, but I don't remember who.
>
> T?was lvh and I remember doing some cheap spelling checking on it back at
> EuroPython 20_11_ but the work seems to has stalled 9 months ago.
>
> Adding him to the loop.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121004/409f618b/attachment-0001.html>

From regebro at gmail.com  Thu Oct  4 20:28:19 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Thu, 4 Oct 2012 20:28:19 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAKJDb-N598YXabHXt1mBH3UTOoKDd1CrFqnoxErTMCeNO3W4PQ@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
	<CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>
	<5068D499.6000802@python.org>
	<CAL0kPAV8U5Hh3CVswqoGoG9nUL7i5a6NDefwVx5Gkf7NG6WT=g@mail.gmail.com>
	<CAKmKYaB4uJNk7f5HRjnjrZ_kHGiXD9B9jkAM1UgaD9wM2PfMQw@mail.gmail.com>
	<CAL0kPAUnH1bt7Je+cRqy2fLd4ErLdcuVDfLyueL4Uez-uGe1aQ@mail.gmail.com>
	<CAKmKYaABSLdRtMX7P0etjs_oEipiNxJ706s=Oj=6NbrVyDTpaA@mail.gmail.com>
	<CAL0kPAXYqHEx5sCwRFRx_2exs+4+d8jSbGJ7U=j7mZoxq3A5GQ@mail.gmail.com>
	<20121001135321.15002ee3@pitrou.net>
	<CADiSq7d5jehL5EB-CFiauBviq0qXA2qs8VPdTkMEdFxxK2_wKQ@mail.gmail.com>
	<CAL0kPAVVQBQGNvByp+koaBgzqh3ZVn-AEVQ9v8oxSuv2_a+LaQ@mail.gmail.com>
	<CAKJDb-MSNHxLW9fi6xAqPkbDqmHtx+Dmc47m0CxDa2w3z7pqNw@mail.gmail.com>
	<CAL0kPAVV-_SD_UaY+QSZy2XaeWFJ1iTStTO4ETYHkQ8ybE8Wvw@mail.gmail.com>
	<20121001161226.089ee544@pitrou.net>
	<20121001190209.D89622500FA@webabinitio.net>
	<CAL0kPAVfW3GwBpr6-Sg5PE0-uzZQ_XcMH_gvf2AphQzB=cioJQ@mail.gmail.com>
	<20121001203939.7313A250030@webabinitio.net>
	<CAKJDb-N598YXabHXt1mBH3UTOoKDd1CrFqnoxErTMCeNO3W4PQ@mail.gmail.com>
Message-ID: <CAL0kPAXE-rbomXbnmWzK2i0ExgVsSuDJQZ2aWDzV-3JhNJYMrQ@mail.gmail.com>

On Thu, Oct 4, 2012 at 7:01 PM, Zachary Ware
<zachary.ware+pydev at gmail.com> wrote:
> It occurred to me this morning that Python already ships a set of
> timezone data with the Windows installer: Tcl/Tk's.  Is there any way
> we could use that as the default on Windows?

We could, but it wouldn't solve any of the problems that need solving,
so it's not very useful.

//Lennart

From victor.stinner at gmail.com  Thu Oct  4 22:30:07 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Thu, 4 Oct 2012 22:30:07 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
Message-ID: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>

Hi,

I would like to split the huge unicodeobject.c file into smaller
files. It's just the longest C file of CPython: 14,849 lines.

I don't know exactly how to split it, but first I would like to know
if you would agree with the idea.

Example:
 - Objects/unicode/codecs.c
 - Objects/unicode/mod_format.c
 - Objects/unicode/methods.c
 - Objects/unicode/operators.c
 - etc.

I don't know if it's better to use a subdirectory, or use a prefix for
new files: Objects/unicode_methods.c, Objects/unicode_codecs.c, etc.
There is already a Python/codecs.c file for example (same filename).

I would like to split the unicodeobject.c because it's hard to
navigate in this huge file between all functions, variables, types,
macros, etc. It's hard to add new code and to fix bugs. For example,
the implementation of str%args takes 1000 lines, 2 types and 10
functions (since my refactor yesterday, in Python 3.3 the main
function is 500 lines long :-)).

I only see one argument against such refactoring: it will be harder to
backport/forwardport bugfixes.

Victor

From andrew.svetlov at gmail.com  Thu Oct  4 22:47:00 2012
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Thu, 4 Oct 2012 23:47:00 +0300
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
Message-ID: <CAL3CFcXBRpP1bT6S=zf3MaBkz-L-mxj2O_YGL+BUzes8QCO43w@mail.gmail.com>

I like the idea. From my perspective better to use subdirectory to
sake of easy finding in grep style.

On Thu, Oct 4, 2012 at 11:30 PM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> Hi,
>
> I would like to split the huge unicodeobject.c file into smaller
> files. It's just the longest C file of CPython: 14,849 lines.
>
> I don't know exactly how to split it, but first I would like to know
> if you would agree with the idea.
>
> Example:
>  - Objects/unicode/codecs.c
>  - Objects/unicode/mod_format.c
>  - Objects/unicode/methods.c
>  - Objects/unicode/operators.c
>  - etc.
>
> I don't know if it's better to use a subdirectory, or use a prefix for
> new files: Objects/unicode_methods.c, Objects/unicode_codecs.c, etc.
> There is already a Python/codecs.c file for example (same filename).
>
> I would like to split the unicodeobject.c because it's hard to
> navigate in this huge file between all functions, variables, types,
> macros, etc. It's hard to add new code and to fix bugs. For example,
> the implementation of str%args takes 1000 lines, 2 types and 10
> functions (since my refactor yesterday, in Python 3.3 the main
> function is 500 lines long :-)).
>
> I only see one argument against such refactoring: it will be harder to
> backport/forwardport bugfixes.
>
> Victor
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com



-- 
Thanks,
Andrew Svetlov

From benjamin at python.org  Thu Oct  4 23:00:14 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 4 Oct 2012 17:00:14 -0400
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
Message-ID: <CAPZV6o_HnU_LTdYqneZrTRHKAgmndV-F7e2pXc6NRcTaRM+uPg@mail.gmail.com>

2012/10/4 Victor Stinner <victor.stinner at gmail.com>:
> I only see one argument against such refactoring: it will be harder to
> backport/forwardport bugfixes.

I imagine it could also prevent inlining of hot paths.



-- 
Regards,
Benjamin

From jcea at jcea.es  Thu Oct  4 23:37:20 2012
From: jcea at jcea.es (Jesus Cea)
Date: Thu, 04 Oct 2012 23:37:20 +0200
Subject: [Python-Dev] Bootstrap issue: "select" is compiled "too late"
In-Reply-To: <CADiSq7fZONfWT7vfXFZ2CTGPfFCA6RRy9EYdm0hryoeow1X=4Q@mail.gmail.com>
References: <506D80FF.2060301@jcea.es>
	<CAPZV6o9v_0wU4SRFj_xCfi5fiFWWk-o0MMKYCbadvTJbnOtNaQ@mail.gmail.com>
	<506D8C84.1060900@jcea.es>
	<CAP1=2W5CVkT8rGp-+_XSGSiF090XNcQSQoETjAH0KLOKV_Bj=g@mail.gmail.com>
	<CADiSq7fZONfWT7vfXFZ2CTGPfFCA6RRy9EYdm0hryoeow1X=4Q@mail.gmail.com>
Message-ID: <506E0190.1050805@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/10/12 18:38, Nick Coghlan wrote:
> You can still technically deadlock, but you need a circular import 
> *and* two different threads that start at different parts of the 
> cycle.

Is this true for 2.7 too, or only for recent 3.x?

- -- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQCVAwUBUG4BkJlgi5GaxT1NAQJ3vgP9EDK6d3IKYV02cx/7JFNehK/+MvSdgIUp
KXbf1hlvnMGsWnKgFFKZ4GC16+apyS5g0zXQU5qea/83l06/krcFaERSfZO++alY
jgMJ4AFNBH8r6svoC9Y12WvutV4spzntVWal7kGYh7qTroFceI2sfxHK0PZrsgvn
+qrCmBxspg0=
=2KBh
-----END PGP SIGNATURE-----

From victor.stinner at gmail.com  Thu Oct  4 23:40:05 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Thu, 4 Oct 2012 23:40:05 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAPZV6o_HnU_LTdYqneZrTRHKAgmndV-F7e2pXc6NRcTaRM+uPg@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<CAPZV6o_HnU_LTdYqneZrTRHKAgmndV-F7e2pXc6NRcTaRM+uPg@mail.gmail.com>
Message-ID: <CAMpsgwaLW13gJCL=ELxPsQMQHMgo1xsn9vJwqow+B7_hB6wxrg@mail.gmail.com>

2012/10/4 Benjamin Peterson <benjamin at python.org>:
> 2012/10/4 Victor Stinner <victor.stinner at gmail.com>:
>> I only see one argument against such refactoring: it will be harder to
>> backport/forwardport bugfixes.
>
> I imagine it could also prevent inlining of hot paths.

It depends how the code is compiled. The stringlib is splitted in many
.h files, but it is able to use Py_LOCAL_INLINE.

If the code is grouped correctly, we may not loose any nice optimization at all.

FYI #include "test.c" is allowed in C ;-)

Victor

From martin at v.loewis.de  Thu Oct  4 23:46:57 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Thu, 04 Oct 2012 23:46:57 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
Message-ID: <20121004234657.Horde.McWZBML8999QbgPRvu5ECIA@webmail.df.eu>


Zitat von Victor Stinner <victor.stinner at gmail.com>:

> I only see one argument against such refactoring: it will be harder to
> backport/forwardport bugfixes.

I'm opposed for a different reason: I think it will be *harder* to maintain.
The amount of code will not be reduced, but now you also need to guess what
file some piece of functionality may be in. Instead of having my text editor
(Emacs) search in one file, it will have to search across multiple files -
but not across all open buffers, but only some of them (since I will have
many other source files open as well).

I really fail to see what problem people have with large source files.
What is it that you want to do that can be done easier if it's multiple
files?

Regards,
Martin



From benjamin at python.org  Fri Oct  5 00:11:34 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 4 Oct 2012 18:11:34 -0400
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAMpsgwaLW13gJCL=ELxPsQMQHMgo1xsn9vJwqow+B7_hB6wxrg@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<CAPZV6o_HnU_LTdYqneZrTRHKAgmndV-F7e2pXc6NRcTaRM+uPg@mail.gmail.com>
	<CAMpsgwaLW13gJCL=ELxPsQMQHMgo1xsn9vJwqow+B7_hB6wxrg@mail.gmail.com>
Message-ID: <CAPZV6o9rDR0FsxHci4xMsxgnrhguo4kGgdeXtkDfsmaAH3i+=w@mail.gmail.com>

2012/10/4 Victor Stinner <victor.stinner at gmail.com>:
> 2012/10/4 Benjamin Peterson <benjamin at python.org>:
>> 2012/10/4 Victor Stinner <victor.stinner at gmail.com>:
>>> I only see one argument against such refactoring: it will be harder to
>>> backport/forwardport bugfixes.
>>
>> I imagine it could also prevent inlining of hot paths.
>
> It depends how the code is compiled. The stringlib is splitted in many
> .h files, but it is able to use Py_LOCAL_INLINE.
>
> If the code is grouped correctly, we may not loose any nice optimization at all.
>
> FYI #include "test.c" is allowed in C ;-)

Yes, but then compilation won't be any faster. ;)



-- 
Regards,
Benjamin

From chris.jerdonek at gmail.com  Fri Oct  5 00:16:40 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Thu, 4 Oct 2012 15:16:40 -0700
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
Message-ID: <CAOTb1wd2667sHm+rPE2GB-JvjoVaX2n14C_PVwtuG3RRGCQkEQ@mail.gmail.com>

On Thu, Oct 4, 2012 at 1:30 PM, Victor Stinner <victor.stinner at gmail.com> wrote:
> I would like to split the huge unicodeobject.c file into smaller
> files. It's just the longest C file of CPython: 14,849 lines.
> ...
> I only see one argument against such refactoring: it will be harder to
> backport/forwardport bugfixes.

I am not siding with either side of the change yet, but an additional
argument against is that history may become less convenient to
navigate and track (e.g. hg annotate may lose information depending on
how the split is done).

Do we have a preferred way to split files?  For example, hg rename
could be used just for the largest chunk.  Or hg copy could be used on
all chunks but one.  I imagine (but have not confirmed) that the
latter would preserve hg annotate and let merges propagate to all
files, but it would also result in spurious merge conflicts in every
one of the files resulting from the split.

--Chris

From jcea at jcea.es  Fri Oct  5 00:18:17 2012
From: jcea at jcea.es (Jesus Cea)
Date: Fri, 05 Oct 2012 00:18:17 +0200
Subject: [Python-Dev] Tru64 support
In-Reply-To: <20120905001114.Horde.eP_PTElCcOxQRnyCCc_14kA@webmail.df.eu>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
	<20120905001114.Horde.eP_PTElCcOxQRnyCCc_14kA@webmail.df.eu>
Message-ID: <506E0B29.5060704@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/09/12 00:11, martin at v.loewis.de wrote:
> We don't need to. It's perfectly fine if it breaks - we just can't 
> actively remove code to support the platform.
> 
> OTOH, if Tru64 was proposed for "unsupport", we could disable
> support in 3.4, and remove support in 3.5. Perhaps that would be a
> good thing to do.

In some other email in this thread you said that support was
(partially) deprecated in 3.0, so removal in 3.4 would be justified.

In any case, somebody have to take a decision and update PEP-11.

- -- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQCVAwUBUG4LKZlgi5GaxT1NAQKeCAQAg43Y6LgrYwJWP/TSDx4X9t+lSxE/47HF
J0cEdFtEhT3hm08qFi3fKhR4p7K8gTVFRxNqVBu89dacjCPUrbbnLVI3tTJjvrst
Joqx5RDZv1q3m/OwoFz+kIXEJFEtVNPY/geQlGGYkyoowc/FqFWl/ygpl8rTdjla
NvFmdoBGFUY=
=DIYs
-----END PGP SIGNATURE-----

From brett at python.org  Fri Oct  5 00:30:24 2012
From: brett at python.org (Brett Cannon)
Date: Thu, 4 Oct 2012 18:30:24 -0400
Subject: [Python-Dev] Bootstrap issue: "select" is compiled "too late"
In-Reply-To: <506E0190.1050805@jcea.es>
References: <506D80FF.2060301@jcea.es>
	<CAPZV6o9v_0wU4SRFj_xCfi5fiFWWk-o0MMKYCbadvTJbnOtNaQ@mail.gmail.com>
	<506D8C84.1060900@jcea.es>
	<CAP1=2W5CVkT8rGp-+_XSGSiF090XNcQSQoETjAH0KLOKV_Bj=g@mail.gmail.com>
	<CADiSq7fZONfWT7vfXFZ2CTGPfFCA6RRy9EYdm0hryoeow1X=4Q@mail.gmail.com>
	<506E0190.1050805@jcea.es>
Message-ID: <CAP1=2W6x5q8okFRKEQ3ncw=DMf825xMdrLWS+2mr4snu3rfOmg@mail.gmail.com>

On Thu, Oct 4, 2012 at 5:37 PM, Jesus Cea <jcea at jcea.es> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 04/10/12 18:38, Nick Coghlan wrote:
> > You can still technically deadlock, but you need a circular import
> > *and* two different threads that start at different parts of the
> > cycle.
>
> Is this true for 2.7 too, or only for recent 3.x?


3.3 only has the improvement.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121004/604e684f/attachment.html>

From victor.stinner at gmail.com  Fri Oct  5 01:28:01 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 5 Oct 2012 01:28:01 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAOTb1wd2667sHm+rPE2GB-JvjoVaX2n14C_PVwtuG3RRGCQkEQ@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<CAOTb1wd2667sHm+rPE2GB-JvjoVaX2n14C_PVwtuG3RRGCQkEQ@mail.gmail.com>
Message-ID: <CAMpsgwaoM0wk3cVxNaovVAdsaa0DC4CR1xyEi-LWowWr2ZCS9Q@mail.gmail.com>

> I am not siding with either side of the change yet, but an additional
> argument against is that history may become less convenient to
> navigate and track (e.g. hg annotate may lose information depending on
> how the split is done).

If new files are created using "hg cp unicodeobject.c
unicode/newfile.c", the historic is kept.

Victor

From benjamin at python.org  Fri Oct  5 01:31:35 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 4 Oct 2012 19:31:35 -0400
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAMpsgwaoM0wk3cVxNaovVAdsaa0DC4CR1xyEi-LWowWr2ZCS9Q@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<CAOTb1wd2667sHm+rPE2GB-JvjoVaX2n14C_PVwtuG3RRGCQkEQ@mail.gmail.com>
	<CAMpsgwaoM0wk3cVxNaovVAdsaa0DC4CR1xyEi-LWowWr2ZCS9Q@mail.gmail.com>
Message-ID: <CAPZV6o_kZnrBvQGjpeOrUs8s2dW9FH9eOuLCT-GsKEQoBZsvKA@mail.gmail.com>

2012/10/4 Victor Stinner <victor.stinner at gmail.com>:
>> I am not siding with either side of the change yet, but an additional
>> argument against is that history may become less convenient to
>> navigate and track (e.g. hg annotate may lose information depending on
>> how the split is done).
>
> If new files are created using "hg cp unicodeobject.c
> unicode/newfile.c", the historic is kept.

Yes, but you can only create one file that way.



-- 
Regards,
Benjamin

From victor.stinner at gmail.com  Fri Oct  5 01:35:19 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 5 Oct 2012 01:35:19 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAPZV6o_kZnrBvQGjpeOrUs8s2dW9FH9eOuLCT-GsKEQoBZsvKA@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<CAOTb1wd2667sHm+rPE2GB-JvjoVaX2n14C_PVwtuG3RRGCQkEQ@mail.gmail.com>
	<CAMpsgwaoM0wk3cVxNaovVAdsaa0DC4CR1xyEi-LWowWr2ZCS9Q@mail.gmail.com>
	<CAPZV6o_kZnrBvQGjpeOrUs8s2dW9FH9eOuLCT-GsKEQoBZsvKA@mail.gmail.com>
Message-ID: <CAMpsgwanaBAh7_jtYL6QBLQKyPgaC01GMNKvrCENSb=Ln=hvSg@mail.gmail.com>

2012/10/5 Benjamin Peterson <benjamin at python.org>:
> 2012/10/4 Victor Stinner <victor.stinner at gmail.com>:
>> If new files are created using "hg cp unicodeobject.c
>> unicode/newfile.c", the historic is kept.
>
> Yes, but you can only create one file that way.

You can create as many files as you want. Try:
---
hg cp unicodeobject.c unicode2.c
hg cp unicodeobject.c unicode3.c
hg ci -m "add new files"
<edit unicode2.c (remove most lines)>
<edit unicode3.c (remove most lines, but other lines)>
hg ci -m "modify"
hg blame unicode2.c
hg blame unicode3.c
---

Victor

From chris.jerdonek at gmail.com  Fri Oct  5 01:37:10 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Thu, 4 Oct 2012 16:37:10 -0700
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAPZV6o_kZnrBvQGjpeOrUs8s2dW9FH9eOuLCT-GsKEQoBZsvKA@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<CAOTb1wd2667sHm+rPE2GB-JvjoVaX2n14C_PVwtuG3RRGCQkEQ@mail.gmail.com>
	<CAMpsgwaoM0wk3cVxNaovVAdsaa0DC4CR1xyEi-LWowWr2ZCS9Q@mail.gmail.com>
	<CAPZV6o_kZnrBvQGjpeOrUs8s2dW9FH9eOuLCT-GsKEQoBZsvKA@mail.gmail.com>
Message-ID: <CAOTb1wc8h6ovNmD3ovv5HgAwHEyjd_CEsaEdzanngY3MuYuHPw@mail.gmail.com>

On Thu, Oct 4, 2012 at 4:31 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2012/10/4 Victor Stinner <victor.stinner at gmail.com>:
>>> I am not siding with either side of the change yet, but an additional
>>> argument against is that history may become less convenient to
>>> navigate and track (e.g. hg annotate may lose information depending on
>>> how the split is done).
>>
>> If new files are created using "hg cp unicodeobject.c
>> unicode/newfile.c", the historic is kept.
>
> Yes, but you can only create one file that way.

You can create multiple files this way.  I just verified it.  But the
problem happens with merging.  You will create merge conflicts in the
deleted portions of every split file on every merge.  There may be a
way to avoid this that I don't know about though (i.e. to record that
merges into the deleted portions should no longer occur).

--Chris

From eric at trueblade.com  Fri Oct  5 01:48:56 2012
From: eric at trueblade.com (Eric V. Smith)
Date: Thu, 04 Oct 2012 19:48:56 -0400
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
Message-ID: <506E2068.20903@trueblade.com>

On 10/4/2012 4:30 PM, Victor Stinner wrote:
> Hi,
> 
> I would like to split the huge unicodeobject.c file into smaller
> files. It's just the longest C file of CPython: 14,849 lines.

What problem are you trying to solve?

-- 
Eric.

From storchaka at gmail.com  Wed Oct  3 21:34:01 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Wed, 03 Oct 2012 22:34:01 +0300
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C7198.40600@mrabarnett.plus.com>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CAD+XWwpF91QvUUAd7XPwuN5d0YQEZHWg9WPF5o2tZWggaGP3WA@mail.gmail.com>
	<506C7198.40600@mrabarnett.plus.com>
Message-ID: <k4i3vf$10b$1@ger.gmane.org>

On 03.10.12 20:10, MRAB wrote:
> I downloaded the alphas to test the support for PEP 393 I'd added to
> the regex module.

Hardly alpha would be useful to you if it released before PEP 393 
implementation.



From solipsis at pitrou.net  Fri Oct  5 00:31:39 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 5 Oct 2012 00:31:39 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<20121004234657.Horde.McWZBML8999QbgPRvu5ECIA@webmail.df.eu>
Message-ID: <20121005003139.38d168f8@pitrou.net>

On Thu, 04 Oct 2012 23:46:57 +0200
martin at v.loewis.de wrote:
> 
> Zitat von Victor Stinner <victor.stinner at gmail.com>:
> 
> > I only see one argument against such refactoring: it will be harder to
> > backport/forwardport bugfixes.
> 
> I'm opposed for a different reason: I think it will be *harder* to maintain.
> The amount of code will not be reduced, but now you also need to guess what
> file some piece of functionality may be in. Instead of having my text editor
> (Emacs) search in one file, it will have to search across multiple files -
> but not across all open buffers, but only some of them (since I will have
> many other source files open as well).
> 
> I really fail to see what problem people have with large source files.
> What is it that you want to do that can be done easier if it's multiple
> files?

Navigate, basically. That is, switch between different pieces of code,
without having to type in some text to search for.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From db3l.net at gmail.com  Wed Oct  3 23:50:32 2012
From: db3l.net at gmail.com (David Bolen)
Date: Wed, 03 Oct 2012 17:50:32 -0400
Subject: [Python-Dev] Proposed schedule for Python 3.4
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<CALFfu7DNQpWNPbH6mWPxHRH+bxp+WgFUz02j=1UdEZsZz3KGDw@mail.gmail.com>
	<20121003203849.Horde.AwjFOqGZi1VQbIY5blExjXA@webmail.df.eu>
Message-ID: <m2a9w3kyxj.fsf@valheru.db3l.homeip.net>

martin at v.loewis.de writes:

> We once had nightly builds of the Windows installers. It required a
> dedicated buildbot operator, since the process tended to break.

For what it's worth, I'd be willing to try to re-institute the daily
Windows installer builds if they'd see usage, but I suspect I'll need
assistance with any initial issues with the packaging process, which
is where the issues are most likely to be.

At the time we shut them down (Dec 2008 I think) they had been failing
for about 6 months without anyone noticing, so it didn't seem
repairing it again.

-- David


From tjreedy at udel.edu  Wed Oct  3 21:40:55 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 03 Oct 2012 15:40:55 -0400
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <506C617B.3020801@hastings.org>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CADiSq7fz=DwhDvch7wvi7dwa6YdPLV-82OR4yf0oewTqad4a-g@mail.gmail.com>
	<506C617B.3020801@hastings.org>
Message-ID: <k4i4ce$531$1@ger.gmane.org>

On 10/3/2012 12:02 PM, Larry Hastings wrote:

> What it really comes down to: I'm a first-time RM, and I lack the
> courage/wisdom to overrule what appears to be a reasonable status quo.
> I feel I don't have to defend the decision to maintain the status quo; I
> feel instead you have to make a case for changing it. So far all I
> recall seeing from you are assertions.  I'd like to see some harder data.
>
>
> On 10/03/2012 05:28 PM, Brian Curtin wrote:
>> The webstats in April 2012 show 5628 downloads of 3.3a1 and 4946
>> downloads of 3.3a2 Windows installers.

I am pretty sure I downloaded both, as well as most of the subsequent 
releases. Our alpha releases are as stable as the existing maintenance 
release, except for bugs in new features and possible interactions. In 
return, one gets access to the new features. We could promote the early 
releases a bit more than we do.

Building a full interpreter (including tkinter and other modules) on 
Windows is a challenge. Installing is another challenge yet. I installed 
.a1 and onwards for everyday use and so I could test IDLE patches on 
installed 3.3. The sooner 3.4.0a1 is released, the sooner I will be able 
to do the same for 3.4. Before that, I will have to hold off pushing 
idle patches from 3.3 to default or hope that all is well and wait until 
.a1 is out to find out for sure. I suppose you would call this 
integration testing for IDLE (and underneath that, tkinter) on Windows.

> We simply don't have a fully-automated process to produce installers for
> all platforms.  In fact, I fear we don't have a fully-automated process
> to produce installers for /any/ platforms.

I really wish there were for Windows. If an installer were available 
monthly, I would use it.

-- 
Terry Jan Reedy


From solipsis at pitrou.net  Fri Oct  5 01:47:53 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 5 Oct 2012 01:47:53 +0200
Subject: [Python-Dev] Bootstrap issue: "select" is compiled "too late"
References: <506D80FF.2060301@jcea.es>
	<CAPZV6o9v_0wU4SRFj_xCfi5fiFWWk-o0MMKYCbadvTJbnOtNaQ@mail.gmail.com>
	<506D8C84.1060900@jcea.es>
	<CAP1=2W5CVkT8rGp-+_XSGSiF090XNcQSQoETjAH0KLOKV_Bj=g@mail.gmail.com>
	<CADiSq7fZONfWT7vfXFZ2CTGPfFCA6RRy9EYdm0hryoeow1X=4Q@mail.gmail.com>
Message-ID: <20121005014753.70280366@pitrou.net>

On Thu, 4 Oct 2012 22:08:40 +0530
Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Thu, Oct 4, 2012 at 7:54 PM, Brett Cannon <brett at python.org> wrote:
> > This used to be a no-no, though, because if someone called that function in
> > a thread during an import it would deadlock. Now Antoine reworked the locks
> > so I don't know if this is still true or not.
> 
> You can still technically deadlock, but you need a circular import
> *and* two different threads that start at different parts of the
> cycle.

Actually, this is detected and handled fine (a partial module is
returned to one of the two threads):
http://hg.python.org/cpython/file/b08416a31d15/Lib/test/test_threaded_import.py#l170
http://hg.python.org/cpython/file/b08416a31d15/Lib/test/test_importlib/test_locks.py#l43

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From nad at acm.org  Wed Oct  3 22:01:07 2012
From: nad at acm.org (Ned Deily)
Date: Wed, 03 Oct 2012 13:01:07 -0700
Subject: [Python-Dev] Proposed schedule for Python 3.4
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<20121003203328.Horde.Za5_TqGZi1VQbIT43MHBgoA@webmail.df.eu>
Message-ID: <nad-8D7194.13010703102012@news.gmane.org>

In article 
<20121003203328.Horde.Za5_TqGZi1VQbIT43MHBgoA at webmail.df.eu>,
 martin at v.loewis.de wrote:
> I wouldn't mind having alpha 1 in April 2013, and alpha 2 in October 2013.
> I share Larry's skepticism, and actually fear that it may confuse users
> (which find that they test something completely different from what gets
> released). However, I don't mind supporting this strategy anyway as an
> experiment.

I'm certainly also game to supporting earlier alphas.  Perhaps not on 
April 1, though.

-- 
 Ned Deily,
 nad at acm.org


From stephen at xemacs.org  Fri Oct  5 03:49:38 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 05 Oct 2012 10:49:38 +0900
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAOTb1wc8h6ovNmD3ovv5HgAwHEyjd_CEsaEdzanngY3MuYuHPw@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<CAOTb1wd2667sHm+rPE2GB-JvjoVaX2n14C_PVwtuG3RRGCQkEQ@mail.gmail.com>
	<CAMpsgwaoM0wk3cVxNaovVAdsaa0DC4CR1xyEi-LWowWr2ZCS9Q@mail.gmail.com>
	<CAPZV6o_kZnrBvQGjpeOrUs8s2dW9FH9eOuLCT-GsKEQoBZsvKA@mail.gmail.com>
	<CAOTb1wc8h6ovNmD3ovv5HgAwHEyjd_CEsaEdzanngY3MuYuHPw@mail.gmail.com>
Message-ID: <87zk41k7rh.fsf@uwakimon.sk.tsukuba.ac.jp>

Chris Jerdonek writes:

 > You can create multiple files this way.  I just verified it.  But the
 > problem happens with merging.  You will create merge conflicts in the
 > deleted portions of every split file on every merge.  There may be a
 > way to avoid this that I don't know about though (i.e. to record that
 > merges into the deleted portions should no longer occur).

"hg commit" will do that automatically, but you need to resolve that
conflict once manually.  If you also happen to be merging *from* a
feature branch *into* the trunk, you need to close the feature branch.
If it needs more work, close it and make a new branch.  Alternatively,
merge from trunk into the feature branch "immediately" to minimize the
accumulation of conflicts.  Then the eventual merge back to trunk will
only have "real" conflicts in it.

Note that "immediately" in the sense needed can be done at any time
because what you need to do is merge the revision created by the "hg
cp" operations.  If Victor tags with "hg tag unicode-refactored", then
you just do "hg merge -r unicode-refactored", and if you haven't made
any changes to the relevant files, you shouldn't get any conflicts.
If you do, then use "hg revert -r unicode-refactored <file> ...",
followed by "hg resolve --mark <file> ...", then fix other conflicts,
resolve, and commit as usual.

There's no other way to do it that I know of in any VCS because they
all track conflicts at the file level.  (It would be straightforward
to generalize git to handle this gracefully, but it would be a hugely
disruptive change.  I don't know if Mercurial would be susceptible to
such an extension.)

Specifically, AFAIK this kind of merge conflict will occur:

    - if the branch being merged was forked before the "hg cp", and

    - for each file in the branch containing changes in the deleted region.

I don't advocate this, just want to make the costs clearer.

From benjamin at python.org  Fri Oct  5 04:00:31 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 4 Oct 2012 22:00:31 -0400
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <20121005003139.38d168f8@pitrou.net>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<20121004234657.Horde.McWZBML8999QbgPRvu5ECIA@webmail.df.eu>
	<20121005003139.38d168f8@pitrou.net>
Message-ID: <CAPZV6o_x5+KG_Cp=5vU1dZ_KuQ4DYcU0XFgisuD529JLW27-FA@mail.gmail.com>

2012/10/4 Antoine Pitrou <solipsis at pitrou.net>:
> On Thu, 04 Oct 2012 23:46:57 +0200
> martin at v.loewis.de wrote:
>>
>> Zitat von Victor Stinner <victor.stinner at gmail.com>:
>>
>> > I only see one argument against such refactoring: it will be harder to
>> > backport/forwardport bugfixes.
>>
>> I'm opposed for a different reason: I think it will be *harder* to maintain.
>> The amount of code will not be reduced, but now you also need to guess what
>> file some piece of functionality may be in. Instead of having my text editor
>> (Emacs) search in one file, it will have to search across multiple files -
>> but not across all open buffers, but only some of them (since I will have
>> many other source files open as well).
>>
>> I really fail to see what problem people have with large source files.
>> What is it that you want to do that can be done easier if it's multiple
>> files?
>
> Navigate, basically. That is, switch between different pieces of code,
> without having to type in some text to search for.

I find it's only possible to navigate without searching for extremely
small files.


-- 
Regards,
Benjamin

From tjreedy at udel.edu  Wed Oct  3 21:57:26 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 03 Oct 2012 15:57:26 -0400
Subject: [Python-Dev] Should vars() return modifiable dict?
In-Reply-To: <506C5B09.60005@pearwood.info>
References: <k4gug7$e1q$1@ger.gmane.org> <506C5B09.60005@pearwood.info>
Message-ID: <k4i5bd$d9u$1@ger.gmane.org>

On 10/3/2012 11:34 AM, Steven D'Aprano wrote:
> On 03/10/12 18:54, Serhiy Storchaka wrote:

>> Should behavior of vars() be corrected for locals?
>
> I believe you are misinterpreting what you are seeing. In this case,
> vars() simply returns locals(), which is an ordinary dict, but changes
> to that dict are not guaranteed to propagate to the actual local
> variables themselves. You make changes to that dict, then call vars()
> again, which returns a fresh locals() dict. So what you are seeing is
> simply a side-effect of the fact that changes to locals() may not
> actually effect the local variables.
>
> Note that in IronPython, the behaviour of your code is different:
>
>
> steve at runes:~$ ipy
> IronPython 2.6 Beta 2 DEBUG (2.6.0.20) on .NET 2.0.50727.1433
> Type "help", "copyright", "credits" or "license" for more information.
>>>> def f():
> ...     x = 42
> ...     print(vars())
> ...     vars()['x'] = 43
> ...     vars()['y'] = 44
> ...     print(x, vars())
> ...
>>>> f()
> {'x': 42}
> (43, {'y': 44, 'x': 43})
>
>
>
>> Should vars() for objects with __slots__ [1] returns modifiable or
>> non-modifiable dict?
>>
>> [1] http://bugs.python.org/issue13290
>
>
> You are assuming that the behaviour of vars(obj) should change. I don't
> think
> that is necessarily the case.
>
> vars(obj) is defined as returning the object __dict__ attribute. If an
> object
> has no __dict__, vars() should (pick one):
>
> 1) Keep the current behaviour and raise an exception.
>
> 2) Return a regular dict containing {slot: value} for each of the slots.
>     Since the dict is a copy, changes to the dict will not effect the
>     original object.
>
> 3) Return a dictproxy containing {slot: value} for each of the slots.
>     Since the dictproxy does not support item assignment, you can't
>     modify it.
>
> 4) Return some other proxy object such that changes to the dict will
>     also change the object's slot attributes.
>
>
> I find myself unable to choose between 2) and 4), which suggests that
> the status quo wins and we keep the current behaviour.
>
>
>


-- 
Terry Jan Reedy


From jcea at jcea.es  Fri Oct  5 04:31:02 2012
From: jcea at jcea.es (Jesus Cea)
Date: Fri, 05 Oct 2012 04:31:02 +0200
Subject: [Python-Dev] =?utf-8?b?d2hhdMK0cyBuZXcgMy4z?=
In-Reply-To: <EFE3877620384242A686D52278B7CCD361F66E@RKV-IT-EXCH104.ccp.ad.local>
References: <EFE3877620384242A686D52278B7CCD361F66E@RKV-IT-EXCH104.ccp.ad.local>
Message-ID: <506E4666.6060906@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 30/09/12 12:55, Kristj?n Valur J?nsson wrote:
> Hi there.
> 
> Not having kept up, I realized I failed to contribute to the What's
> new thingie.
> 
> Here's stuff I remember working on and putting in:
> 
> 1. pickling support for built in iterators (#14288) 2. inter
> process socket duplication for windows (#14310) 3. Progress
> callback for gc module (#10576) 4. Faster locking on windows
> (#15038)
> 
> 
> 
> DiSome of this should probably be mentioned in the What's new
> document, even if only in its online version.

Please, update "Doc/whatsnew/3.3.rst" in 3.3 branch. The changes are
automatically available in the online documentation after a while (a
few hours).

- -- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQCVAwUBUG5GZplgi5GaxT1NAQKhaQP/RKzxA44RN2VZeD9MTIK/tiS8ddg/uAWA
9YPYYwVoDQf/sF1S+W/zb/Vf3YjuoHsFFLGTzWIHkY6G8hiVGGfks91/3Ur7n/GO
kE6CHQRpCzMYYnR2MvMjqlO99BD1++ex/+MKSiYFTUwsTIBVLSFicB5oG5WzWH/a
cKl8n0DCv8w=
=ULJD
-----END PGP SIGNATURE-----

From stephen at xemacs.org  Fri Oct  5 05:02:26 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 05 Oct 2012 12:02:26 +0900
Subject: [Python-Dev] Proposed schedule for Python 3.4
In-Reply-To: <k4i3vf$10b$1@ger.gmane.org>
References: <506C2129.5010908@hastings.org>
	<CADiSq7dQqsAZwyvh8fyn24gSs+5=-ZBKnbngpoRBUkymPxmEEg@mail.gmail.com>
	<506C25A4.5090900@hastings.org>
	<CADiSq7dEq1NvsTS=LfdC5qRqtsDd6pJoFBpQryU4CCa+n30YAA@mail.gmail.com>
	<506C560E.80306@hastings.org>
	<CAD+XWwpF91QvUUAd7XPwuN5d0YQEZHWg9WPF5o2tZWggaGP3WA@mail.gmail.com>
	<506C7198.40600@mrabarnett.plus.com> <k4i3vf$10b$1@ger.gmane.org>
Message-ID: <87vcepk4e5.fsf@uwakimon.sk.tsukuba.ac.jp>

Executive summary:

Larry admits that doing an early alpha is a plausible experiment, he
just has reasons not to do it himself.  OTOH, Nick has a reason for
wanting to do it in *this* release cycle.

It's Larry's call; we should restrict ourselves to giving him the
information he requests to make a decision on which reasons are more
important to him.

Serhiy Storchaka writes:
 > On 03.10.12 20:10, MRAB wrote:
 > > I downloaded the alphas to test the support for PEP 393 I'd added to
 > > the regex module.
 > 
 > Hardly alpha would be useful to you if it released before PEP 393 
 > implementation.

Of course not.  But Nick is requesting this precisely because the PEPs
he wants to get extra testing are already implemented, or very close
to it, and presumably his (implied) promise to have "reasonably
complete" implementations available by alpha 1 is credible.  So your
counterfactual is hardly relevant.

MRAB's example of how alphas are useful is relevant because it's a
real example, with rationale, of the kind of third-party integration
testing Larry suspects *won't happen* if an alpha is released way
early.


From benno at benno.id.au  Fri Oct  5 05:56:45 2012
From: benno at benno.id.au (Ben Leslie)
Date: Thu, 4 Oct 2012 23:56:45 -0400
Subject: [Python-Dev] Thread/garbage collection race in Popen
Message-ID: <CABZ0LtA0=s3BMMMzyG-MhNC0fBSVOmaiudo5WPZPCGMMrCX3Eg@mail.gmail.com>

Hi all,

I have a Python program where I have many threads each calling Popen, and I
was hitting some trouble.

I've been seeing this on 3.2.3, however I believe the same issue is still
potentially a problem on head.

The error manifests itself when a call to os.close(errpipe_read) fails with
EBADF (
http://hg.python.org/releasing/3.2.3/file/86d1421a552c/Lib/subprocess.py#l1314
)

I believe the root cause of this problem is due to a double close() on a
different file descriptor (which is then reused as errpipe_read).

The file descriptors: p2cwrite, c2pread and errread are all closed at the
end of the _execute_child method:

http://hg.python.org/releasing/3.2.3/file/86d1421a552c/Lib/subprocess.py#l1351

However, these filedescriptors are wrapped up into file objects during
__init__ (see:
http://hg.python.org/releasing/3.2.3/file/86d1421a552c/Lib/subprocess.py#l725
)

As far as I can tell at the point where the garbage collector kicks in
Popen.{stdin,stdout,stderr} all go out of scope, and will be deallocated,
and the underlying filedescriptor closed.

However because the filedescriptor is already closed, and by this time is
actually reused, this deallocation closes what ends up being an incorrect
file-descriptor.

Since closing a file object where the underlying fd is already closed is
silenced (
http://hg.python.org/releasing/3.2.3/file/86d1421a552c/Modules/_io/iobase.c#l235)
this would not normally be very apparent.

This race between a new filedescriptor being allocated and the garbage
collector deallocating the file descriptors certainly hits when using a few
threads, but I guess depending on the exact behaviour of the garbage
collector it could potentially also occur in a single threaded case as well.

I think a fix would be to remove the explicit close of these file
descriptors at the end of _execute_child, and let the garbage collector
close them. Of course that may leak file descriptors, if the GC doesn't
kick in for a while, so the other option would be to close the file object,
rather than just the file descriptor.

Hopefully someone more intimately familiar with the module can point me in
the right direction to verify this, and provide a fix.

Thanks,

Benno
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121004/6c896813/attachment-0001.html>

From greg at krypto.org  Fri Oct  5 09:08:58 2012
From: greg at krypto.org (Gregory P. Smith)
Date: Fri, 5 Oct 2012 00:08:58 -0700
Subject: [Python-Dev] Thread/garbage collection race in Popen
In-Reply-To: <CABZ0LtA0=s3BMMMzyG-MhNC0fBSVOmaiudo5WPZPCGMMrCX3Eg@mail.gmail.com>
References: <CABZ0LtA0=s3BMMMzyG-MhNC0fBSVOmaiudo5WPZPCGMMrCX3Eg@mail.gmail.com>
Message-ID: <CAGE7PN+CA2DvgyO-+uNzn02WVpwr8XK7Mu6WixxUqrF_xR1S4Q@mail.gmail.com>

On Thu, Oct 4, 2012 at 8:56 PM, Ben Leslie <benno at benno.id.au> wrote:

> Hi all,
>
> I have a Python program where I have many threads each calling Popen, and
> I was hitting some trouble.
>
> I've been seeing this on 3.2.3, however I believe the same issue is still
> potentially a problem on head.
>
> The error manifests itself when a call to os.close(errpipe_read) fails
> with EBADF (
> http://hg.python.org/releasing/3.2.3/file/86d1421a552c/Lib/subprocess.py#l1314
> )
>
> I believe the root cause of this problem is due to a double close() on a
> different file descriptor (which is then reused as errpipe_read).
>
> The file descriptors: p2cwrite, c2pread and errread are all closed at the
> end of the _execute_child method:
>
>
> http://hg.python.org/releasing/3.2.3/file/86d1421a552c/Lib/subprocess.py#l1351
>
> However, these filedescriptors are wrapped up into file objects during
> __init__ (see:
> http://hg.python.org/releasing/3.2.3/file/86d1421a552c/Lib/subprocess.py#l725
> )
>
> As far as I can tell at the point where the garbage collector kicks in
> Popen.{stdin,stdout,stderr} all go out of scope, and will be deallocated,
> and the underlying filedescriptor closed.
>
> However because the filedescriptor is already closed, and by this time is
> actually reused, this deallocation closes what ends up being an incorrect
> file-descriptor.
>
> Since closing a file object where the underlying fd is already closed is
> silenced (
> http://hg.python.org/releasing/3.2.3/file/86d1421a552c/Modules/_io/iobase.c#l235)
> this would not normally be very apparent.
>
> This race between a new filedescriptor being allocated and the garbage
> collector deallocating the file descriptors certainly hits when using a few
> threads, but I guess depending on the exact behaviour of the garbage
> collector it could potentially also occur in a single threaded case as well.
>
> I think a fix would be to remove the explicit close of these file
> descriptors at the end of _execute_child, and let the garbage collector
> close them. Of course that may leak file descriptors, if the GC doesn't
> kick in for a while, so the other option would be to close the file object,
> rather than just the file descriptor.
>
> Hopefully someone more intimately familiar with the module can point me in
> the right direction to verify this, and provide a fix.
>
> Thanks,
>
> Benno
>

I've filed http://bugs.python.org/issue16140 to track this.  In general for
bug reports always start by filing it there (then point to it for
discussion on python-dev). :)

What you describe makes sense to me.  I'll follow up there.

-gps (subprocess maintainer)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121005/848338b4/attachment.html>

From mal at egenix.com  Fri Oct  5 09:27:17 2012
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 05 Oct 2012 09:27:17 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
Message-ID: <506E8BD5.5050402@egenix.com>

Victor Stinner wrote:
> Hi,
> 
> I would like to split the huge unicodeobject.c file into smaller
> files. It's just the longest C file of CPython: 14,849 lines.
> 
> I don't know exactly how to split it, but first I would like to know
> if you would agree with the idea.
> 
> Example:
>  - Objects/unicode/codecs.c
>  - Objects/unicode/mod_format.c
>  - Objects/unicode/methods.c
>  - Objects/unicode/operators.c
>  - etc.
> 
> I don't know if it's better to use a subdirectory, or use a prefix for
> new files: Objects/unicode_methods.c, Objects/unicode_codecs.c, etc.
> There is already a Python/codecs.c file for example (same filename).

Better follow the already existing pattern of using "unicode" as
prefix, e.g. unicodectype.c and unicodetype_db.h.

> I would like to split the unicodeobject.c because it's hard to
> navigate in this huge file between all functions, variables, types,
> macros, etc. It's hard to add new code and to fix bugs. For example,
> the implementation of str%args takes 1000 lines, 2 types and 10
> functions (since my refactor yesterday, in Python 3.3 the main
> function is 500 lines long :-)).
> 
> I only see one argument against such refactoring: it will be harder to
> backport/forwardport bugfixes.

When making such a change, you have to pay close attention to
functions that the compiler can potentially inline. AFAIK, moving
such functions into a separate file would prevent such
inlining/optimizations, e.g. the str formatter wouldn't be
able to inline codec calls if placed in separate .c files.

It may be better to split the file into multiple .h files which
then get recombined into the one unicodeobject.c file.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 05 2012)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2012-09-27: Released eGenix PyRun 1.1.0 ...       http://egenix.com/go35
2012-09-26: Released mxODBC.Connect 2.0.1 ...     http://egenix.com/go34
2012-09-25: Released mxODBC 3.2.1 ...             http://egenix.com/go33
2012-10-23: Python Meeting Duesseldorf ...                 18 days to go

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From andrew.svetlov at gmail.com  Fri Oct  5 09:31:53 2012
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Fri, 5 Oct 2012 10:31:53 +0300
Subject: [Python-Dev] [Python-checkins] cpython: Issue #14446: Remove
 deprecated tkinter functions: Delete an unused function to
In-Reply-To: <3XXvPv5FVtzQqk@mail.python.org>
References: <3XXvPv5FVtzQqk@mail.python.org>
Message-ID: <CAL3CFcUsNtYDrs-GJ3cqzP-kkwRgAmtKXYgN7XP0SjNgk-4zww@mail.gmail.com>

Thank you, Jesus. I've missed this function.

On Fri, Oct 5, 2012 at 5:03 AM, jesus.cea <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/e278f3ab0190
> changeset:   79484:e278f3ab0190
> user:        Jesus Cea <jcea at jcea.es>
> date:        Fri Oct 05 04:02:41 2012 +0200
> summary:
>   Issue #14446: Remove deprecated tkinter functions: Delete an unused function to avoid a warning
>
> files:
>   Modules/_tkinter.c |  30 ------------------------------
>   1 files changed, 0 insertions(+), 30 deletions(-)
>
>
> diff --git a/Modules/_tkinter.c b/Modules/_tkinter.c
> --- a/Modules/_tkinter.c
> +++ b/Modules/_tkinter.c
> @@ -323,36 +323,6 @@
>  #endif /* WITH_THREAD */
>
>
> -static char *
> -AsString(PyObject *value, PyObject *tmp)
> -{
> -    if (PyBytes_Check(value))
> -        return PyBytes_AsString(value);
> -    else if (PyUnicode_Check(value)) {
> -        PyObject *v = PyUnicode_AsUTF8String(value);
> -        if (v == NULL)
> -            return NULL;
> -        if (PyList_Append(tmp, v) != 0) {
> -            Py_DECREF(v);
> -            return NULL;
> -        }
> -        Py_DECREF(v);
> -        return PyBytes_AsString(v);
> -    }
> -    else {
> -        PyObject *v = PyObject_Str(value);
> -        if (v == NULL)
> -            return NULL;
> -        if (PyList_Append(tmp, v) != 0) {
> -            Py_DECREF(v);
> -            return NULL;
> -        }
> -        Py_DECREF(v);
> -        return PyBytes_AsString(v);
> -    }
> -}
> -
> -
>
>  #define ARGSZ 64
>
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>



-- 
Thanks,
Andrew Svetlov

From jeanpierreda at gmail.com  Fri Oct  5 13:59:37 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Fri, 5 Oct 2012 07:59:37 -0400
Subject: [Python-Dev] Should vars() return modifiable dict?
In-Reply-To: <506C5B09.60005@pearwood.info>
References: <k4gug7$e1q$1@ger.gmane.org> <506C5B09.60005@pearwood.info>
Message-ID: <CABicbJLK=3444Hh1Z5Ut0ZfvTS20NzbtJRooDB-2GAJcZody2A@mail.gmail.com>

On Wed, Oct 3, 2012 at 11:34 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> I find myself unable to choose between 2) and 4), which suggests that
> the status quo wins and we keep the current behaviour.

What is the benefit of having a dict that represents a namespace, but
whose mutations don't mutate said namespace? Compare with option 2,
where the dict is mutable, and whose mutations mutate the namespace it
represents. That behavior is altogether significantly less surprising.

I've never understood why locals() returned a mutable dictionary
either, except that Python has no immutable dictionary type.

-- Devin

From ncoghlan at gmail.com  Fri Oct  5 14:58:38 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 5 Oct 2012 18:28:38 +0530
Subject: [Python-Dev] Should vars() return modifiable dict?
In-Reply-To: <CABicbJLK=3444Hh1Z5Ut0ZfvTS20NzbtJRooDB-2GAJcZody2A@mail.gmail.com>
References: <k4gug7$e1q$1@ger.gmane.org> <506C5B09.60005@pearwood.info>
	<CABicbJLK=3444Hh1Z5Ut0ZfvTS20NzbtJRooDB-2GAJcZody2A@mail.gmail.com>
Message-ID: <CADiSq7fozd6Wa=eeehHpq3W_zAQEW3-7aNxk0h_SK11G-AhMoA@mail.gmail.com>

On Fri, Oct 5, 2012 at 5:29 PM, Devin Jeanpierre <jeanpierreda at gmail.com> wrote:
> On Wed, Oct 3, 2012 at 11:34 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>> I find myself unable to choose between 2) and 4), which suggests that
>> the status quo wins and we keep the current behaviour.
>
> What is the benefit of having a dict that represents a namespace, but
> whose mutations don't mutate said namespace? Compare with option 2,
> where the dict is mutable, and whose mutations mutate the namespace it
> represents. That behavior is altogether significantly less surprising.
>
> I've never understood why locals() returned a mutable dictionary
> either, except that Python has no immutable dictionary type.

There's no benefit, it's just a historical artefact imposed by
backwards compatibility constraints. We should have taken the
opportunity to fix it in Python 3.0 (just as we dropped support for
"from x import *" at function scope) but I don't believe anyone
suggested it at the time.

The problem is that, while updating it to return
types.MappingProxyType instead would produce more obvious error
messages for those that don't know you can't use it to update local
variables inside a function (even though it works at module and class
scopes), such a change would *break* existing code which knows the
dictionary doesn't get written back, and just uses it for its own
purposes (e.g passing it to exec).

As for *why* changes don't get written back, it's because the compiler
is allowed to assume it knows about every variable name that exists in
the local scope (by design), and that doesn't fit with writable
locals() for functions.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From larry at hastings.org  Fri Oct  5 15:01:02 2012
From: larry at hastings.org (Larry Hastings)
Date: Fri, 05 Oct 2012 15:01:02 +0200
Subject: [Python-Dev] Should vars() return modifiable dict?
In-Reply-To: <CABicbJLK=3444Hh1Z5Ut0ZfvTS20NzbtJRooDB-2GAJcZody2A@mail.gmail.com>
References: <k4gug7$e1q$1@ger.gmane.org> <506C5B09.60005@pearwood.info>
	<CABicbJLK=3444Hh1Z5Ut0ZfvTS20NzbtJRooDB-2GAJcZody2A@mail.gmail.com>
Message-ID: <506EDA0E.9090200@hastings.org>

On 10/05/2012 01:59 PM, Devin Jeanpierre wrote:
> I've never understood why locals() returned a mutable dictionary
> either, except that Python has no immutable dictionary type.

Ah, but these days it has types.MappingProxyType which provides a 
read-only wrapper around another mapping type.  Perhaps that should be 
employed here.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121005/7aee8a0b/attachment.html>

From steve at pearwood.info  Fri Oct  5 15:47:12 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 05 Oct 2012 23:47:12 +1000
Subject: [Python-Dev] Should vars() return modifiable dict?
In-Reply-To: <CADiSq7fozd6Wa=eeehHpq3W_zAQEW3-7aNxk0h_SK11G-AhMoA@mail.gmail.com>
References: <k4gug7$e1q$1@ger.gmane.org> <506C5B09.60005@pearwood.info>
	<CABicbJLK=3444Hh1Z5Ut0ZfvTS20NzbtJRooDB-2GAJcZody2A@mail.gmail.com>
	<CADiSq7fozd6Wa=eeehHpq3W_zAQEW3-7aNxk0h_SK11G-AhMoA@mail.gmail.com>
Message-ID: <506EE4E0.4020006@pearwood.info>

On 05/10/12 22:58, Nick Coghlan wrote about locals():

> As for *why* changes don't get written back, it's because the compiler
> is allowed to assume it knows about every variable name that exists in
> the local scope (by design), and that doesn't fit with writable
> locals() for functions.

And to be clear, that is implementation-dependent. IronPython locals()
does write back to local variables.

Trying to modify locals() can lead to some really unintuitive behaviour.
For example, in CPython 3.2:

py> def test():
...     x = 1
...     locals()['x'] = 2  # writing to locals has no effect
...     locals()['y'] = 3  # except when it does...
...     print(x, locals())
...
py> test()
1 {'y': 3, 'x': 1}



-- 
Steven

From status at bugs.python.org  Fri Oct  5 18:07:11 2012
From: status at bugs.python.org (Python tracker)
Date: Fri,  5 Oct 2012 18:07:11 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20121005160711.03DEF1CB9D@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2012-09-28 - 2012-10-05)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3770 (+16)
  closed 24149 (+43)
  total  27919 (+59)

Open issues with patches: 1676 


Issues opened (44)
==================

#3982: support .format for bytes
http://bugs.python.org/issue3982  reopened by exarkun

#15111: Wrong ImportError message with importlib
http://bugs.python.org/issue15111  reopened by brett.cannon

#15379: Charmap decoding of no-BMP characters
http://bugs.python.org/issue15379  reopened by serhiy.storchaka

#16086: tp_flags: Undefined behaviour with 32 bits long
http://bugs.python.org/issue16086  opened by haypo

#16088: http.server's send_error could set the content-length of the e
http://bugs.python.org/issue16088  opened by pitrou

#16094: Tuple extraction in a lambda isn't supported by 2to3
http://bugs.python.org/issue16094  opened by GlitchMr

#16095: urllib2 failing with squid proxy and digest authentication
http://bugs.python.org/issue16095  opened by toobaz

#16096: Get rid of dangerous integer overflow tricks
http://bugs.python.org/issue16096  opened by serhiy.storchaka

#16098: Bisect optimization in heapq.nsmallest is never used
http://bugs.python.org/issue16098  opened by haldean

#16099: robotparser doesn't support request rate and crawl delay param
http://bugs.python.org/issue16099  opened by XapaJIaMnu

#16100: Compiling vim with Python 3.3 support fails
http://bugs.python.org/issue16100  opened by v_core

#16101: Verify all imported modules at startup are needed
http://bugs.python.org/issue16101  opened by brett.cannon

#16102: uuid._netbios_getnode() is outdated
http://bugs.python.org/issue16102  opened by serhiy.storchaka

#16103: Help() fails at raw_input readline (IDLE 2.7.3, Win7, pythonw)
http://bugs.python.org/issue16103  opened by terry.reedy

#16104: Use multiprocessing in compileall script
http://bugs.python.org/issue16104  opened by dholth

#16105: Pass read only FD to signal.set_wakeup_fd
http://bugs.python.org/issue16105  opened by felipecruz

#16106: antigravity tests
http://bugs.python.org/issue16106  opened by ezio.melotti

#16107: distutils2.version doesn't str() "1.0.post1" correctly
http://bugs.python.org/issue16107  opened by richard

#16108: Include maintainer information in register/upload
http://bugs.python.org/issue16108  opened by richard

#16110: Provide logging.config.configParserConfig
http://bugs.python.org/issue16110  opened by thbach

#16111: Python 2.7.3 Windows MSI installer installs the VC++ 9 dlls di
http://bugs.python.org/issue16111  opened by freaksterrao

#16112: platform.architecture does not correctly escape argument to /u
http://bugs.python.org/issue16112  opened by David.Benjamin

#16113: Add SHA-3 (Keccak) support
http://bugs.python.org/issue16113  opened by christian.heimes

#16114: incorrect path in subprocess.Popen() FileNotFoundError message
http://bugs.python.org/issue16114  opened by chris.jerdonek

#16115: test that executable arg to Popen() takes precedence over args
http://bugs.python.org/issue16115  opened by chris.jerdonek

#16116: Can not install C extension modules to inside a venv on Python
http://bugs.python.org/issue16116  opened by msmhrt

#16117: python2.7.3 struct misaligned when returned
http://bugs.python.org/issue16117  opened by iroli

#16119: Python 2.7 _socket DLL import error on Windows Vista
http://bugs.python.org/issue16119  opened by sk7

#16120: Use |yield from| in stdlib
http://bugs.python.org/issue16120  opened by berker.peksag

#16121: shlex.shlex.error_leader() reports incorrect line number
http://bugs.python.org/issue16121  opened by Arfrever

#16123: IDLE - deprecate running without a subprocess
http://bugs.python.org/issue16123  opened by serwy

#16124: fcntl_ioctl still not 100% sane with unsigned longs
http://bugs.python.org/issue16124  opened by yaneurabeya

#16125: open accepts arbitrary mode strings as long as they contain U
http://bugs.python.org/issue16125  opened by nneonneo

#16128: hashable documentation error
http://bugs.python.org/issue16128  opened by max

#16129: No good way to set 'PYTHONIOENCODING' when embedding python.
http://bugs.python.org/issue16129  opened by ideasman42

#16132: ctypes incorrectly encodes .format attribute of memory views
http://bugs.python.org/issue16132  opened by dabeaz

#16133: asyncore.dispatcher.recv doesn't handle EAGAIN / EWOULDBLOCK
http://bugs.python.org/issue16133  opened by Nidan

#16134: Add support for RTMP schemes to urlparse
http://bugs.python.org/issue16134  opened by Jorge.Gomes

#16136: Removal of VMS support
http://bugs.python.org/issue16136  opened by jcea

#16137: Using time.asctime() with an array with negative tm_hour cause
http://bugs.python.org/issue16137  opened by Viktor.Chynarov

#16139: Python 3.3 fails when starting from read-only FS
http://bugs.python.org/issue16139  opened by andrewjcg

#16140: subprocess.Popen the os.close calls in _execute_child can rais
http://bugs.python.org/issue16140  opened by gregory.p.smith

#16141: Possible simplification for logging.StreamHandler exception ha
http://bugs.python.org/issue16141  opened by gvanrossum

#16142: ArgumentParser inconsistent with parse_known_args
http://bugs.python.org/issue16142  opened by idank



Most recent 15 issues with no replies (15)
==========================================

#16141: Possible simplification for logging.StreamHandler exception ha
http://bugs.python.org/issue16141

#16133: asyncore.dispatcher.recv doesn't handle EAGAIN / EWOULDBLOCK
http://bugs.python.org/issue16133

#16132: ctypes incorrectly encodes .format attribute of memory views
http://bugs.python.org/issue16132

#16128: hashable documentation error
http://bugs.python.org/issue16128

#16124: fcntl_ioctl still not 100% sane with unsigned longs
http://bugs.python.org/issue16124

#16121: shlex.shlex.error_leader() reports incorrect line number
http://bugs.python.org/issue16121

#16117: python2.7.3 struct misaligned when returned
http://bugs.python.org/issue16117

#16111: Python 2.7.3 Windows MSI installer installs the VC++ 9 dlls di
http://bugs.python.org/issue16111

#16108: Include maintainer information in register/upload
http://bugs.python.org/issue16108

#16102: uuid._netbios_getnode() is outdated
http://bugs.python.org/issue16102

#16095: urllib2 failing with squid proxy and digest authentication
http://bugs.python.org/issue16095

#16070: Structure and native Structure (LittleEndianStructure on Windo
http://bugs.python.org/issue16070

#16067: UAC prompt for installation shows temporary file name
http://bugs.python.org/issue16067

#16058: ConfigParser no longer deepcopy compatible in 2.7
http://bugs.python.org/issue16058

#16047: Tools/freeze no longer works in Python 3
http://bugs.python.org/issue16047



Most recent 15 issues waiting for review (15)
=============================================

#16125: open accepts arbitrary mode strings as long as they contain U
http://bugs.python.org/issue16125

#16123: IDLE - deprecate running without a subprocess
http://bugs.python.org/issue16123

#16120: Use |yield from| in stdlib
http://bugs.python.org/issue16120

#16115: test that executable arg to Popen() takes precedence over args
http://bugs.python.org/issue16115

#16114: incorrect path in subprocess.Popen() FileNotFoundError message
http://bugs.python.org/issue16114

#16113: Add SHA-3 (Keccak) support
http://bugs.python.org/issue16113

#16112: platform.architecture does not correctly escape argument to /u
http://bugs.python.org/issue16112

#16110: Provide logging.config.configParserConfig
http://bugs.python.org/issue16110

#16108: Include maintainer information in register/upload
http://bugs.python.org/issue16108

#16107: distutils2.version doesn't str() "1.0.post1" correctly
http://bugs.python.org/issue16107

#16106: antigravity tests
http://bugs.python.org/issue16106

#16105: Pass read only FD to signal.set_wakeup_fd
http://bugs.python.org/issue16105

#16099: robotparser doesn't support request rate and crawl delay param
http://bugs.python.org/issue16099

#16098: Bisect optimization in heapq.nsmallest is never used
http://bugs.python.org/issue16098

#16096: Get rid of dangerous integer overflow tricks
http://bugs.python.org/issue16096



Top 10 most discussed issues (10)
=================================

#8800: add threading.RWLock
http://bugs.python.org/issue8800  50 msgs

#16112: platform.architecture does not correctly escape argument to /u
http://bugs.python.org/issue16112  27 msgs

#16105: Pass read only FD to signal.set_wakeup_fd
http://bugs.python.org/issue16105  15 msgs

#16079: list duplicate test names with patchcheck
http://bugs.python.org/issue16079  14 msgs

#3982: support .format for bytes
http://bugs.python.org/issue3982  13 msgs

#13290: get vars for object with __slots__
http://bugs.python.org/issue13290  12 msgs

#16113: Add SHA-3 (Keccak) support
http://bugs.python.org/issue16113  11 msgs

#16074: bad error message in os.rename
http://bugs.python.org/issue16074  10 msgs

#12947: doctest directive examples in library/doctest.html lack the fl
http://bugs.python.org/issue12947   9 msgs

#16116: Can not install C extension modules to inside a venv on Python
http://bugs.python.org/issue16116   9 msgs



Issues closed (45)
==================

#2489: Patch for bugs in pty.py
http://bugs.python.org/issue2489  closed by gregory.p.smith

#9650: format codes in time.strptime docstrings
http://bugs.python.org/issue9650  closed by belopolsky

#12069: test_signal.test_without_siginterrupt() failure on AMD64 OpenI
http://bugs.python.org/issue12069  closed by haypo

#13398: _cursesmodule does not build, doesn't find Python.h on Solaris
http://bugs.python.org/issue13398  closed by jcea

#13801: The Python 3 Docs don't highlight nonlocal
http://bugs.python.org/issue13801  closed by georg.brandl

#14161: python2 file __repr__ does not escape filename
http://bugs.python.org/issue14161  closed by ezio.melotti

#14331: Python/import.c uses a lot of stack space due to MAXPATHLEN
http://bugs.python.org/issue14331  closed by gregory.p.smith

#14446: Remove deprecated tkinter functions
http://bugs.python.org/issue14446  closed by asvetlov

#14503: docs: "yield from" breaks Pygments syntax coloring in doc exam
http://bugs.python.org/issue14503  closed by georg.brandl

#14997: IDLE tries to run shell window if line is completed with F5 ra
http://bugs.python.org/issue14997  closed by asvetlov

#15201: C argument errors and Python arguments error are different
http://bugs.python.org/issue15201  closed by benjamin.peterson

#15417: Add support for csh and fish in venv activation scripts
http://bugs.python.org/issue15417  closed by asvetlov

#15488: Closed files keep their buffer alive
http://bugs.python.org/issue15488  closed by python-dev

#15609: Format string: add more fast-path
http://bugs.python.org/issue15609  closed by haypo

#15756: subprocess.poll() does not handle errno.ECHILD "No child proce
http://bugs.python.org/issue15756  closed by gregory.p.smith

#15766: _imp.load_dynamic() does crash with non-ASCII path and uses th
http://bugs.python.org/issue15766  closed by python-dev

#15897: zipimport.c doesn't check return value of fseek()
http://bugs.python.org/issue15897  closed by python-dev

#15923: Building from a fresh clone breaks on Parser/asdl_c.py
http://bugs.python.org/issue15923  closed by ezio.melotti

#15979: Improve timeit documentation
http://bugs.python.org/issue15979  closed by ezio.melotti

#16001: small ints: cache string representation
http://bugs.python.org/issue16001  closed by haypo

#16029: pickle.dumps(xrange(sys.maxsize)) produces xrange(0)
http://bugs.python.org/issue16029  closed by mark.dickinson

#16030: xrange repr broken for large arguments
http://bugs.python.org/issue16030  closed by mark.dickinson

#16034: bz2 module appears slower in Python 3.x versus Python 2.x
http://bugs.python.org/issue16034  closed by nadeem.vawda

#16036: simplify int() signature docs
http://bugs.python.org/issue16036  closed by python-dev

#16065: setup.py: passing --prefix=/ makes --root ignored
http://bugs.python.org/issue16065  closed by eric.araujo

#16071: fix link to email.message.Message in mailbox docs
http://bugs.python.org/issue16071  closed by r.david.murray

#16077: fix code example in docs for built-in reduce()
http://bugs.python.org/issue16077  closed by chris.jerdonek

#16084: couple of code examples not highlighted in "What's new in Pyth
http://bugs.python.org/issue16084  closed by mark.dickinson

#16085: Minor config parser doc change
http://bugs.python.org/issue16085  closed by r.david.murray

#16087: How to open a csv file in universal new line mode?
http://bugs.python.org/issue16087  closed by r.david.murray

#16089: _elementtree.TreeBuilder broken with a non-C-deriving element_
http://bugs.python.org/issue16089  closed by pitrou

#16090: but is needed somewhere in http://www.python.org/dev/peps/pep-
http://bugs.python.org/issue16090  closed by georg.brandl

#16091: changing sidebar links in http://docs.python.org/
http://bugs.python.org/issue16091  closed by python-dev

#16092: test_recursion_limit in test_threading fails on Mac OS X in 3.
http://bugs.python.org/issue16092  closed by ned.deily

#16093: Documentation: links to 3.3
http://bugs.python.org/issue16093  closed by georg.brandl

#16097: Minor fix in comment in codecs.py
http://bugs.python.org/issue16097  closed by tim.golden

#16109: urllib2.urlopen throws HTTP Error in python 2.7.2, 2.7.3, but 
http://bugs.python.org/issue16109  closed by pitrou

#16118: Implement SHA-3
http://bugs.python.org/issue16118  closed by brett.cannon

#16122: Allow *open* to accept file-like objects
http://bugs.python.org/issue16122  closed by benjamin.peterson

#16126: PyErr_Format format mismatch in _testcapimodule.c
http://bugs.python.org/issue16126  closed by python-dev

#16127: Correct mentions of narrow build in documentation and comments
http://bugs.python.org/issue16127  closed by ezio.melotti

#16130: "Porting to Python 3.3" needs to be fixed in "What's new In Py
http://bugs.python.org/issue16130  closed by python-dev

#16131: Pylauncher is being installed in Windows system folder
http://bugs.python.org/issue16131  closed by loewis

#16135: Removal of OS/2 support
http://bugs.python.org/issue16135  closed by python-dev

#16138: In the glossary there is a small typo about __len__() in the s
http://bugs.python.org/issue16138  closed by asvetlov

From tjreedy at udel.edu  Fri Oct  5 20:32:58 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 05 Oct 2012 14:32:58 -0400
Subject: [Python-Dev] Should vars() return modifiable dict?
In-Reply-To: <506EDA0E.9090200@hastings.org>
References: <k4gug7$e1q$1@ger.gmane.org> <506C5B09.60005@pearwood.info>
	<CABicbJLK=3444Hh1Z5Ut0ZfvTS20NzbtJRooDB-2GAJcZody2A@mail.gmail.com>
	<506EDA0E.9090200@hastings.org>
Message-ID: <506F27DA.9010507@udel.edu>

On 10/5/2012 9:01 AM, Larry Hastings wrote:
> On 10/05/2012 01:59 PM, Devin Jeanpierre wrote:
>> I've never understood why locals() returned a mutable dictionary
>> either, except that Python has no immutable dictionary type.
>
> Ah, but these days it has types.MappingProxyType which provides a
> read-only wrapper around another mapping type.  Perhaps that should be
> employed here.

I would like to put this particular tracker issue in a larger context.

I believe that the issue of expanding the meaning of vars() intersects 
with the issue of changing the return type of stat and other functions 
from a tuple or elaborated tuple to a read-only attribute namespace 
object (as I believe Larry and some others wish to do). I understand the 
proposal to be to change to something like a named tuple, with dotted 
name access to attributes, but without all the tuple behaviors.

The are two major behaviors of tuple to be considered. One is 
indexibility. That, I agree, can go. The other is iterability, which 
also means introspect-ability. That, I would argue, should not go.
However, one might want to iterate or introspect names, values, or 
pairs. We already have a way to do that -- dicts.

So I propose that the meaning of vars(ob), where ob is an object with 
attributes, should be expanded from 'return ob.__dict__, if there is 
one' to 'return a name-attribute dict for ob (which will be ob.__dict__ 
when there is one)'. I believe this is in line with the original intent 
of vars(), given that all attribute objects once had a __dict__. (I 
could be mistaken here, but my memory of 1.4 on this is a bit fuzzy.)

The addition of __slots__ broke that, but it was not the only one. 
Giving a meaning to vars(slotted-object) is a partial step in fixing the 
break, but only a partial step.

One way to expand the meaning of vars is to update it as necessary when 
new internal types or mechanisms are added. This is the proposal in the 
tracker. But this is contrary to the principle of information 
localization and using __special__ names to access type-specific behavior.

An alternative to teaching vars about various categories and classes of 
objects would be to give slotted objects, etcetera, a __dict__ property. 
Its __get__ could return a dict_proxy or whatever. This would be a 
better place to localize knowledge of how to get the name-attribute 
mapping. Then vars() would work on such objects as it currently is, 
without change. This is similar to how iter() and next() work with new 
classes -- the classes provide the necessary special method. A solution 
the to problem of an attribute object not having a __dict__ for vars() 
to return is to give it one.

Coming back to my starting point, if read-only attribute objects have a 
__dict__ property, they could be introspected and indirectly iterated 
in three different ways, without having __iter__ and __next__.

-- 
Terry Jan Reedy


From chris.jerdonek at gmail.com  Fri Oct  5 22:32:07 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Fri, 5 Oct 2012 13:32:07 -0700
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <87zk41k7rh.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<CAOTb1wd2667sHm+rPE2GB-JvjoVaX2n14C_PVwtuG3RRGCQkEQ@mail.gmail.com>
	<CAMpsgwaoM0wk3cVxNaovVAdsaa0DC4CR1xyEi-LWowWr2ZCS9Q@mail.gmail.com>
	<CAPZV6o_kZnrBvQGjpeOrUs8s2dW9FH9eOuLCT-GsKEQoBZsvKA@mail.gmail.com>
	<CAOTb1wc8h6ovNmD3ovv5HgAwHEyjd_CEsaEdzanngY3MuYuHPw@mail.gmail.com>
	<87zk41k7rh.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAOTb1wdD_ZkPT9Xf4iDLO9fufwByOuGRdgJRWTrrMkbqaxn04Q@mail.gmail.com>

On Thu, Oct 4, 2012 at 6:49 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Chris Jerdonek writes:
>
>  > You can create multiple files this way.  I just verified it.  But the
>  > problem happens with merging.  You will create merge conflicts in the
>  > deleted portions of every split file on every merge.  There may be a
>  > way to avoid this that I don't know about though (i.e. to record that
>  > merges into the deleted portions should no longer occur).
> ...
> There's no other way to do it that I know of in any VCS because they
> all track conflicts at the file level.  (It would be straightforward
> to generalize git to handle this gracefully, but it would be a hugely
> disruptive change.  I don't know if Mercurial would be susceptible to
> such an extension.)

FWIW, I filed an issue in Mercurial's tracker to add support for
splitting files and copying subsets of files:

http://bz.selenic.com/show_bug.cgi?id=3649

As I thought it might be, the idea was rejected.

--Chris

From nad at acm.org  Sat Oct  6 01:45:11 2012
From: nad at acm.org (Ned Deily)
Date: Fri, 05 Oct 2012 16:45:11 -0700
Subject: [Python-Dev] Bad python 2.5 build on OSX 10.8 mountain lion
References: <CAP7+vJLQLLgDOB83XBg9-=+FyS7+1qEkr7inAL5igD4bJHpiiA@mail.gmail.com>
	<CAP7+vJLYBgPCmKNT5UVzQ9Na3qBdumE9CmJ5PL6yZqR75AyAnw@mail.gmail.com>
	<nad-AD48B0.00182002102012@news.gmane.org>
	<20121002073135.GA26567@sleipnir.bytereef.org>
	<nad-B4E67A.00475902102012@news.gmane.org>
Message-ID: <nad-7C84D6.16451105102012@news.gmane.org>

In article <nad-B4E67A.00475902102012 at news.gmane.org>,
 Ned Deily <nad at acm.org> wrote:
> In article <20121002073135.GA26567 at sleipnir.bytereef.org>,
>  Stefan Krah <stefan at bytereef.org> wrote:
> > Ned Deily <nad at acm.org> wrote:
> > > > Forgot the link...
> > > > http://code.google.com/p/googleappengine/issues/detail?id=7885
> > > > On Monday, October 1, 2012, Guido van Rossum wrote:
> > > > > As discussed here, the python 2.5 binary distributed by Apple on 
> > > > > mountain
> > > > > lion is broken. Could someone file an official complaint?
> > > I've filed a bug against 10.8 python2.5.   The 10.8 versions of Apple's 
> > > pythons are compile with clang and we did see some sign extension issues 
> > > with ctypes.  The 10.7 version of Apple's python2.5 is compiled with 
> > > llvm-gcc and handles 2**31 correctly.
> > Yes, this looks like http://bugs.python.org/issue11149 . 
> Ah, right, thanks.  I've updated the Apple issue accordingly.

Update: the bug I filed has been closed as a duplicate of #11932488 
which apparently at the moment is still open.  No other information is 
available.

-- 
 Ned Deily,
 nad at acm.org


From g.brandl at gmx.net  Sat Oct  6 14:35:24 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 06 Oct 2012 14:35:24 +0200
Subject: [Python-Dev] cpython: Issue #16148: implemented PEP 424
In-Reply-To: <3XYmtX50syzM7Q@mail.python.org>
References: <3XYmtX50syzM7Q@mail.python.org>
Message-ID: <k4p8g7$3ib$1@ger.gmane.org>

Am 06.10.2012 14:12, schrieb armin.ronacher:
> http://hg.python.org/cpython/rev/a7ec0a1b0f7c
> changeset:   79511:a7ec0a1b0f7c
> parent:      79507:3c1df1ede882
> user:        Armin Ronacher <armin.ronacher at active-4.com>
> date:        Sat Oct 06 14:03:24 2012 +0200
> summary:
>   Issue #16148: implemented PEP 424
> 
> files:
>   Doc/c-api/object.rst       |   7 ++
>   Doc/library/operator.rst   |   6 +
>   Include/abstract.h         |   5 +-
>   Lib/test/test_enumerate.py |   9 +-
>   Lib/test/test_iterlen.py   |  62 +++++++++----------
>   Lib/test/test_itertools.py |   5 +-
>   Lib/test/test_operator.py  |  25 ++++++++
>   Lib/test/test_set.py       |   2 -
>   Modules/operator.c         |  27 ++++++++
>   Objects/abstract.c         |  80 +++++++++++++++----------
>   Objects/bytearrayobject.c  |   2 +-
>   Objects/bytesobject.c      |   2 +-
>   Objects/iterobject.c       |  11 ++-
>   Objects/listobject.c       |   2 +-
>   14 files changed, 162 insertions(+), 83 deletions(-)
> 
> 
> diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst
> --- a/Doc/c-api/object.rst
> +++ b/Doc/c-api/object.rst
> @@ -342,6 +342,13 @@
>     returned.  This is the equivalent to the Python expression ``len(o)``.
>  
>  
> +.. c:function:: Py_ssize_t PyObject_LengthHint(PyObject *o, Py_ssize_t default)
> +
> +   Return an estimated length for the object *o*. First trying to return its
> +   actual length, then an estimate using ``__length_hint__``, and finally
> +   returning the default value. On error ``-1`` is returned. This is the
> +   equivalent to the Python expression ``operator.length_hint(o, default)``.

Needs a versionadded.

Since __length_hint__ is now official, it needs an entry in
Doc/reference/datamodel.rst (which you can link to here.)

>  .. c:function:: PyObject* PyObject_GetItem(PyObject *o, PyObject *key)
>  
>     Return element of *o* corresponding to the object *key* or *NULL* on failure.
> diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst
> --- a/Doc/library/operator.rst
> +++ b/Doc/library/operator.rst
> @@ -235,6 +235,12 @@
>  
>  .. XXX: find a better, readable, example
>  
> +.. function:: length_hint(obj, default=0)
> +
> +   Return an estimated length for the object *o*. First trying to return its
> +   actual length, then an estimate using ``__length_hint__``, and finally
> +   returning the default value.

This one also needs versionadded and a link to __length_hint__.

>  The :mod:`operator` module also defines tools for generalized attribute and item
>  lookups.  These are useful for making fast field extractors as arguments for
>  :func:`map`, :func:`sorted`, :meth:`itertools.groupby`, or other functions that
> diff --git a/Include/abstract.h b/Include/abstract.h
> --- a/Include/abstract.h
> +++ b/Include/abstract.h
> @@ -403,9 +403,8 @@
>       PyAPI_FUNC(Py_ssize_t) PyObject_Length(PyObject *o);
>  #define PyObject_Length PyObject_Size
>  
> -#ifndef Py_LIMITED_API
> -     PyAPI_FUNC(Py_ssize_t) _PyObject_LengthHint(PyObject *o, Py_ssize_t);
> -#endif
> +PyAPI_FUNC(int) _PyObject_HasLen(PyObject *o);
> +PyAPI_FUNC(Py_ssize_t) PyObject_LengthHint(PyObject *o, Py_ssize_t);

Not sure if new functions should be included in the limited API.  I seem to
recall some discussion about giving Py_LIMITED_API a numeric value for the
required API version?  PEP 384 is silent about it.

>         /*
>       Guess the size of object o using len(o) or o.__length_hint__().
> diff --git a/Lib/test/test_enumerate.py b/Lib/test/test_enumerate.py
> --- a/Lib/test/test_enumerate.py
> +++ b/Lib/test/test_enumerate.py
> @@ -1,4 +1,5 @@
>  import unittest
> +import operator
>  import sys
>  import pickle
>  
> @@ -168,15 +169,13 @@
>          x = range(1)
>          self.assertEqual(type(reversed(x)), type(iter(x)))
>  
> -    @support.cpython_only
>      def test_len(self):
>          # This is an implementation detail, not an interface requirement

If it's not cpython_only anymore, this comment should also vanish?

> -        from test.test_iterlen import len
>          for s in ('hello', tuple('hello'), list('hello'), range(5)):
> -            self.assertEqual(len(reversed(s)), len(s))
> +            self.assertEqual(operator.length_hint(reversed(s)), len(s))
>              r = reversed(s)
>              list(r)
> -            self.assertEqual(len(r), 0)
> +            self.assertEqual(operator.length_hint(r), 0)
>          class SeqWithWeirdLen:
>              called = False
>              def __len__(self):
> @@ -187,7 +186,7 @@
>              def __getitem__(self, index):
>                  return index
>          r = reversed(SeqWithWeirdLen())
> -        self.assertRaises(ZeroDivisionError, len, r)
> +        self.assertRaises(ZeroDivisionError, operator.length_hint, r)
>  
>  
>      def test_gc(self):
> diff --git a/Lib/test/test_iterlen.py b/Lib/test/test_iterlen.py
> --- a/Lib/test/test_iterlen.py
> +++ b/Lib/test/test_iterlen.py
> @@ -45,31 +45,21 @@
>  from test import support
>  from itertools import repeat
>  from collections import deque
> -from builtins import len as _len
> +from operator import length_hint

>  ## ------- Concrete Type Tests -------
>  
> @@ -92,10 +82,6 @@
>      def setUp(self):
>          self.it = repeat(None, n)
>  
> -    def test_no_len_for_infinite_repeat(self):
> -        # The repeat() object can also be infinite
> -        self.assertRaises(TypeError, len, repeat(None))

Why is this removed?  I can see it was duplicated in test_itertools (below),
but you removed both instances.


> diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py
> --- a/Lib/test/test_itertools.py
> +++ b/Lib/test/test_itertools.py
> @@ -1723,9 +1723,8 @@
>  class LengthTransparency(unittest.TestCase):
>  
>      def test_repeat(self):
> -        from test.test_iterlen import len
> -        self.assertEqual(len(repeat(None, 50)), 50)
> -        self.assertRaises(TypeError, len, repeat(None))
> +        self.assertEqual(operator.length_hint(repeat(None, 50)), 50)
> +        self.assertEqual(operator.length_hint(repeat(None), 12), 12)
>  
>  class RegressionTests(unittest.TestCase):
>  
> diff --git a/Lib/test/test_operator.py b/Lib/test/test_operator.py
> --- a/Lib/test/test_operator.py
> +++ b/Lib/test/test_operator.py
> @@ -410,6 +410,31 @@
>          self.assertEqual(operator.__ixor__     (c, 5), "ixor")
>          self.assertEqual(operator.__iconcat__  (c, c), "iadd")
>  
> +    def test_length_hint(self):
> +        class X(object):
> +            def __init__(self, value):
> +                self.value = value
> +
> +            def __length_hint__(self):
> +                if type(self.value) is type:
> +                    raise self.value
> +                else:
> +                    return self.value
> +
> +        self.assertEqual(operator.length_hint([], 2), 0)
> +        self.assertEqual(operator.length_hint(iter([1, 2, 3])), 3)
> +
> +        self.assertEqual(operator.length_hint(X(2)), 2)
> +        self.assertEqual(operator.length_hint(X(NotImplemented), 4), 4)
> +        self.assertEqual(operator.length_hint(X(TypeError), 12), 12)
> +        with self.assertRaises(TypeError):
> +            operator.length_hint(X("abc"))
> +        with self.assertRaises(ValueError):
> +            operator.length_hint(X(-2))
> +        with self.assertRaises(LookupError):
> +            operator.length_hint(X(LookupError))
> +
> +
>  def test_main(verbose=None):
>      import sys
>      test_classes = (
> diff --git a/Lib/test/test_set.py b/Lib/test/test_set.py
> --- a/Lib/test/test_set.py
> +++ b/Lib/test/test_set.py
> @@ -848,8 +848,6 @@
>          for v in self.set:
>              self.assertIn(v, self.values)
>          setiter = iter(self.set)
> -        # note: __length_hint__ is an internal undocumented API,
> -        # don't rely on it in your own programs
>          self.assertEqual(setiter.__length_hint__(), len(self.set))
>  
>      def test_pickling(self):
> diff --git a/Modules/operator.c b/Modules/operator.c
> --- a/Modules/operator.c
> +++ b/Modules/operator.c
> @@ -208,6 +208,31 @@
>      return (result == 0);
>  }
>  
> +PyDoc_STRVAR(length_hint__doc__,
> +"length_hint(obj, default=0) -> int\n"
> +"Return an estimate of the number of items in obj.\n"
> +"This is useful for presizing containers when building from an\n"
> +"iterable.\n"
> +"\n"
> +"If the object supports len(), the result will be\n"
> +"exact. Otherwise, it may over- or under-estimate by an\n"
> +"arbitrary amount. The result will be an integer >= 0.");
> +
> +static PyObject *length_hint(PyObject *self, PyObject *args)
> +{
> +    PyObject *obj;
> +    Py_ssize_t defaultvalue = 0, res;
> +    if (!PyArg_ParseTuple(args, "O|n:length_hint", &obj, &defaultvalue)) {
> +        return NULL;
> +    }
> +    res = PyObject_LengthHint(obj, defaultvalue);
> +    if (res == -1 && PyErr_Occurred()) {
> +        return NULL;
> +    }
> +    return PyLong_FromSsize_t(res);
> +}
> +
> +
>  PyDoc_STRVAR(compare_digest__doc__,
>  "compare_digest(a, b) -> bool\n"
>  "\n"
> @@ -366,6 +391,8 @@
>  
>      {"_compare_digest", (PyCFunction)compare_digest, METH_VARARGS,
>       compare_digest__doc__},
> +     {"length_hint", (PyCFunction)length_hint, METH_VARARGS,
> +     length_hint__doc__},
>      {NULL,              NULL}           /* sentinel */
>  
>  };
> diff --git a/Objects/abstract.c b/Objects/abstract.c
> --- a/Objects/abstract.c
> +++ b/Objects/abstract.c
> @@ -64,49 +64,67 @@
>  }
>  #define PyObject_Length PyObject_Size
>  
> +int
> +_PyObject_HasLen(PyObject *o) {
> +    return (Py_TYPE(o)->tp_as_sequence && Py_TYPE(o)->tp_as_sequence->sq_length) ||
> +        (Py_TYPE(o)->tp_as_mapping && Py_TYPE(o)->tp_as_mapping->mp_length);
> +}
>  
>  /* The length hint function returns a non-negative value from o.__len__()
> -   or o.__length_hint__().  If those methods aren't found or return a negative
> -   value, then the defaultvalue is returned.  If one of the calls fails,
> -   this function returns -1.
> +   or o.__length_hint__().  If those methods aren't found.  If one of the calls
                                                          ^^^^^
Sentence incomplete.

> +   fails this function returns -1.
>  */
>  
>  Py_ssize_t
> -_PyObject_LengthHint(PyObject *o, Py_ssize_t defaultvalue)
> +PyObject_LengthHint(PyObject *o, Py_ssize_t defaultvalue)

Should we put "#define _PyObject_LengthHint PyObject_LengthHint" in some header
to make porting easier?

>  {
>      _Py_IDENTIFIER(__length_hint__);
> -    PyObject *ro, *hintmeth;
> -    Py_ssize_t rv;
> -
> -    /* try o.__len__() */
> -    rv = PyObject_Size(o);
> -    if (rv >= 0)
> -        return rv;
> -    if (PyErr_Occurred()) {
> -        if (!PyErr_ExceptionMatches(PyExc_TypeError))
> +    Py_ssize_t res = PyObject_Length(o);
> +    if (res < 0 && PyErr_Occurred()) {
> +        if (!PyErr_ExceptionMatches(PyExc_TypeError)) {
>              return -1;
> +        }
>          PyErr_Clear();
>      }
> -
> -    /* try o.__length_hint__() */
> -    hintmeth = _PyObject_LookupSpecial(o, &PyId___length_hint__);
> -    if (hintmeth == NULL) {
> -        if (PyErr_Occurred())
> +    else {
> +        return res;
> +    }
> +    PyObject *hint = _PyObject_LookupSpecial(o, &PyId___length_hint__);
> +    if (hint == NULL) {
> +        if (PyErr_Occurred()) {
>              return -1;
> -        else
> -            return defaultvalue;
> -    }
> -    ro = PyObject_CallFunctionObjArgs(hintmeth, NULL);
> -    Py_DECREF(hintmeth);
> -    if (ro == NULL) {
> -        if (!PyErr_ExceptionMatches(PyExc_TypeError))
> -            return -1;
> -        PyErr_Clear();
> +        }
>          return defaultvalue;
>      }
> -    rv = PyLong_Check(ro) ? PyLong_AsSsize_t(ro) : defaultvalue;
> -    Py_DECREF(ro);
> -    return rv;
> +    PyObject *result = PyObject_CallFunctionObjArgs(hint, NULL);
> +    Py_DECREF(hint);
> +    if (result == NULL) {
> +        if (PyErr_ExceptionMatches(PyExc_TypeError)) {
> +            PyErr_Clear();
> +            return defaultvalue;
> +        }
> +        return -1;
> +    }
> +    else if (result == Py_NotImplemented) {
> +        Py_DECREF(result);
> +        return defaultvalue;
> +    }
> +    if (!PyLong_Check(result)) {
> +        PyErr_Format(PyExc_TypeError, "Length hint must be an integer, not %s",

We usually limit the string size here, e.g. "%.100s".

> +            Py_TYPE(result)->tp_name);
> +        Py_DECREF(result);
> +        return -1;
> +    }
> +    defaultvalue = PyLong_AsSsize_t(result);

Not sure the micro-optimization is worth the confusion of reassigning defaultvalue.

> +    Py_DECREF(result);
> +    if (defaultvalue < 0 && PyErr_Occurred()) {
> +        return -1;
> +    }
> +    if (defaultvalue < 0) {
> +        PyErr_Format(PyExc_ValueError, "__length_hint__() should return >= 0");

Exception message is inconsistent with above: "Length hint" vs "__length_hint__()".

> +        return -1;
> +    }
> +    return defaultvalue;
>  }

> diff --git a/Objects/iterobject.c b/Objects/iterobject.c
> --- a/Objects/iterobject.c
> +++ b/Objects/iterobject.c
> @@ -76,9 +76,14 @@
>      Py_ssize_t seqsize, len;
>  
>      if (it->it_seq) {
> -        seqsize = PySequence_Size(it->it_seq);
> -        if (seqsize == -1)
> -            return NULL;
> +        if (_PyObject_HasLen(it->it_seq)) {
> +            seqsize = PySequence_Size(it->it_seq);
> +            if (seqsize == -1)
> +                return NULL;
> +        }
> +        else {
> +            return Py_NotImplemented;

An INCREF is missing here, as discussed on IRC.

> +        }
>          len = seqsize - it->it_index;
>          if (len >= 0)
>              return PyLong_FromSsize_t(len);



From solipsis at pitrou.net  Sat Oct  6 14:37:02 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 6 Oct 2012 14:37:02 +0200
Subject: [Python-Dev] cpython: Issue #16148: implemented PEP 424
References: <3XYmtX50syzM7Q@mail.python.org>
Message-ID: <20121006143702.12ebbb9d@pitrou.net>

On Sat,  6 Oct 2012 14:12:36 +0200 (CEST)
armin.ronacher <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/a7ec0a1b0f7c
> changeset:   79511:a7ec0a1b0f7c
> parent:      79507:3c1df1ede882
> user:        Armin Ronacher <armin.ronacher at active-4.com>
> date:        Sat Oct 06 14:03:24 2012 +0200
> summary:
>   Issue #16148: implemented PEP 424
> 
[...]
>  
> +.. c:function:: Py_ssize_t PyObject_LengthHint(PyObject *o, Py_ssize_t default)
> +
> +   Return an estimated length for the object *o*. First trying to return its
> +   actual length, then an estimate using ``__length_hint__``, and finally
> +   returning the default value. On error ``-1`` is returned. This is the
> +   equivalent to the Python expression ``operator.length_hint(o, default)``.
> +

You need a "versionadded" marker.

> +.. function:: length_hint(obj, default=0)
> +
> +   Return an estimated length for the object *o*. First trying to return its
> +   actual length, then an estimate using ``__length_hint__``, and finally
> +   returning the default value.
> +

Here as well.

> -#ifndef Py_LIMITED_API
> -     PyAPI_FUNC(Py_ssize_t) _PyObject_LengthHint(PyObject *o, Py_ssize_t);
> -#endif
> +PyAPI_FUNC(int) _PyObject_HasLen(PyObject *o);
> +PyAPI_FUNC(Py_ssize_t) PyObject_LengthHint(PyObject *o, Py_ssize_t);

If _PyObject_HasLen is private, it shouldn't be in the Py_LIMITED_API.

> diff --git a/Lib/test/test_enumerate.py b/Lib/test/test_enumerate.py
> --- a/Lib/test/test_enumerate.py
> +++ b/Lib/test/test_enumerate.py
> @@ -1,4 +1,5 @@
>  import unittest
> +import operator
>  import sys
>  import pickle
>  
> @@ -168,15 +169,13 @@
>          x = range(1)
>          self.assertEqual(type(reversed(x)), type(iter(x)))
>  
> -    @support.cpython_only
>      def test_len(self):
>          # This is an implementation detail, not an interface requirement
> -        from test.test_iterlen import len
>          for s in ('hello', tuple('hello'), list('hello'), range(5)):
> -            self.assertEqual(len(reversed(s)), len(s))
> +            self.assertEqual(operator.length_hint(reversed(s)), len(s))

You should still test len() as well, no?

> +    else {
> +        return res;
> +    }
> +    PyObject *hint = _PyObject_LookupSpecial(o, &PyId___length_hint__);

Putting variable declarations in the middle of code blocks is not
C89-compliant. You probably broke a couple of buildbots :-)

> +        if (_PyObject_HasLen(it->it_seq)) {
> +            seqsize = PySequence_Size(it->it_seq);
> +            if (seqsize == -1)
> +                return NULL;
> +        }
> +        else {
> +            return Py_NotImplemented;
> +        }

Lacks a Py_INCREF?

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From guido at python.org  Sat Oct  6 19:48:06 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 6 Oct 2012 10:48:06 -0700
Subject: [Python-Dev] Bad python 2.5 build on OSX 10.8 mountain lion
In-Reply-To: <nad-7C84D6.16451105102012@news.gmane.org>
References: <CAP7+vJLQLLgDOB83XBg9-=+FyS7+1qEkr7inAL5igD4bJHpiiA@mail.gmail.com>
	<CAP7+vJLYBgPCmKNT5UVzQ9Na3qBdumE9CmJ5PL6yZqR75AyAnw@mail.gmail.com>
	<nad-AD48B0.00182002102012@news.gmane.org>
	<20121002073135.GA26567@sleipnir.bytereef.org>
	<nad-B4E67A.00475902102012@news.gmane.org>
	<nad-7C84D6.16451105102012@news.gmane.org>
Message-ID: <CAP7+vJKeXqYyvfwipOR+7yTdtUs2eDhvybV3tyZb3ZkX6hta5g@mail.gmail.com>

On Fri, Oct 5, 2012 at 4:45 PM, Ned Deily <nad at acm.org> wrote:
> In article <nad-B4E67A.00475902102012 at news.gmane.org>,
>  Ned Deily <nad at acm.org> wrote:
>> In article <20121002073135.GA26567 at sleipnir.bytereef.org>,
>>  Stefan Krah <stefan at bytereef.org> wrote:
>> > Ned Deily <nad at acm.org> wrote:
>> > > > Forgot the link...
>> > > > http://code.google.com/p/googleappengine/issues/detail?id=7885
>> > > > On Monday, October 1, 2012, Guido van Rossum wrote:
>> > > > > As discussed here, the python 2.5 binary distributed by Apple on
>> > > > > mountain
>> > > > > lion is broken. Could someone file an official complaint?
>> > > I've filed a bug against 10.8 python2.5.   The 10.8 versions of Apple's
>> > > pythons are compile with clang and we did see some sign extension issues
>> > > with ctypes.  The 10.7 version of Apple's python2.5 is compiled with
>> > > llvm-gcc and handles 2**31 correctly.
>> > Yes, this looks like http://bugs.python.org/issue11149 .
>> Ah, right, thanks.  I've updated the Apple issue accordingly.
>
> Update: the bug I filed has been closed as a duplicate of #11932488
> which apparently at the moment is still open.  No other information is
> available.

Thanks Ned! Is there any way that I could see that bug myself and
attach myself to updates? Otherwise, can you keep us here appraised of
developments (even if Apple decides not to fix it)?

-- 
--Guido van Rossum (python.org/~guido)

From nad at acm.org  Sun Oct  7 00:42:42 2012
From: nad at acm.org (Ned Deily)
Date: Sat, 06 Oct 2012 15:42:42 -0700
Subject: [Python-Dev] Bad python 2.5 build on OSX 10.8 mountain lion
References: <CAP7+vJLQLLgDOB83XBg9-=+FyS7+1qEkr7inAL5igD4bJHpiiA@mail.gmail.com>
	<CAP7+vJLYBgPCmKNT5UVzQ9Na3qBdumE9CmJ5PL6yZqR75AyAnw@mail.gmail.com>
	<nad-AD48B0.00182002102012@news.gmane.org>
	<20121002073135.GA26567@sleipnir.bytereef.org>
	<nad-B4E67A.00475902102012@news.gmane.org>
	<nad-7C84D6.16451105102012@news.gmane.org>
	<CAP7+vJKeXqYyvfwipOR+7yTdtUs2eDhvybV3tyZb3ZkX6hta5g@mail.gmail.com>
Message-ID: <nad-86D1EA.15424206102012@news.gmane.org>

In article 
<CAP7+vJKeXqYyvfwipOR+7yTdtUs2eDhvybV3tyZb3ZkX6hta5g at mail.gmail.com>,
 Guido van Rossum <guido at python.org> wrote:
> On Fri, Oct 5, 2012 at 4:45 PM, Ned Deily <nad at acm.org> wrote:
> > In article <nad-B4E67A.00475902102012 at news.gmane.org>,
> > Update: the bug I filed has been closed as a duplicate of #11932488
> > which apparently at the moment is still open.  No other information is
> > available.
> 
> Thanks Ned! Is there any way that I could see that bug myself and
> attach myself to updates? Otherwise, can you keep us here appraised of
> developments (even if Apple decides not to fix it)?

I don't think there's any way to see any bug other than ones you have 
submitted yourself.  All I can see is that the bug I submitted is closed 
as a duplicate and now has a Related Problem section that only gives the 
other incident number and its status (Open).  I can't view anything else 
about that other incident.  I don't know if I'll get an email update if 
its status changes.  I'll keep an eye on mine and perhaps ask for a 
status update if nothing changes in a few weeks.

-- 
 Ned Deily,
 nad at acm.org


From guido at python.org  Sun Oct  7 05:47:54 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 6 Oct 2012 20:47:54 -0700
Subject: [Python-Dev] Bad python 2.5 build on OSX 10.8 mountain lion
In-Reply-To: <nad-86D1EA.15424206102012@news.gmane.org>
References: <CAP7+vJLQLLgDOB83XBg9-=+FyS7+1qEkr7inAL5igD4bJHpiiA@mail.gmail.com>
	<CAP7+vJLYBgPCmKNT5UVzQ9Na3qBdumE9CmJ5PL6yZqR75AyAnw@mail.gmail.com>
	<nad-AD48B0.00182002102012@news.gmane.org>
	<20121002073135.GA26567@sleipnir.bytereef.org>
	<nad-B4E67A.00475902102012@news.gmane.org>
	<nad-7C84D6.16451105102012@news.gmane.org>
	<CAP7+vJKeXqYyvfwipOR+7yTdtUs2eDhvybV3tyZb3ZkX6hta5g@mail.gmail.com>
	<nad-86D1EA.15424206102012@news.gmane.org>
Message-ID: <CAP7+vJJ+SB1d2_bLVQUSpneyWOHKGxLvh0XAr=A1UvZna1v_nA@mail.gmail.com>

Thanks, Ned. Do you think it would be worth it to use our contacts at Apple
to raise the priority of this embarrassment?

On Saturday, October 6, 2012, Ned Deily wrote:

> In article
> <CAP7+vJKeXqYyvfwipOR+7yTdtUs2eDhvybV3tyZb3ZkX6hta5g at mail.gmail.com<javascript:;>
> >,
>  Guido van Rossum <guido at python.org <javascript:;>> wrote:
> > On Fri, Oct 5, 2012 at 4:45 PM, Ned Deily <nad at acm.org <javascript:;>>
> wrote:
> > > In article <nad-B4E67A.00475902102012 at news.gmane.org <javascript:;>>,
> > > Update: the bug I filed has been closed as a duplicate of #11932488
> > > which apparently at the moment is still open.  No other information is
> > > available.
> >
> > Thanks Ned! Is there any way that I could see that bug myself and
> > attach myself to updates? Otherwise, can you keep us here appraised of
> > developments (even if Apple decides not to fix it)?
>
> I don't think there's any way to see any bug other than ones you have
> submitted yourself.  All I can see is that the bug I submitted is closed
> as a duplicate and now has a Related Problem section that only gives the
> other incident number and its status (Open).  I can't view anything else
> about that other incident.  I don't know if I'll get an email update if
> its status changes.  I'll keep an eye on mine and perhaps ask for a
> status update if nothing changes in a few weeks.
>
> --
>  Ned Deily,
>  nad at acm.org <javascript:;>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org <javascript:;>
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121006/a055f4c6/attachment.html>

From nad at acm.org  Sun Oct  7 06:11:56 2012
From: nad at acm.org (Ned Deily)
Date: Sat, 6 Oct 2012 21:11:56 -0700
Subject: [Python-Dev] Bad python 2.5 build on OSX 10.8 mountain lion
In-Reply-To: <CAP7+vJJ+SB1d2_bLVQUSpneyWOHKGxLvh0XAr=A1UvZna1v_nA@mail.gmail.com>
References: <CAP7+vJLQLLgDOB83XBg9-=+FyS7+1qEkr7inAL5igD4bJHpiiA@mail.gmail.com>
	<CAP7+vJLYBgPCmKNT5UVzQ9Na3qBdumE9CmJ5PL6yZqR75AyAnw@mail.gmail.com>
	<nad-AD48B0.00182002102012@news.gmane.org>
	<20121002073135.GA26567@sleipnir.bytereef.org>
	<nad-B4E67A.00475902102012@news.gmane.org>
	<nad-7C84D6.16451105102012@news.gmane.org>
	<CAP7+vJKeXqYyvfwipOR+7yTdtUs2eDhvybV3tyZb3ZkX6hta5g@mail.gmail.com>
	<nad-86D1EA.15424206102012@news.gmane.org>
	<CAP7+vJJ+SB1d2_bLVQUSpneyWOHKGxLvh0XAr=A1UvZna1v_nA@mail.gmail.com>
Message-ID: <EDE22476-E877-4E78-8DF8-33A5FC8E4802@acm.org>


On Oct 6, 2012, at 20:47 , Guido van Rossum <guido at python.org> wrote:
> On Saturday, October 6, 2012, Ned Deily wrote:
> In article
>> <CAP7+vJKeXqYyvfwipOR+7yTdtUs2eDhvybV3tyZb3ZkX6hta5g at mail.gmail.com>,
>>  Guido van Rossum <guido at python.org> wrote:
>> > On Fri, Oct 5, 2012 at 4:45 PM, Ned Deily <nad at acm.org> wrote:
>> > > In article <nad-B4E67A.00475902102012 at news.gmane.org>,
>> > > Update: the bug I filed has been closed as a duplicate of #11932488
>> > > which apparently at the moment is still open.  No other information is
>> > > available.
>> >
>> > Thanks Ned! Is there any way that I could see that bug myself and
>> > attach myself to updates? Otherwise, can you keep us here appraised of
>> > developments (even if Apple decides not to fix it)?
>> 
>> I don't think there's any way to see any bug other than ones you have
>> submitted yourself.  All I can see is that the bug I submitted is closed
>> as a duplicate and now has a Related Problem section that only gives the
>> other incident number and its status (Open).  I can't view anything else
>> about that other incident.  I don't know if I'll get an email update if
>> its status changes.  I'll keep an eye on mine and perhaps ask for a
>> status update if nothing changes in a few weeks.
> Thanks, Ned. Do you think it would be worth it to use our contacts at Apple to raise the priority of this embarrassment?

It might help if anyone wants to try.  On the other hand, Python 2.5 is probably not the highest priority for Apple since they also ship Python 2.6 and 2.7 with 10.8, neither of which have this problem.  I'll see if I can get an official status update on mine (#12411585).

--
  Ned Deily
  nad at acm.org -- []



From victor.stinner at gmail.com  Sun Oct  7 23:09:34 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Sun, 7 Oct 2012 23:09:34 +0200
Subject: [Python-Dev] Issue #16153: PyUnicode_FromFormatV() must fail if the
 format string is invalid
Message-ID: <CAMpsgwaOHQQ39sSE10Y5mJQi+Wqs9GpWfb+vtjcT0T0rvPkvDw@mail.gmail.com>

Hi,

Does anyone know if there is a good reason why PyUnicode_FromFormatV()
does not fail on invalid format string, but copies the format string
as-if into the result?

I would like to change this to raise a ValueError instead.
PyUnicode_FromFormatV() is used by PyErr_Format(), so PyErr_Format()
may now fail with a ValueError if the format string is invalid.

Victor

From victor.stinner at gmail.com  Sun Oct  7 23:17:27 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Sun, 7 Oct 2012 23:17:27 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <20121004234657.Horde.McWZBML8999QbgPRvu5ECIA@webmail.df.eu>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<20121004234657.Horde.McWZBML8999QbgPRvu5ECIA@webmail.df.eu>
Message-ID: <CAMpsgwZfg3pmOd868Ag3gYg2--Coy-g4USUAA8rOxj2VSYX6WQ@mail.gmail.com>

> The amount of code will not be reduced, but now you also need to guess what
> file some piece of functionality may be in.

How do you search a piece of code? If you search for a function by its
name, it does not matter in which file it is defined if you an IDE or
vim/emacs with a correct configuration. For example, I type ":tag
PyUnicode_Format" to go to the PyUnicode_Format() function.

> Instead of having my text editor
> (Emacs) search in one file, it will have to search across multiple files -
> but not across all open buffers, but only some of them (since I will have
> many other source files open as well).

Does it mean that it would be more practical to merge all C files into
one unique file?

> I really fail to see what problem people have with large source files.
> What is it that you want to do that can be done easier if it's multiple
> files?

Another problem with huge files is to handle "dependencies" with
static functions. If the function A calls the function B which calls
the function C, you have to order A, B and C "correctly" if these
functions are private and not declared at the top of the file.

If functions are grouped correctly, you just lhave to add the function
to the right file, or reorder the files.

I also prefer short files beacuse it's easier to review/audit a small
file. My brain cannot store too many functions :-)

Victor

From benjamin at python.org  Sun Oct  7 23:26:04 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 7 Oct 2012 17:26:04 -0400
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAMpsgwZfg3pmOd868Ag3gYg2--Coy-g4USUAA8rOxj2VSYX6WQ@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<20121004234657.Horde.McWZBML8999QbgPRvu5ECIA@webmail.df.eu>
	<CAMpsgwZfg3pmOd868Ag3gYg2--Coy-g4USUAA8rOxj2VSYX6WQ@mail.gmail.com>
Message-ID: <CAPZV6o9fnBMmufS-X+p5UWawuxLa0+BB8fXQ+HFh-uNnjKxOjQ@mail.gmail.com>

2012/10/7 Victor Stinner <victor.stinner at gmail.com>:
> Another problem with huge files is to handle "dependencies" with
> static functions. If the function A calls the function B which calls
> the function C, you have to order A, B and C "correctly" if these
> functions are private and not declared at the top of the file.

Having separate files doesn't alleviate this, though. If they're in
separate files, you have to have header files of prototypes.



-- 
Regards,
Benjamin

From rosuav at gmail.com  Sun Oct  7 23:33:57 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 8 Oct 2012 08:33:57 +1100
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAMpsgwZfg3pmOd868Ag3gYg2--Coy-g4USUAA8rOxj2VSYX6WQ@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<20121004234657.Horde.McWZBML8999QbgPRvu5ECIA@webmail.df.eu>
	<CAMpsgwZfg3pmOd868Ag3gYg2--Coy-g4USUAA8rOxj2VSYX6WQ@mail.gmail.com>
Message-ID: <CAPTjJmrC0ZLWxfCcV26PeVCyYT8zXiP-2FUGfqtFcF4QpxZ6vA@mail.gmail.com>

On Mon, Oct 8, 2012 at 8:17 AM, Victor Stinner <victor.stinner at gmail.com> wrote:
> Another problem with huge files is to handle "dependencies" with
> static functions. If the function A calls the function B which calls
> the function C, you have to order A, B and C "correctly" if these
> functions are private and not declared at the top of the file.
>
> If functions are grouped correctly, you just lhave to add the function
> to the right file, or reorder the files.

This isn't a fundamental problem, since you can always declare a
private function if it's mutually recursive with another private
function. But - forgive me if this is false in CPython - this isn't
usually that common. Also, ordering the functions in (at least an
approximation of) Define Before Use makes it easy to locate the one
you're calling, even in a non-smart editor: just go to the top of the
file and search for the function's name; the first hit will be the
definition. It's not usually difficult to sort functions
appropriately, and can pay dividends in readability.

ChrisA

From martin at v.loewis.de  Sun Oct  7 23:34:10 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Sun, 07 Oct 2012 23:34:10 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles?
In-Reply-To: <CAMpsgwZfg3pmOd868Ag3gYg2--Coy-g4USUAA8rOxj2VSYX6WQ@mail.gmail.com>
References: <CAMpsgwajAtvqiBa7ruD3rX39BC7ZCgNuyq6vza0yZ77t6icx5A@mail.gmail.com>
	<20121004234657.Horde.McWZBML8999QbgPRvu5ECIA@webmail.df.eu>
	<CAMpsgwZfg3pmOd868Ag3gYg2--Coy-g4USUAA8rOxj2VSYX6WQ@mail.gmail.com>
Message-ID: <20121007233410.Horde.WLYhJ8L8999QcfVSe93X9TA@webmail.df.eu>


Zitat von Victor Stinner <victor.stinner at gmail.com>:

>> The amount of code will not be reduced, but now you also need to guess what
>> file some piece of functionality may be in.
>
> How do you search a piece of code?

I type /<pattern> in vim, or Ctrl-s (incremental search) in Emacs.

> If you search for a function by its
> name, it does not matter in which file it is defined if you an IDE or
> vim/emacs with a correct configuration. For example, I type ":tag
> PyUnicode_Format" to go to the PyUnicode_Format() function.

I don't like tag files. I want to search in all source code (including
comments and strings), and I want to do a substring search (not sure
whether that is supported in tag files).

>> Instead of having my text editor
>> (Emacs) search in one file, it will have to search across multiple files -
>> but not across all open buffers, but only some of them (since I will have
>> many other source files open as well).
>
> Does it mean that it would be more practical to merge all C files into
> one unique file?

That would be extreme, of course. It may cause problems with the
responsiveness of the editor, and with compile times; it may also cause
problems with merging in version control. In addition, there might
be naming conflicts which make it impractical (e.g. many structures
containing the same tp_* struct slots, so when you search for tp_new,
for example, you would get too many hits).

But in principle, I don't mind maintaining *very* large source files.
unicodeobject.c isn't really *that* large.


>> What is it that you want to do that can be done easier if it's multiple
>> files?
>
> Another problem with huge files is to handle "dependencies" with
> static functions. If the function A calls the function B which calls
> the function C, you have to order A, B and C "correctly" if these
> functions are private and not declared at the top of the file.
>
> If functions are grouped correctly, you just lhave to add the function
> to the right file, or reorder the files.

I don't understand. Do you envision that A, B, and C are in separate files?
If so, they cannot be all static anymore, unless you still combine all files
with #include directives, or unless you put them still all in the same file.
I don't see how multiple files gives any improvement. It seems to make matters
worse:
- if you put A, B, C in the same file, you have the same issue that you
   had when unicodeobject.c was a large file - you have to order them
   "correctly".
- if you put them in different files, it gets worse: you need to place
   A in a file that gets included after the file that has B, even if it
   would be more logical to put them reverse.

> I also prefer short files beacuse it's easier to review/audit a small
> file. My brain cannot store too many functions :-)

This is what I don't understand. Why do you have to remember all functions
when reviewing or auditing a file? You can safely ignore all functions
but the one you are reviewing - whether the other functions are in a different
file or in the same file.

Why can you ignore the functions only if they are stored in a different
file?

Regards,
Martin




From steve at pearwood.info  Mon Oct  8 00:47:37 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 08 Oct 2012 09:47:37 +1100
Subject: [Python-Dev] Issue 8492 [was Re: [Python-dev] History stepping in
 interactive session?]
In-Reply-To: <506EEDA7.9000108@pearwood.info>
References: <506EA800.1080106@insectnation.org>
	<20121005140927.759293ed@pitrou.net>
	<506EEDA7.9000108@pearwood.info>
Message-ID: <50720689.4020105@pearwood.info>

Over on python-ideas, a question about readline was raised and, I think,
resolved. But while investigating the question, it became obvious to me
that the ability to inspect the current readline bindings from Python
was both useful and important.

I wrote:

> I don't believe that there is any direct mechanism for querying the current
> readline bindings in Python,

But it was requested some time ago: http://bugs.python.org/issue8492


Is there anyone willing and able to give this issue some attention please?

(Replies to python-dev only please.)



-- 
Steven

From barry at python.org  Mon Oct  8 16:19:54 2012
From: barry at python.org (Barry Warsaw)
Date: Mon, 8 Oct 2012 10:19:54 -0400
Subject: [Python-Dev] Issue #16153: PyUnicode_FromFormatV() must fail if
 the format string is invalid
In-Reply-To: <CAMpsgwaOHQQ39sSE10Y5mJQi+Wqs9GpWfb+vtjcT0T0rvPkvDw@mail.gmail.com>
References: <CAMpsgwaOHQQ39sSE10Y5mJQi+Wqs9GpWfb+vtjcT0T0rvPkvDw@mail.gmail.com>
Message-ID: <20121008101954.2e433a2a@limelight.wooz.org>

On Oct 07, 2012, at 11:09 PM, Victor Stinner wrote:

>Does anyone know if there is a good reason why PyUnicode_FromFormatV()
>does not fail on invalid format string, but copies the format string
>as-if into the result?
>
>I would like to change this to raise a ValueError instead.
>PyUnicode_FromFormatV() is used by PyErr_Format(), so PyErr_Format()
>may now fail with a ValueError if the format string is invalid.

This behavior is inherited from PyUnicode_FromFormat() which documents the
current behavior, so you can't change this in backward incompatible way.

Cheers,
-Barry

From victor.stinner at gmail.com  Mon Oct  8 23:23:59 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Mon, 8 Oct 2012 23:23:59 +0200
Subject: [Python-Dev] Issue #16153: PyUnicode_FromFormatV() must fail if
 the format string is invalid
In-Reply-To: <20121008101954.2e433a2a@limelight.wooz.org>
References: <CAMpsgwaOHQQ39sSE10Y5mJQi+Wqs9GpWfb+vtjcT0T0rvPkvDw@mail.gmail.com>
	<20121008101954.2e433a2a@limelight.wooz.org>
Message-ID: <CAMpsgwatB3kowi55coz7V1Gz0Bfd1gwRUTo5ieYLRk68TXAAsA@mail.gmail.com>

2012/10/8 Barry Warsaw <barry at python.org>:
>>Does anyone know if there is a good reason why PyUnicode_FromFormatV()
>>does not fail on invalid format string, but copies the format string
>>as-if into the result?
>>
>>I would like to change this to raise a ValueError instead.
>>PyUnicode_FromFormatV() is used by PyErr_Format(), so PyErr_Format()
>>may now fail with a ValueError if the format string is invalid.
>
> This behavior is inherited from PyUnicode_FromFormat() which documents the
> current behavior, so you can't change this in backward incompatible way.

Yes, the current behaviour is documented, but I consider it as a bug. Would
it be possible to change the behaviour to fix a "bug"?

Victor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121008/c15b8fdf/attachment.html>

From barry at python.org  Mon Oct  8 23:50:55 2012
From: barry at python.org (Barry Warsaw)
Date: Mon, 8 Oct 2012 17:50:55 -0400
Subject: [Python-Dev] Issue #16153: PyUnicode_FromFormatV() must fail if
 the format string is invalid
In-Reply-To: <CAMpsgwatB3kowi55coz7V1Gz0Bfd1gwRUTo5ieYLRk68TXAAsA@mail.gmail.com>
References: <CAMpsgwaOHQQ39sSE10Y5mJQi+Wqs9GpWfb+vtjcT0T0rvPkvDw@mail.gmail.com>
	<20121008101954.2e433a2a@limelight.wooz.org>
	<CAMpsgwatB3kowi55coz7V1Gz0Bfd1gwRUTo5ieYLRk68TXAAsA@mail.gmail.com>
Message-ID: <20121008175055.7a6cff76@resist.wooz.org>

On Oct 08, 2012, at 11:23 PM, Victor Stinner wrote:

>2012/10/8 Barry Warsaw <barry at python.org>:
>>>Does anyone know if there is a good reason why PyUnicode_FromFormatV()
>>>does not fail on invalid format string, but copies the format string
>>>as-if into the result?
>>>
>>>I would like to change this to raise a ValueError instead.
>>>PyUnicode_FromFormatV() is used by PyErr_Format(), so PyErr_Format()
>>>may now fail with a ValueError if the format string is invalid.
>>
>> This behavior is inherited from PyUnicode_FromFormat() which documents the
>> current behavior, so you can't change this in backward incompatible way.
>
>Yes, the current behaviour is documented, but I consider it as a bug. Would
>it be possible to change the behaviour to fix a "bug"?

If it's documented to behave that way, why would you still consider it a bug?
The current behavior is clearly intentional, the function is working as
intended, and there may be code out there that depends on this documented
functionality, or at least, it won't be prepared to handle the new exception.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121008/ab282ad9/attachment.pgp>

From storchaka at gmail.com  Tue Oct  9 00:05:02 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 09 Oct 2012 01:05:02 +0300
Subject: [Python-Dev] Issue #16153: PyUnicode_FromFormatV() must fail if
 the format string is invalid
In-Reply-To: <CAMpsgwatB3kowi55coz7V1Gz0Bfd1gwRUTo5ieYLRk68TXAAsA@mail.gmail.com>
References: <CAMpsgwaOHQQ39sSE10Y5mJQi+Wqs9GpWfb+vtjcT0T0rvPkvDw@mail.gmail.com>
	<20121008101954.2e433a2a@limelight.wooz.org>
	<CAMpsgwatB3kowi55coz7V1Gz0Bfd1gwRUTo5ieYLRk68TXAAsA@mail.gmail.com>
Message-ID: <k4vimg$ckt$1@ger.gmane.org>

On 09.10.12 00:23, Victor Stinner wrote:
> Yes, the current behaviour is documented, but I consider it as a bug.
> Would it be possible to change the behaviour to fix a "bug"?

You should follow the deprecation procedure.



From roundup-admin at psf.upfronthosting.co.za  Tue Oct  9 17:16:41 2012
From: roundup-admin at psf.upfronthosting.co.za (Python tracker)
Date: Tue, 09 Oct 2012 15:16:41 +0000
Subject: [Python-Dev] Failed issue tracker submission
Message-ID: <20121009151641.D9B7D1CE1C@psf.upfronthosting.co.za>



The node specified by the designator in the subject of your message
("16712") does not exist.

Subject was: "[issue16712] [status=closed; resolution=fixed; stage=committed/rejected]"



Mail Gateway Help
=================
Incoming messages are examined for multiple parts:
 . In a multipart/mixed message or part, each subpart is extracted and
   examined. The text/plain subparts are assembled to form the textual
   body of the message, to be stored in the file associated with a "msg"
   class node. Any parts of other types are each stored in separate files
   and given "file" class nodes that are linked to the "msg" node.
 . In a multipart/alternative message or part, we look for a text/plain
   subpart and ignore the other parts.
 . A message/rfc822 is treated similar tomultipart/mixed (except for
   special handling of the first text part) if unpack_rfc822 is set in
   the mailgw config section.

Summary
-------
The "summary" property on message nodes is taken from the first non-quoting
section in the message body. The message body is divided into sections by
blank lines. Sections where the second and all subsequent lines begin with
a ">" or "|" character are considered "quoting sections". The first line of
the first non-quoting section becomes the summary of the message.

Addresses
---------
All of the addresses in the To: and Cc: headers of the incoming message are
looked up among the user nodes, and the corresponding users are placed in
the "recipients" property on the new "msg" node. The address in the From:
header similarly determines the "author" property of the new "msg"
node. The default handling for addresses that don't have corresponding
users is to create new users with no passwords and a username equal to the
address. (The web interface does not permit logins for users with no
passwords.) If we prefer to reject mail from outside sources, we can simply
register an auditor on the "user" class that prevents the creation of user
nodes with no passwords.

Actions
-------
The subject line of the incoming message is examined to determine whether
the message is an attempt to create a new item or to discuss an existing
item. A designator enclosed in square brackets is sought as the first thing
on the subject line (after skipping any "Fwd:" or "Re:" prefixes).

If an item designator (class name and id number) is found there, the newly
created "msg" node is added to the "messages" property for that item, and
any new "file" nodes are added to the "files" property for the item.

If just an item class name is found there, we attempt to create a new item
of that class with its "messages" property initialized to contain the new
"msg" node and its "files" property initialized to contain any new "file"
nodes.

Triggers
--------
Both cases may trigger detectors (in the first case we are calling the
set() method to add the message to the item's spool; in the second case we
are calling the create() method to create a new node). If an auditor raises
an exception, the original message is bounced back to the sender with the
explanatory message given in the exception.
-------------- next part --------------
Return-Path: <python-dev at python.org>
X-Original-To: report at bugs.python.org
Delivered-To: roundup+tracker at psf.upfronthosting.co.za
Received: from mail.python.org (mail.python.org [82.94.164.166])
	by psf.upfronthosting.co.za (Postfix) with ESMTPS id 93DC71CDEC
	for <report at bugs.python.org>; Tue,  9 Oct 2012 17:16:41 +0200 (CEST)
Received: from albatross.python.org (localhost [127.0.0.1])
	by mail.python.org (Postfix) with ESMTP id 3XbhqY1FH2zR2h
	for <report at bugs.python.org>; Tue,  9 Oct 2012 17:16:41 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=python.org; s=200901;
	t=1349795801; bh=iCK/DXUZXT5YdN3qczalnWeJLXZntJSOwSJCLK4oXiY=;
	h=MIME-Version:Content-Type:Content-Transfer-Encoding:From:To:
	 Subject:Message-Id:Date;
	b=SI5InhadwHC9jRtzWowdxRNjdaicnEoMCQwjiei8u4Bm5ENmn1E+BWLOHeXlEnVXE
	 x7UffWiTa3swqNHrNtoNoChufCekDEvPkRE8uGsAp8gwxAQkVzZbjhIaq3P25Npl2v
	 gIbfbBuyMBoq1R/lr9yJ2XY/mCRLfZ3X/p2bWHTk=
Received: from localhost (HELO mail.python.org) (127.0.0.1)
  by albatross.python.org with SMTP; 09 Oct 2012 17:16:41 +0200
Received: from virt-7yvsjn.psf.osuosl.org (virt-7yvsjn.psf.osuosl.org [140.211.10.72])
	by mail.python.org (Postfix) with ESMTP
	for <report at bugs.python.org>; Tue,  9 Oct 2012 17:16:40 +0200 (CEST)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
From: python-dev at python.org
To: report at bugs.python.org
Subject: [issue16712] [status=closed; resolution=fixed;
 stage=committed/rejected]
Message-Id: <3XbhqY1FH2zR2h at mail.python.org>
Date: Tue,  9 Oct 2012 17:16:41 +0200 (CEST)

TmV3IGNoYW5nZXNldCAzMzJiOTNjZTdiZjkgYnkgQmVuamFtaW4gUGV0ZXJzb24gaW4gYnJhbmNo
ICczLjMnOgpjb21wYXJlIHNpbmdsZXRvbnMgYnkgaWRlbnRpdHkgbm90IGVxdWFsaXR5IChjbG9z
ZXMgIzE2NzEyKQpodHRwOi8vaGcucHl0aG9uLm9yZy9jcHl0aG9uL3Jldi8zMzJiOTNjZTdiZjkK

From chris.jerdonek at gmail.com  Wed Oct 10 13:20:30 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Wed, 10 Oct 2012 04:20:30 -0700
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> 3.3):
 Fixes Issue #16114: The subprocess module no longer provides a
In-Reply-To: <3XcBwZ3MK1zQxX@mail.python.org>
References: <3XcBwZ3MK1zQxX@mail.python.org>
Message-ID: <CAOTb1weM2OOkkUf_-XWtAeYpUTh2PQRD441LGTdhb12Teqh1fQ@mail.gmail.com>

On Wed, Oct 10, 2012 at 3:52 AM, gregory.p.smith
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/ee30d7ef70be
> changeset:   79633:ee30d7ef70be
> branch:      3.3
> parent:      79630:9451908da615
> parent:      79632:e938fa6be509
> user:        Gregory P. Smith <greg at krypto.org>
> date:        Wed Oct 10 03:44:47 2012 -0700
> summary:
>   Fixes Issue #16114: The subprocess module no longer provides a
> misleading error message stating that args[0] did not exist when
> either the cwd or executable keyword arguments specified a path that
> did not exist.

> +    def test_exception_cwd(self):
> +        """Test error in the child raised in the parent for a bad cwd."""
> +        desired_exception = self._get_chdir_exception()
>          try:
>              p = subprocess.Popen([sys.executable, "-c", ""],
> -                                 cwd=nonexistent_dir)
> +                                 cwd=self._nonexistent_dir)
>          except OSError as e:
>              # Test that the child process chdir failure actually makes
>              # it up to the parent process as the correct exception.
> @@ -1064,6 +1070,33 @@
>          else:
>              self.fail("Expected OSError: %s" % desired_exception)

I was on this issue but didn't have an opportunity to comment.

It would be cleaner to use the self.assertRaises() pattern here and
also probably better to share code across the three test methods which
are nearly identical to one another (there is a fourth scenario I
would also add of shell=True).

I would also check for FileNotFoundError instead of OSError in the 3.3
and later versions.

--Chris

>
> +    def test_exception_bad_executable(self):
> +        """Test error in the child raised in the parent for a bad executable."""
> +        desired_exception = self._get_chdir_exception()
> +        try:
> +            p = subprocess.Popen([sys.executable, "-c", ""],
> +                                 executable=self._nonexistent_dir)
> +        except OSError as e:
> +            # Test that the child process exec failure actually makes
> +            # it up to the parent process as the correct exception.
> +            self.assertEqual(desired_exception.errno, e.errno)
> +            self.assertEqual(desired_exception.strerror, e.strerror)
> +        else:
> +            self.fail("Expected OSError: %s" % desired_exception)
> +
> +    def test_exception_bad_args_0(self):
> +        """Test error in the child raised in the parent for a bad args[0]."""
> +        desired_exception = self._get_chdir_exception()
> +        try:
> +            p = subprocess.Popen([self._nonexistent_dir, "-c", ""])
> +        except OSError as e:
> +            # Test that the child process exec failure actually makes
> +            # it up to the parent process as the correct exception.
> +            self.assertEqual(desired_exception.errno, e.errno)
> +            self.assertEqual(desired_exception.strerror, e.strerror)
> +        else:
> +            self.fail("Expected OSError: %s" % desired_exception)
> +
>      def test_restore_signals(self):
>          # Code coverage for both values of restore_signals to make sure it
>          # at least does not blow up.
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -37,6 +37,10 @@
>  Library
>  -------
>
> +- Issue #16114: The subprocess module no longer provides a misleading error
> +  message stating that args[0] did not exist when either the cwd or executable
> +  keyword arguments specified a path that did not exist.
> +
>  - Issue #16169: Fix ctypes.WinError()'s confusion between errno and winerror.
>
>  - Issue #16089: Allow ElementTree.TreeBuilder to work again with a non-Element
> diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c
> --- a/Modules/_posixsubprocess.c
> +++ b/Modules/_posixsubprocess.c
> @@ -356,7 +356,7 @@
>             PyObject *preexec_fn,
>             PyObject *preexec_fn_args_tuple)
>  {
> -    int i, saved_errno, unused;
> +    int i, saved_errno, unused, reached_preexec = 0;
>      PyObject *result;
>      const char* err_msg = "";
>      /* Buffer large enough to hold a hex integer.  We can't malloc. */
> @@ -440,6 +440,7 @@
>          POSIX_CALL(setsid());
>  #endif
>
> +    reached_preexec = 1;
>      if (preexec_fn != Py_None && preexec_fn_args_tuple) {
>          /* This is where the user has asked us to deadlock their program. */
>          result = PyObject_Call(preexec_fn, preexec_fn_args_tuple, NULL);
> @@ -489,6 +490,10 @@
>          }
>          unused = write(errpipe_write, cur, hex_errno + sizeof(hex_errno) - cur);
>          unused = write(errpipe_write, ":", 1);
> +        if (!reached_preexec) {
> +            /* Indicate to the parent that the error happened before exec(). */
> +            unused = write(errpipe_write, "noexec", 6);
> +        }
>          /* We can't call strerror(saved_errno).  It is not async signal safe.
>           * The parent process will look the error message up. */
>      } else {
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>

From khabkr at yahoo.com  Thu Oct 11 06:46:41 2012
From: khabkr at yahoo.com (Khalid Abu Bakr)
Date: Wed, 10 Oct 2012 21:46:41 -0700 (PDT)
Subject: [Python-Dev] hg verify warnings
In-Reply-To: <1349930732.86144.YahooMailNeo@web125706.mail.ne1.yahoo.com>
References: <1349930732.86144.YahooMailNeo@web125706.mail.ne1.yahoo.com>
Message-ID: <1349930801.82500.YahooMailNeo@web125705.mail.ne1.yahoo.com>

Hello. Running hg verify on a newly checked out cpython repo on Windows Vista 32Bits yields three warnings:



$ hg verify
repository uses revlog format 1
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
warning: copy source of 'Modules/_threadmodule.c' not in parents of 60ad83716733

warning: copy source of 'Objects/bytesobject.c' not in parents of 64bb1d258322
warning: copy source of 'Objects/stringobject.c' not in parents of 357e268e7c5f
9799 files, 79660 changesets, 176851 total revisions
3 warnings encountered!


$ hg --version
Mercurial Distributed SCM (version 2.3.2)
(see http://mercurial.selenic.com for more information)

Copyright (C) 2005-2012 Matt Mackall and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


I have checked out and tried to verify the cloned repo twice with the same results. So is this serious? Or can these warnings be safely ignored?

Thanks.

From chris.jerdonek at gmail.com  Thu Oct 11 07:03:22 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Wed, 10 Oct 2012 22:03:22 -0700
Subject: [Python-Dev] hg verify warnings
In-Reply-To: <1349930801.82500.YahooMailNeo@web125705.mail.ne1.yahoo.com>
References: <1349930732.86144.YahooMailNeo@web125706.mail.ne1.yahoo.com>
	<1349930801.82500.YahooMailNeo@web125705.mail.ne1.yahoo.com>
Message-ID: <CAOTb1wccq7H4k8c8bSFcz=EApZthMsvZefkBEBnv_O+D5wJPtA@mail.gmail.com>

On Wed, Oct 10, 2012 at 9:46 PM, Khalid Abu Bakr <khabkr at yahoo.com> wrote:
> Hello. Running hg verify on a newly checked out cpython repo on Windows Vista 32Bits yields three warnings:
>
> $ hg verify
> repository uses revlog format 1
> checking changesets
> checking manifests
> crosschecking files in changesets and manifests
> checking files
> warning: copy source of 'Modules/_threadmodule.c' not in parents of 60ad83716733
>
> warning: copy source of 'Objects/bytesobject.c' not in parents of 64bb1d258322
> warning: copy source of 'Objects/stringobject.c' not in parents of 357e268e7c5f
> 9799 files, 79660 changesets, 176851 total revisions
> 3 warnings encountered!

There was a thread about this a couple months ago on python-dev:

http://mail.python.org/pipermail/python-dev/2012-August/121390.html

> I have checked out and tried to verify the cloned repo twice with the same results. So is this serious? Or can these warnings be safely ignored?

>From the thread, it didn't seem like there was a need for corrective
action, so I believe the latter.

--Chris

From anand.jeyahar at gmail.com  Thu Oct 11 10:42:24 2012
From: anand.jeyahar at gmail.com (Anand Jeyahar)
Date: Thu, 11 Oct 2012 14:12:24 +0530
Subject: [Python-Dev] Issue 8492 [was Re: [Python-dev] History stepping
 in interactive session?]
In-Reply-To: <50720689.4020105@pearwood.info>
References: <506EA800.1080106@insectnation.org>
	<20121005140927.759293ed@pitrou.net>
	<506EEDA7.9000108@pearwood.info> <50720689.4020105@pearwood.info>
Message-ID: <CAKERc2ZvboUejnRhCAetQDoRUyBMc6+7kMn57WxAP3RW6uYhsA@mail.gmail.com>

Hi Steven,
     Yep am interested. been looking to get back into C development for
sometime now. Though, don't hold your breath, as the last time i wrote any
C code was 10 years ago and i have no experience with the  cpython code
base.

Will look into it over the weekend.  So far i read up the issue and
understood the requirement.
  Next steps for me to do:
      1. to figure out the readline library bindings
      2. Find the source code in cpython. My first suspicion is that it
should be  in the Python/ folder.


Any suggestions welcome
==============================================
Anand Jeyahar
http://www.blog.anandjeyahar.in/
https://github.com/anandjeyahar
==============================================
The man who is really serious,
with the urge to find out what truth is,
has no style at all. He lives only in what is.
                  ~Bruce Lee




On 8 October 2012 04:17, Steven D'Aprano <steve at pearwood.info> wrote:

> Over on python-ideas, a question about readline was raised and, I think,
> resolved. But while investigating the question, it became obvious to me
> that the ability to inspect the current readline bindings from Python
> was both useful and important.
>
> I wrote:
>
>  I don't believe that there is any direct mechanism for querying the
>> current
>> readline bindings in Python,
>>
>
> But it was requested some time ago: http://bugs.python.org/**issue8492<http://bugs.python.org/issue8492>
>
>
> Is there anyone willing and able to give this issue some attention please?
>
> (Replies to python-dev only please.)
>
>
>
> --
> Steven
> ______________________________**_________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> anand.jeyahar%40gmail.com<http://mail.python.org/mailman/options/python-dev/anand.jeyahar%40gmail.com>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121011/61901582/attachment.html>

From vinay_sajip at yahoo.co.uk  Thu Oct 11 18:06:06 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 11 Oct 2012 16:06:06 +0000 (UTC)
Subject: [Python-Dev] Improved evaluator added to ast module
Message-ID: <loom.20121011T175200-792@post.gmane.org>

In response to http://bugs.python.org/issue15452, I've created an improved
evaluator in the ast module in my sandbox repo. The evaluator supports lookup of
names in a supplied namespace. The basic interface is

def lookup_eval(source_string_or_ast_node, namespace, allow_imports=False):
   # perform limited evaluation of Python expressions

Function calls are not allowed in expressions, but the following are:

* Names (looked up in namespace, and imported if not found there and
  allow_imports is True)
* Literals, just as literal_eval() does
* Array indexing and slicing
* Attribute access
* Arithmetic operators
* Bitwise operators
* Comparison operators
* in / not in
* and / or
* Unary operators

The patch is attached to the issue, and includes changes to replace the use
of eval() by logging.config.fileConfig() to use ast.lookup_eval().

I would welcome review of the patch, particularly as there may be security
implications (the issue is titled "Improve the security model for logging
listener").

Barring objections, I plan to commit it in a week or so.

Regards,

Vinay Sajip


From benjamin at python.org  Thu Oct 11 18:34:52 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 11 Oct 2012 12:34:52 -0400
Subject: [Python-Dev] Improved evaluator added to ast module
In-Reply-To: <loom.20121011T175200-792@post.gmane.org>
References: <loom.20121011T175200-792@post.gmane.org>
Message-ID: <CAPZV6o9H4NHT3v_n2uTLaf-aaes1LkAN161++j9MiyhmH9Z7EQ@mail.gmail.com>

2012/10/11 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
> In response to http://bugs.python.org/issue15452, I've created an improved
> evaluator in the ast module in my sandbox repo. The evaluator supports lookup of
> names in a supplied namespace. The basic interface is
>
> def lookup_eval(source_string_or_ast_node, namespace, allow_imports=False):
>    # perform limited evaluation of Python expressions
>
> Function calls are not allowed in expressions, but the following are:
>
> * Names (looked up in namespace, and imported if not found there and
>   allow_imports is True)
> * Literals, just as literal_eval() does
> * Array indexing and slicing
> * Attribute access
> * Arithmetic operators
> * Bitwise operators
> * Comparison operators
> * in / not in
> * and / or
> * Unary operators

With this operations, you can still cause a lot of trouble.

>
> The patch is attached to the issue, and includes changes to replace the use
> of eval() by logging.config.fileConfig() to use ast.lookup_eval().
>
> I would welcome review of the patch, particularly as there may be security
> implications (the issue is titled "Improve the security model for logging
> listener").

What exactly are you trying to prevent?


-- 
Regards,
Benjamin

From dholth at gmail.com  Thu Oct 11 18:55:16 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 11 Oct 2012 12:55:16 -0400
Subject: [Python-Dev] Improved evaluator added to ast module
In-Reply-To: <CAPZV6o9H4NHT3v_n2uTLaf-aaes1LkAN161++j9MiyhmH9Z7EQ@mail.gmail.com>
References: <loom.20121011T175200-792@post.gmane.org>
	<CAPZV6o9H4NHT3v_n2uTLaf-aaes1LkAN161++j9MiyhmH9Z7EQ@mail.gmail.com>
Message-ID: <CAG8k2+7H=hAeKpZ_BmN7hRx0qQjDQ=AQEuest2viRRteA3ua=g@mail.gmail.com>

On Thu, Oct 11, 2012 at 12:34 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2012/10/11 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
>> In response to http://bugs.python.org/issue15452, I've created an improved
>> evaluator in the ast module in my sandbox repo. The evaluator supports lookup of
>> names in a supplied namespace. The basic interface is
>>
>> def lookup_eval(source_string_or_ast_node, namespace, allow_imports=False):
>>    # perform limited evaluation of Python expressions
>>
>> Function calls are not allowed in expressions, but the following are:
>>
>> * Names (looked up in namespace, and imported if not found there and
>>   allow_imports is True)
>> * Literals, just as literal_eval() does
>> * Array indexing and slicing
>> * Attribute access
>> * Arithmetic operators
>> * Bitwise operators
>> * Comparison operators
>> * in / not in
>> * and / or
>> * Unary operators
>
> With this operations, you can still cause a lot of trouble.
>
>>
>> The patch is attached to the issue, and includes changes to replace the use
>> of eval() by logging.config.fileConfig() to use ast.lookup_eval().
>>
>> I would welcome review of the patch, particularly as there may be security
>> implications (the issue is titled "Improve the security model for logging
>> listener").
>
> What exactly are you trying to prevent?

How does this compare to the markerlib approach? In markerlib you just
make sure all the AST nodes are in a set of allowed nodes, currently
(Compare, BoolOp, Attribute, Name, Load, Str, cmpop, boolop), and then
use the normal eval(). Is one way more secure / fast / flexible than
the other?

(https://bitbucket.org/dholth/markerlib/src/tip/markerlib/markers.py)

Thanks,

Daniel H

From vinay_sajip at yahoo.co.uk  Thu Oct 11 19:25:13 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 11 Oct 2012 17:25:13 +0000 (UTC)
Subject: [Python-Dev] Improved evaluator added to ast module
References: <loom.20121011T175200-792@post.gmane.org>
	<CAPZV6o9H4NHT3v_n2uTLaf-aaes1LkAN161++j9MiyhmH9Z7EQ@mail.gmail.com>
Message-ID: <loom.20121011T190022-80@post.gmane.org>

Benjamin Peterson <benjamin <at> python.org> writes:

> 
> With this operations, you can still cause a lot of trouble.
> 

Perhaps; I am hoping that some more specific information (about the kind of
trouble this can cause) will emerge. Hence the request for review.

> What exactly are you trying to prevent?

The issue was raised because people felt that the use of unrestricted eval()
in logging.config.fileConfig() - usage intended to convert class names and
constructor arguments to Python objects suitable for actually creating objects
like logging handlers - was a potential security hole, because configurations
can be sent and received over the network (not from remote clients, but there's
still a potential vulnerability for machines with multiple users like non-VPS
hosting boxes). No specific hole was described, so I don't have a specific list
of things I'm trying to prevent, other than function calls.

I added support for the items I thought would be useful in an evaluator slightly
more capable than literal_eval, but it's quite possible that I've allowed more
things than needed (e.g. array indexing and slicing). Clearly, replacing eval()
with literal_eval() will mean potential breakage of code in config files out
there - but that's a trade-off that may need to be made if security
considerations prevail.

Alternatively, it may be considered that the changes I've already committed
to logging's listen() function - to allow a verifier callback to be specified
- is sufficient to allay the concerns that led to the issue being raised in the
first place. In which case, I can close the issue without committing this patch.

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Thu Oct 11 19:36:37 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Thu, 11 Oct 2012 17:36:37 +0000 (UTC)
Subject: [Python-Dev] Improved evaluator added to ast module
References: <loom.20121011T175200-792@post.gmane.org>
	<CAPZV6o9H4NHT3v_n2uTLaf-aaes1LkAN161++j9MiyhmH9Z7EQ@mail.gmail.com>
	<CAG8k2+7H=hAeKpZ_BmN7hRx0qQjDQ=AQEuest2viRRteA3ua=g@mail.gmail.com>
Message-ID: <loom.20121011T192849-311@post.gmane.org>

Daniel Holth <dholth <at> gmail.com> writes:

> How does this compare to the markerlib approach? In markerlib you just
> make sure all the AST nodes are in a set of allowed nodes, currently
> (Compare, BoolOp, Attribute, Name, Load, Str, cmpop, boolop), and then
> use the normal eval(). Is one way more secure / fast / flexible than
> the other?

I don't think performance is an issue, and the markerlib approach seems just
as reasonable as the one I've taken, except that it calls eval(), whereas my
approach doesn't. It boils down to what should be allowed in expressions, and
what shouldn't be.

ISTM there is a space for a limited evaluator that's less limiting than
literal_eval(). I do realise that this type of sandboxing is not easy to achieve,
and I'm not aiming to advance the state of the art here - I just want to close
the issue in the best way I can.

Regards,

Vinay Sajip



From dholth at gmail.com  Thu Oct 11 20:03:17 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 11 Oct 2012 14:03:17 -0400
Subject: [Python-Dev] Improved evaluator added to ast module
In-Reply-To: <loom.20121011T192849-311@post.gmane.org>
References: <loom.20121011T175200-792@post.gmane.org>
	<CAPZV6o9H4NHT3v_n2uTLaf-aaes1LkAN161++j9MiyhmH9Z7EQ@mail.gmail.com>
	<CAG8k2+7H=hAeKpZ_BmN7hRx0qQjDQ=AQEuest2viRRteA3ua=g@mail.gmail.com>
	<loom.20121011T192849-311@post.gmane.org>
Message-ID: <CAG8k2+55-R9pEq2yQcGrh9BvqbdSE0+H0p1yzDOwHfGUq6Qm6w@mail.gmail.com>

> ISTM there is a space for a limited evaluator that's less limiting than
> literal_eval(). I do realise that this type of sandboxing is not easy to achieve,
> and I'm not aiming to advance the state of the art here - I just want to close
> the issue in the best way I can.

It is certainly a useful facility.

From storchaka at gmail.com  Thu Oct 11 20:12:37 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 11 Oct 2012 21:12:37 +0300
Subject: [Python-Dev] Improved evaluator added to ast module
In-Reply-To: <CAPZV6o9H4NHT3v_n2uTLaf-aaes1LkAN161++j9MiyhmH9Z7EQ@mail.gmail.com>
References: <loom.20121011T175200-792@post.gmane.org>
	<CAPZV6o9H4NHT3v_n2uTLaf-aaes1LkAN161++j9MiyhmH9Z7EQ@mail.gmail.com>
Message-ID: <k5726s$mr5$1@ger.gmane.org>

On 11.10.12 19:34, Benjamin Peterson wrote:
> With this operations, you can still cause a lot of trouble.

Agree. Simple example: 9**9**9.



From storchaka at gmail.com  Thu Oct 11 20:13:35 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 11 Oct 2012 21:13:35 +0300
Subject: [Python-Dev] Improved evaluator added to ast module
In-Reply-To: <loom.20121011T175200-792@post.gmane.org>
References: <loom.20121011T175200-792@post.gmane.org>
Message-ID: <k5728e$mr5$2@ger.gmane.org>

On 11.10.12 19:06, Vinay Sajip wrote:
> Function calls are not allowed in expressions, but the following are:

Are properties allowed?



From g.brandl at gmx.net  Thu Oct 11 20:26:53 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 11 Oct 2012 20:26:53 +0200
Subject: [Python-Dev] Improved evaluator added to ast module
In-Reply-To: <k5728e$mr5$2@ger.gmane.org>
References: <loom.20121011T175200-792@post.gmane.org>
	<k5728e$mr5$2@ger.gmane.org>
Message-ID: <k572v7$u8o$1@ger.gmane.org>

Am 11.10.2012 20:13, schrieb Serhiy Storchaka:
> On 11.10.12 19:06, Vinay Sajip wrote:
>> Function calls are not allowed in expressions, but the following are:
> 
> Are properties allowed?

Yes, since attribute access also means properties.

This doesn't have to be a problem, since the objects on which you can
access properties are not controlled by the user.

I'm not sure if you can get at "dangerous" objects, such as sys.modules,
via attribute and item access chains if the accessible objects are
of built-in types, but the case is very difficult to decide in practice
for objects of user-supplied classes.

Georg


From benjamin at python.org  Thu Oct 11 20:32:52 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 11 Oct 2012 14:32:52 -0400
Subject: [Python-Dev] Improved evaluator added to ast module
In-Reply-To: <loom.20121011T190022-80@post.gmane.org>
References: <loom.20121011T175200-792@post.gmane.org>
	<CAPZV6o9H4NHT3v_n2uTLaf-aaes1LkAN161++j9MiyhmH9Z7EQ@mail.gmail.com>
	<loom.20121011T190022-80@post.gmane.org>
Message-ID: <CAPZV6o_hf9zAu_boFNn4banQEMB7-F7usGigO5HyOZ0QVbo8pw@mail.gmail.com>

2012/10/11 Vinay Sajip <vinay_sajip at yahoo.co.uk>:
> Benjamin Peterson <benjamin <at> python.org> writes:
>
>>
>> With this operations, you can still cause a lot of trouble.
>>
>
> Perhaps; I am hoping that some more specific information (about the kind of
> trouble this can cause) will emerge. Hence the request for review.

For example, if you have attribute access you can easily get to
important modules like "sys" and "os". Banning function calls is
essentially a red-herring, since almost any operation in Python can
cause arbitrary code execution.

>
>> What exactly are you trying to prevent?
>
> The issue was raised because people felt that the use of unrestricted eval()
> in logging.config.fileConfig() - usage intended to convert class names and
> constructor arguments to Python objects suitable for actually creating objects
> like logging handlers - was a potential security hole, because configurations
> can be sent and received over the network (not from remote clients, but there's
> still a potential vulnerability for machines with multiple users like non-VPS
> hosting boxes). No specific hole was described, so I don't have a specific list
> of things I'm trying to prevent, other than function calls.

Surely, if a hacker can get the app to use logging configs he sends,
he can have it send sensitive log data to himself.

>
> I added support for the items I thought would be useful in an evaluator slightly
> more capable than literal_eval, but it's quite possible that I've allowed more
> things than needed (e.g. array indexing and slicing). Clearly, replacing eval()
> with literal_eval() will mean potential breakage of code in config files out
> there - but that's a trade-off that may need to be made if security
> considerations prevail.
>
> Alternatively, it may be considered that the changes I've already committed
> to logging's listen() function - to allow a verifier callback to be specified
> - is sufficient to allay the concerns that led to the issue being raised in the
> first place. In which case, I can close the issue without committing this patch.

I like that. I don't exactly unterstand the usecase for logging
untrusted logging configurations.


-- 
Regards,
Benjamin

From chris.jerdonek at gmail.com  Fri Oct 12 01:06:34 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Thu, 11 Oct 2012 16:06:34 -0700
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
	Merge 3.2
In-Reply-To: <3Xd4dx1PKvzR3q@mail.python.org>
References: <3Xd4dx1PKvzR3q@mail.python.org>
Message-ID: <CAOTb1wc+sH43cPeJWt9ERMi9iuAA5o_F8o_RpdeeY67+oayxrw@mail.gmail.com>

On Thu, Oct 11, 2012 at 2:13 PM, brian.curtin
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/2802b26c64a9
> changeset:   79670:2802b26c64a9
> parent:      79667:c3c188a0325a
> parent:      79668:8245333db1e5
> user:        Brian Curtin <brian at python.org>
> date:        Thu Oct 11 16:12:47 2012 -0500
> summary:
>   Merge 3.2

FYI, it looks like a merge from 3.3 to default is missing from this
set of changes.

--Chris

>
> files:
>   Lib/platform.py |  7 ++++++-
>   Misc/NEWS       |  2 ++
>   2 files changed, 8 insertions(+), 1 deletions(-)
>
>
> diff --git a/Lib/platform.py b/Lib/platform.py
> --- a/Lib/platform.py
> +++ b/Lib/platform.py
> @@ -595,8 +595,13 @@
>                      release = '7'
>                  else:
>                      release = '2008ServerR2'
> +            elif min == 2:
> +                if product_type == VER_NT_WORKSTATION:
> +                    release = '8'
> +                else:
> +                    release = '2012Server'
>              else:
> -                release = 'post2008Server'
> +                release = 'post2012Server'
>
>      else:
>          if not release:
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -45,6 +45,8 @@
>  Library
>  -------
>
> +- Issue #16176: Properly identify Windows 8 via platform.platform()
> +
>  - Issue #16088: BaseHTTPRequestHandler's send_error method includes a
>    Content-Length header in it's response now. Patch by Antoine Pitrou.
>
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>

From status at bugs.python.org  Fri Oct 12 18:07:21 2012
From: status at bugs.python.org (Python tracker)
Date: Fri, 12 Oct 2012 18:07:21 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20121012160721.9F5A91CA8E@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2012-10-05 - 2012-10-12)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3789 (+19)
  closed 24201 (+52)
  total  27990 (+71)

Open issues with patches: 1694 


Issues opened (47)
==================

#8489: Support UTF8SMTP as part of RFC 5336 in smptlib
http://bugs.python.org/issue8489  reopened by r.david.murray

#14700: Integer overflow in classic string formatting
http://bugs.python.org/issue14700  reopened by mark.dickinson

#16143: Building with configure option "--without-doc-strings" crashes
http://bugs.python.org/issue16143  opened by Gregory.Andersen

#16144: misleading sentence in reference/import
http://bugs.python.org/issue16144  opened by mpg

#16145: Abort in _csv module
http://bugs.python.org/issue16145  opened by Arfrever

#16151: Deferred KeyboardInterrupt in interactive mode
http://bugs.python.org/issue16151  opened by serhiy.storchaka

#16152: Trailing whitespace makes tokenize.generate_tokens pathologica
http://bugs.python.org/issue16152  opened by nedbat

#16153: PyUnicode_FromFormatV() must fail if the format string is inva
http://bugs.python.org/issue16153  opened by haypo

#16154: Some minor doc fixes in Doc/library
http://bugs.python.org/issue16154  opened by Ravi.Sinha

#16155: Some minor doc fixes in Doc/faq
http://bugs.python.org/issue16155  opened by Ravi.Sinha

#16157: Irrelevant references to Misc/News
http://bugs.python.org/issue16157  opened by serhiy.storchaka

#16158: sporadic test_multiprocessing failure
http://bugs.python.org/issue16158  opened by pitrou

#16160: subclassing types.SimpleNamespace does not work
http://bugs.python.org/issue16160  opened by r.david.murray

#16163: Wrong name in Lib/pkgutil.py:iter_importers
http://bugs.python.org/issue16163  opened by berker.peksag

#16165: sched.scheduler.run() blocks scheduler
http://bugs.python.org/issue16165  opened by serhiy.storchaka

#16166: Add PY_BYTE_ORDER macro to get endianess of platform
http://bugs.python.org/issue16166  opened by christian.heimes

#16175: Add I/O Completion Ports wrapper
http://bugs.python.org/issue16175  opened by christian.heimes

#16177: IDLE Crash on Open Parens
http://bugs.python.org/issue16177  opened by brianckeegan

#16178: atexit._run_exitfuncs should be a public API
http://bugs.python.org/issue16178  opened by gregory.p.smith

#16180: cannot quit pdb when there is a syntax error in the debuggee (
http://bugs.python.org/issue16180  opened by xdegaye

#16181: cookielib.http2time raises ValueError for invalid date.
http://bugs.python.org/issue16181  opened by Charles.Jones

#16182: readline: Wrong tab completion scope indices in Unicode termin
http://bugs.python.org/issue16182  opened by kunkku

#16183: ZipExtFile object close without file handle closed
http://bugs.python.org/issue16183  opened by V.E.O

#16185: include path in subprocess.Popen() file not found error messag
http://bugs.python.org/issue16185  opened by chris.jerdonek

#16186: shlex bug?
http://bugs.python.org/issue16186  opened by jwfang

#16188: Windows C Runtime Library Mismatch
http://bugs.python.org/issue16188  opened by rlinscheer

#16189: ld_so_aix not found
http://bugs.python.org/issue16189  opened by rjs

#16190: Misleading warning in random module docs
http://bugs.python.org/issue16190  opened by christian.heimes

#16192: ctypes - documentation example
http://bugs.python.org/issue16192  opened by Thorney

#16193: display full e-mail name in hg.python.org annotate pages
http://bugs.python.org/issue16193  opened by chris.jerdonek

#16194: imp.load_dynamic imports wrong module when called several time
http://bugs.python.org/issue16194  opened by eudoxos

#16195: Difficult or impossible to figure out how garbage collector an
http://bugs.python.org/issue16195  opened by exarkun

#16197: Several small errors in winreg documentation
http://bugs.python.org/issue16197  opened by zach.ware

#16198: IDLE - tabbing in a string always brings up file completion wi
http://bugs.python.org/issue16198  opened by serwy

#16200: Setting .posix=True for shlex object causes infinite loop in _
http://bugs.python.org/issue16200  opened by serwy

#16201: socket.gethostbyname incorrectly parses ip
http://bugs.python.org/issue16201  opened by maker

#16202: sys.path[0] security issues
http://bugs.python.org/issue16202  opened by jdemeyer

#16203: Proposal: add re.fullmatch() method
http://bugs.python.org/issue16203  opened by gvanrossum

#16204: PyBuffer_FillInfo returns 'B' buffer, whose behavior has chang
http://bugs.python.org/issue16204  opened by piro

#16205: update :class:`str` references to link to the str type section
http://bugs.python.org/issue16205  opened by chris.jerdonek

#16206: dict() docs should display multiple signatures
http://bugs.python.org/issue16206  opened by chris.jerdonek

#16207: distinguish methods from non-methods in classes in the stdtype
http://bugs.python.org/issue16207  opened by chris.jerdonek

#16208: getaddrinfo returns wrong results if IPv6 is disabled
http://bugs.python.org/issue16208  opened by schmir

#16209: add a "class str" entry to the docs
http://bugs.python.org/issue16209  opened by chris.jerdonek

#16210: combine the two type() definitions in built-in function docs
http://bugs.python.org/issue16210  opened by chris.jerdonek

#16212: mmap() dumps core upon resizing the underlying file
http://bugs.python.org/issue16212  opened by Vladimir.Ushakov

#16213: Expose private functions in marshal used by importlib
http://bugs.python.org/issue16213  opened by brett.cannon



Most recent 15 issues with no replies (15)
==========================================

#16210: combine the two type() definitions in built-in function docs
http://bugs.python.org/issue16210

#16209: add a "class str" entry to the docs
http://bugs.python.org/issue16209

#16208: getaddrinfo returns wrong results if IPv6 is disabled
http://bugs.python.org/issue16208

#16206: dict() docs should display multiple signatures
http://bugs.python.org/issue16206

#16204: PyBuffer_FillInfo returns 'B' buffer, whose behavior has chang
http://bugs.python.org/issue16204

#16188: Windows C Runtime Library Mismatch
http://bugs.python.org/issue16188

#16185: include path in subprocess.Popen() file not found error messag
http://bugs.python.org/issue16185

#16183: ZipExtFile object close without file handle closed
http://bugs.python.org/issue16183

#16182: readline: Wrong tab completion scope indices in Unicode termin
http://bugs.python.org/issue16182

#16181: cookielib.http2time raises ValueError for invalid date.
http://bugs.python.org/issue16181

#16180: cannot quit pdb when there is a syntax error in the debuggee (
http://bugs.python.org/issue16180

#16178: atexit._run_exitfuncs should be a public API
http://bugs.python.org/issue16178

#16158: sporadic test_multiprocessing failure
http://bugs.python.org/issue16158

#16151: Deferred KeyboardInterrupt in interactive mode
http://bugs.python.org/issue16151

#16144: misleading sentence in reference/import
http://bugs.python.org/issue16144



Most recent 15 issues waiting for review (15)
=============================================

#16202: sys.path[0] security issues
http://bugs.python.org/issue16202

#16201: socket.gethostbyname incorrectly parses ip
http://bugs.python.org/issue16201

#16200: Setting .posix=True for shlex object causes infinite loop in _
http://bugs.python.org/issue16200

#16198: IDLE - tabbing in a string always brings up file completion wi
http://bugs.python.org/issue16198

#16197: Several small errors in winreg documentation
http://bugs.python.org/issue16197

#16192: ctypes - documentation example
http://bugs.python.org/issue16192

#16182: readline: Wrong tab completion scope indices in Unicode termin
http://bugs.python.org/issue16182

#16175: Add I/O Completion Ports wrapper
http://bugs.python.org/issue16175

#16166: Add PY_BYTE_ORDER macro to get endianess of platform
http://bugs.python.org/issue16166

#16165: sched.scheduler.run() blocks scheduler
http://bugs.python.org/issue16165

#16163: Wrong name in Lib/pkgutil.py:iter_importers
http://bugs.python.org/issue16163

#16160: subclassing types.SimpleNamespace does not work
http://bugs.python.org/issue16160

#16155: Some minor doc fixes in Doc/faq
http://bugs.python.org/issue16155

#16154: Some minor doc fixes in Doc/library
http://bugs.python.org/issue16154

#16153: PyUnicode_FromFormatV() must fail if the format string is inva
http://bugs.python.org/issue16153



Top 10 most discussed issues (10)
=================================

#16145: Abort in _csv module
http://bugs.python.org/issue16145  17 msgs

#16166: Add PY_BYTE_ORDER macro to get endianess of platform
http://bugs.python.org/issue16166  17 msgs

#16175: Add I/O Completion Ports wrapper
http://bugs.python.org/issue16175  14 msgs

#16114: incorrect path in subprocess.Popen() FileNotFoundError message
http://bugs.python.org/issue16114  12 msgs

#15278: UnicodeDecodeError when readline in codecs.py
http://bugs.python.org/issue15278  11 msgs

#16096: Get rid of dangerous integer overflow tricks
http://bugs.python.org/issue16096  11 msgs

#14900: cProfile does not take its result headers as sort arguments
http://bugs.python.org/issue14900  10 msgs

#15348: IDLE - shell becomes unresponsive if debugger windows is close
http://bugs.python.org/issue15348  10 msgs

#16113: Add SHA-3 (Keccak) support
http://bugs.python.org/issue16113  10 msgs

#16194: imp.load_dynamic imports wrong module when called several time
http://bugs.python.org/issue16194  10 msgs



Issues closed (49)
==================

#7735: python creates IPv6 DNS requests even when built with --disabl
http://bugs.python.org/issue7735  closed by loewis

#8416: python 2.6.5 documentation can't search
http://bugs.python.org/issue8416  closed by georg.brandl

#9957: SpooledTemporayFile.truncate should take size parameter
http://bugs.python.org/issue9957  closed by r.david.murray

#10968: threading.Timer should be a class so that it can be derived
http://bugs.python.org/issue10968  closed by r.david.murray

#11710: Landing pages in docs for standard library packages
http://bugs.python.org/issue11710  closed by python-dev

#12322: ElementPath 1.3 expressions documentation
http://bugs.python.org/issue12322  closed by eli.bendersky

#12986: Using getrandbits() in uuid.uuid4() is faster and more readabl
http://bugs.python.org/issue12986  closed by pitrou

#13896: Make shelf instances work with 'with' as context managers
http://bugs.python.org/issue13896  closed by asvetlov

#14668: Document the path option in the Windows installer
http://bugs.python.org/issue14668  closed by brian.curtin

#14783: Make int() and str() docstrings correct
http://bugs.python.org/issue14783  closed by chris.jerdonek

#14955: hmac.secure_compare() is not time-independent for unicode stri
http://bugs.python.org/issue14955  closed by christian.heimes

#15776: Allow pyvenv to work in existing directory
http://bugs.python.org/issue15776  closed by python-dev

#15888: ipaddress doctest examples have some errors
http://bugs.python.org/issue15888  closed by eli.bendersky

#15922: make howto/urllib2.rst doctests pass
http://bugs.python.org/issue15922  closed by orsenthil

#15991: BaseHTTPServer with ThreadingMixIn serving wrong data sometime
http://bugs.python.org/issue15991  closed by pitrou

#16025: Minor corrections to the zipfile documentation
http://bugs.python.org/issue16025  closed by serhiy.storchaka

#16059: Serialize MD5 computation-state and resume later
http://bugs.python.org/issue16059  closed by gregory.p.smith

#16082: xml.etree.ElementTree.tostringlist does not conform to its doc
http://bugs.python.org/issue16082  closed by python-dev

#16088: http.server's send_error could set the content-length of the e
http://bugs.python.org/issue16088  closed by orsenthil

#16115: test the executable arg to Popen() and improve related docs
http://bugs.python.org/issue16115  closed by asvetlov

#16119: Python 2.7 _socket DLL import error on Windows Vista
http://bugs.python.org/issue16119  closed by sk7

#16120: Use |yield from| in stdlib
http://bugs.python.org/issue16120  closed by asvetlov

#16139: Python 3.3 fails when starting from read-only FS
http://bugs.python.org/issue16139  closed by neologix

#16146: MIMEApplication cannot access
http://bugs.python.org/issue16146  closed by r.david.murray

#16147: Rewrite PyUnicode_FromFormatV() to use the _PyUnicodeWriter AP
http://bugs.python.org/issue16147  closed by haypo

#16148: Implement PEP 424
http://bugs.python.org/issue16148  closed by aronacher

#16149: "Decimal(2) != float(2)" documentation bug
http://bugs.python.org/issue16149  closed by python-dev

#16150: Implement generator interface in itertools.chain.
http://bugs.python.org/issue16150  closed by ncoghlan

#16156: Crash every time I copy or paste any code in IDLE
http://bugs.python.org/issue16156  closed by christian.heimes

#16159: Closing Debugger freezes IDLE shell
http://bugs.python.org/issue16159  closed by serwy

#16161: broken link to Visual Studio 2008 in devguide
http://bugs.python.org/issue16161  closed by chris.jerdonek

#16162: Py_FileSystemDefaultEncoding should be updated on locale.setlo
http://bugs.python.org/issue16162  closed by r.david.murray

#16164: there is no easy way to force unittest.main to use stdout rath
http://bugs.python.org/issue16164  closed by michael.foord

#16167: Allow printing of str-derived classes
http://bugs.python.org/issue16167  closed by benjamin.peterson

#16168: SysLogHandler constructor ignores socktype arg when address re
http://bugs.python.org/issue16168  closed by python-dev

#16169: ctypes.WinError() confuses errno with winerror
http://bugs.python.org/issue16169  closed by sbt

#16170: remove Linux skip from test_subprocess's test_executable() tes
http://bugs.python.org/issue16170  closed by chris.jerdonek

#16171: "do nothing" load_tests could be improved
http://bugs.python.org/issue16171  closed by michael.foord

#16172: Fix is/== misuse
http://bugs.python.org/issue16172  closed by benjamin.peterson

#16173: Wrong offset on SyntaxError when identifier contains non-ascii
http://bugs.python.org/issue16173  closed by ezio.melotti

#16174: Fix suggested usage of dummy_threading module
http://bugs.python.org/issue16174  closed by asvetlov

#16176: platform.platform() identifies Windows 8 as post2008Server
http://bugs.python.org/issue16176  closed by brian.curtin

#16179: hashlib.md5 / json inconsistency
http://bugs.python.org/issue16179  closed by r.david.murray

#16184: Attack against the pseudorandom number generator
http://bugs.python.org/issue16184  closed by christian.heimes

#16187: incoming.irker hook raised an exception: [Errno 111] Connectio
http://bugs.python.org/issue16187  closed by georg.brandl

#16191: ceval cleanup
http://bugs.python.org/issue16191  closed by python-dev

#16196: Typo in devguide stdlibchanges.html
http://bugs.python.org/issue16196  closed by ezio.melotti

#16199: 2to3 very minor doc change
http://bugs.python.org/issue16199  closed by chris.jerdonek

#16211: MacInstalls break PyDev
http://bugs.python.org/issue16211  closed by ned.deily

From petri at digip.org  Fri Oct 12 19:50:39 2012
From: petri at digip.org (Petri Lehtinen)
Date: Fri, 12 Oct 2012 20:50:39 +0300
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <506B685C.3050608@netwok.org>
References: <5064D399.5050208@netwok.org> <506B36DB.1090001@netwok.org>
	<CAPTjJmqGWfoCKKBHuJwJGpg-roNoQQ0NMt_HVDCAaau4U+4MQA@mail.gmail.com>
	<506B685C.3050608@netwok.org>
Message-ID: <20121012175039.GA1284@chang>

?ric Araujo wrote:
> Hi Chris,
> 
> Le 02/10/2012 18:14, Chris Angelico a ?crit :
> > Apologies if this is a stupid question (or just completely
> > misdirected), but does this mean that if I'm interested in
> > participating in the bug day, the first step should be to join
> > core-mentorship@?
> 
> It?s not required, but you are welcome to do it.  I?ll send an
> announcement with details about participating soon?.

It's two and a half weeks left, but I've not seen any announcements
yet!

Also, my issue list at http://piratepad.net/pyconfi-sprint-issues
still lacks entries, so everyone interested in making the bug day a
huge success is welcome to add issues to the list. It doesn't need
registration or anything, just open it in your browser and edit.

Petri

From chris.jerdonek at gmail.com  Sat Oct 13 02:47:35 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Fri, 12 Oct 2012 17:47:35 -0700
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default):
	merge 3.3
In-Reply-To: <3XdYmJ3rK4zR4J@mail.python.org>
References: <3XdYmJ3rK4zR4J@mail.python.org>
Message-ID: <CAOTb1wcd7_xhEMWp8oHKy1NqGs2R24de5HU-oSqjeOQMYqpybA@mail.gmail.com>

On Fri, Oct 12, 2012 at 9:05 AM, benjamin.peterson
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/6e721c72683f
> changeset:   79703:6e721c72683f
> parent:      79700:d3d4737fa093
> parent:      79702:de8787029fe4
> user:        Benjamin Peterson <benjamin at python.org>
> date:        Fri Oct 12 12:05:09 2012 -0400
> summary:
>   merge 3.3

It looks like this check-in reverted previous valid work.

--Chris

>
> files:
>   Doc/howto/functional.rst  |   5 ++---
>   Doc/library/functions.rst |  25 +++++++++----------------
>   2 files changed, 11 insertions(+), 19 deletions(-)
>
>
> diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst
> --- a/Doc/howto/functional.rst
> +++ b/Doc/howto/functional.rst
> @@ -292,14 +292,13 @@
>  Applying :func:`iter` to a dictionary always loops over the keys, but
>  dictionaries have methods that return other iterators.  If you want to iterate
>  over values or key/value pairs, you can explicitly call the
> -:meth:`~dict.values` or :meth:`~dict.items` methods to get an appropriate
> -iterator.
> +:meth:`~dict.values` or :meth:`~dict.items` methods to get an appropriate iterator.
>
>  The :func:`dict` constructor can accept an iterator that returns a finite stream
>  of ``(key, value)`` tuples:
>
>      >>> L = [('Italy', 'Rome'), ('France', 'Paris'), ('US', 'Washington DC')]
> -    >>> dict(iter(L))  #doctest: +SKIP
> +    >>> dict(iter(L))
>      {'Italy': 'Rome', 'US': 'Washington DC', 'France': 'Paris'}
>
>  Files also support iteration by calling the :meth:`~io.TextIOBase.readline`
> diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
> --- a/Doc/library/functions.rst
> +++ b/Doc/library/functions.rst
> @@ -122,8 +122,6 @@
>
>     Without an argument, an array of size 0 is created.
>
> -   See also :ref:`binaryseq` and :ref:`typebytearray`.
> -
>
>  .. _func-bytes:
>  .. function:: bytes([source[, encoding[, errors]]])
> @@ -137,8 +135,6 @@
>
>     Bytes objects can also be created with literals, see :ref:`strings`.
>
> -   See also :ref:`binaryseq`, :ref:`typebytes`, and :ref:`bytes-methods`.
> -
>
>  .. function:: callable(object)
>
> @@ -692,8 +688,6 @@
>     *sentinel*, :exc:`StopIteration` will be raised, otherwise the value will
>     be returned.
>
> -   See also :ref:`typeiter`.
> -
>     One useful application of the second form of :func:`iter` is to read lines of
>     a file until a certain line is reached.  The following example reads a file
>     until the :meth:`readline` method returns an empty string::
> @@ -714,7 +708,7 @@
>     :noindex:
>
>     Rather than being a function, :class:`list` is actually a mutable
> -   sequence type, as documented in :ref:`typesseq-list` and :ref:`typesseq`.
> +   sequence type, as documented in :ref:`typesseq`.
>
>
>  .. function:: locals()
> @@ -1088,7 +1082,7 @@
>     :noindex:
>
>     Rather than being a function, :class:`range` is actually an immutable
> -   sequence type, as documented in :ref:`typesseq-range` and :ref:`typesseq`.
> +   sequence type, as documented in :ref:`typesseq`.
>
>
>  .. function:: repr(object)
> @@ -1213,8 +1207,7 @@
>  .. function:: str(object='')
>                str(object[, encoding[, errors]])
>
> -   Return a :ref:`string <textseq>` version of an object, using one of the
> -   following modes:
> +   Return a string version of an object, using one of the following modes:
>
>     If *encoding* and/or *errors* are given, :func:`str` will decode the
>     *object* which can either be a byte string or a character buffer using
> @@ -1237,9 +1230,11 @@
>     Objects can specify what ``str(object)`` returns by defining a :meth:`__str__`
>     special method.
>
> -   For more information on strings and string methods, see the :ref:`textseq`
> -   section.  To output formatted strings, see the :ref:`string-formatting`
> -   section.  In addition, see the :ref:`stringservices` section.
> +   For more information on strings see :ref:`typesseq` which describes sequence
> +   functionality (strings are sequences), and also the string-specific methods
> +   described in the :ref:`string-methods` section. To output formatted strings,
> +   see the :ref:`string-formatting` section. In addition see the
> +   :ref:`stringservices` section.
>
>
>  .. function:: sum(iterable[, start])
> @@ -1316,7 +1311,7 @@
>     :noindex:
>
>     Rather than being a function, :class:`tuple` is actually an immutable
> -   sequence type, as documented in :ref:`typesseq-tuple` and :ref:`typesseq`.
> +   sequence type, as documented in :ref:`typesseq`.
>
>
>  .. function:: type(object)
> @@ -1349,8 +1344,6 @@
>        ...
>        >>> X = type('X', (object,), dict(a=1))
>
> -   See also :ref:`bltin-type-objects`.
> -
>
>  .. function:: vars([object])
>
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>

From chris.jerdonek at gmail.com  Sat Oct 13 03:25:55 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Fri, 12 Oct 2012 18:25:55 -0700
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> 3.3):
	merge heads
In-Reply-To: <3XdYmG6jxSzR3X@mail.python.org>
References: <3XdYmG6jxSzR3X@mail.python.org>
Message-ID: <CAOTb1wc0S-sJ4+UF3B8LZn4Uj=DkT0nCWazAbJs-bmWwfdD5RA@mail.gmail.com>

On Fri, Oct 12, 2012 at 9:05 AM, benjamin.peterson
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/de8787029fe4
> changeset:   79702:de8787029fe4
> branch:      3.3
> parent:      79701:d4ab5859721e
> parent:      79690:0cddf0bd19f8

It looks like this 3.3 check-in also reverted previous valid work.
The following command shows the diff from the "mainline", which isn't
visible in the e-mail contents below:

hg diff -r 0cddf0bd19f8 -r de8787029fe4

It's hard for me to piece together exactly what happened, or else I
would try to fix it myself.

--Chris


> user:        Benjamin Peterson <benjamin at python.org>
> date:        Fri Oct 12 12:05:01 2012 -0400
> summary:
>   merge heads
>
> files:
>   Doc/howto/unicode.rst         |  2 +-
>   Doc/library/exceptions.rst    |  4 ++--
>   Doc/library/stdtypes.rst      |  8 ++++----
>   Doc/library/string.rst        |  2 +-
>   Doc/tutorial/introduction.rst |  2 +-
>   Misc/NEWS                     |  2 ++
>   6 files changed, 11 insertions(+), 9 deletions(-)
>
>
> diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst
> --- a/Doc/howto/unicode.rst
> +++ b/Doc/howto/unicode.rst
> @@ -414,7 +414,7 @@
>  ----------
>
>  The ``str`` type is described in the Python library reference at
> -:ref:`typesseq`.
> +:ref:`textseq`.
>
>  The documentation for the :mod:`unicodedata` module.
>
> diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst
> --- a/Doc/library/exceptions.rst
> +++ b/Doc/library/exceptions.rst
> @@ -275,8 +275,8 @@
>  .. exception:: StopIteration
>
>     Raised by built-in function :func:`next` and an :term:`iterator`\'s
> -   :meth:`__next__` method to signal that there are no further items to be
> -   produced by the iterator.
> +   :meth:`~iterator.__next__` method to signal that there are no further
> +   items produced by the iterator.
>
>     The exception object has a single attribute :attr:`value`, which is
>     given as an argument when constructing the exception, and defaults
> diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
> --- a/Doc/library/stdtypes.rst
> +++ b/Doc/library/stdtypes.rst
> @@ -1358,8 +1358,8 @@
>     object: io.StringIO
>
>
> -Textual data in Python is handled with :class:`str` objects, which are
> -immutable sequences of Unicode code points.  String literals are
> +Textual data in Python is handled with ``str`` objects, which are immutable
> +:ref:`sequences <typesseq>` of Unicode code points.  String literals are
>  written in a variety of ways:
>
>  * Single quotes: ``'allows embedded "double" quotes'``
> @@ -1377,8 +1377,8 @@
>  including supported escape sequences, and the ``r`` ("raw") prefix that
>  disables most escape sequence processing.
>
> -Strings may also be created from other objects with the :ref:`str <func-str>`
> -built-in.
> +Strings may also be created from other objects with the built-in
> +function :func:`str`.
>
>  Since there is no separate "character" type, indexing a string produces
>  strings of length 1. That is, for a non-empty string *s*, ``s[0] == s[0:1]``.
> diff --git a/Doc/library/string.rst b/Doc/library/string.rst
> --- a/Doc/library/string.rst
> +++ b/Doc/library/string.rst
> @@ -10,7 +10,7 @@
>
>  .. seealso::
>
> -   :ref:`typesseq`
> +   :ref:`textseq`
>
>     :ref:`string-methods`
>
> diff --git a/Doc/tutorial/introduction.rst b/Doc/tutorial/introduction.rst
> --- a/Doc/tutorial/introduction.rst
> +++ b/Doc/tutorial/introduction.rst
> @@ -390,7 +390,7 @@
>
>  .. seealso::
>
> -   :ref:`typesseq`
> +   :ref:`textseq`
>        Strings are examples of *sequence types*, and support the common
>        operations supported by such types.
>
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -41,6 +41,8 @@
>  Library
>  -------
>
> +- Issue #16176: Properly identify Windows 8 via platform.platform()
> +
>  - Issue #16114: The subprocess module no longer provides a misleading error
>    message stating that args[0] did not exist when either the cwd or executable
>    keyword arguments specified a path that did not exist.
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>

From benjamin at python.org  Sat Oct 13 04:40:55 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 12 Oct 2012 22:40:55 -0400
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> 3.3):
	merge heads
In-Reply-To: <CAOTb1wc0S-sJ4+UF3B8LZn4Uj=DkT0nCWazAbJs-bmWwfdD5RA@mail.gmail.com>
References: <3XdYmG6jxSzR3X@mail.python.org>
	<CAOTb1wc0S-sJ4+UF3B8LZn4Uj=DkT0nCWazAbJs-bmWwfdD5RA@mail.gmail.com>
Message-ID: <CAPZV6o_yFeZrWRr_fBtj3OOsqG-Uhqgs4W1ZkNvbH9s74kY44w@mail.gmail.com>

2012/10/12 Chris Jerdonek <chris.jerdonek at gmail.com>:
> On Fri, Oct 12, 2012 at 9:05 AM, benjamin.peterson
> <python-checkins at python.org> wrote:
>> http://hg.python.org/cpython/rev/de8787029fe4
>> changeset:   79702:de8787029fe4
>> branch:      3.3
>> parent:      79701:d4ab5859721e
>> parent:      79690:0cddf0bd19f8
>
> It looks like this 3.3 check-in also reverted previous valid work.
> The following command shows the diff from the "mainline", which isn't
> visible in the e-mail contents below:
>
> hg diff -r 0cddf0bd19f8 -r de8787029fe4
>
> It's hard for me to piece together exactly what happened, or else I
> would try to fix it myself.

Sorry about that. I merged 3.2 -> default instead of 3.2 -> 3.3 ->
default and these were my attempts to fix it.


-- 
Regards,
Benjamin

From chris.jerdonek at gmail.com  Sat Oct 13 05:22:16 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Fri, 12 Oct 2012 20:22:16 -0700
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> 3.3):
	merge heads
In-Reply-To: <CAPZV6o_yFeZrWRr_fBtj3OOsqG-Uhqgs4W1ZkNvbH9s74kY44w@mail.gmail.com>
References: <3XdYmG6jxSzR3X@mail.python.org>
	<CAOTb1wc0S-sJ4+UF3B8LZn4Uj=DkT0nCWazAbJs-bmWwfdD5RA@mail.gmail.com>
	<CAPZV6o_yFeZrWRr_fBtj3OOsqG-Uhqgs4W1ZkNvbH9s74kY44w@mail.gmail.com>
Message-ID: <CAOTb1weEKpzLLDKuHq8A_VQLfn4yO1wKEkqQDpH8epv+ZW_6Nw@mail.gmail.com>

On Fri, Oct 12, 2012 at 7:40 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2012/10/12 Chris Jerdonek <chris.jerdonek at gmail.com>:
>> On Fri, Oct 12, 2012 at 9:05 AM, benjamin.peterson
>> <python-checkins at python.org> wrote:
>>> http://hg.python.org/cpython/rev/de8787029fe4
>>> changeset:   79702:de8787029fe4
>>> branch:      3.3
>>> parent:      79701:d4ab5859721e
>>> parent:      79690:0cddf0bd19f8
>>
>> It looks like this 3.3 check-in also reverted previous valid work.
>> The following command shows the diff from the "mainline", which isn't
>> visible in the e-mail contents below:
>>
>> hg diff -r 0cddf0bd19f8 -r de8787029fe4
>>
>> It's hard for me to piece together exactly what happened, or else I
>> would try to fix it myself.
>
> Sorry about that. I merged 3.2 -> default instead of 3.2 -> 3.3 ->
> default and these were my attempts to fix it.

Are you already fixing it?  If not, it looks like I could fix it.
I'll start on that..

--Chris


>
>
> --
> Regards,
> Benjamin
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins

From chris.jerdonek at gmail.com  Sat Oct 13 05:44:09 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Fri, 12 Oct 2012 20:44:09 -0700
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> 3.3):
	merge heads
In-Reply-To: <CAOTb1weEKpzLLDKuHq8A_VQLfn4yO1wKEkqQDpH8epv+ZW_6Nw@mail.gmail.com>
References: <3XdYmG6jxSzR3X@mail.python.org>
	<CAOTb1wc0S-sJ4+UF3B8LZn4Uj=DkT0nCWazAbJs-bmWwfdD5RA@mail.gmail.com>
	<CAPZV6o_yFeZrWRr_fBtj3OOsqG-Uhqgs4W1ZkNvbH9s74kY44w@mail.gmail.com>
	<CAOTb1weEKpzLLDKuHq8A_VQLfn4yO1wKEkqQDpH8epv+ZW_6Nw@mail.gmail.com>
Message-ID: <CAOTb1weBTTd27e5LzRODwsqyPWkZoB4z0TXy+0NiB7pfKqDkww@mail.gmail.com>

On Fri, Oct 12, 2012 at 8:22 PM, Chris Jerdonek
<chris.jerdonek at gmail.com> wrote:
> On Fri, Oct 12, 2012 at 7:40 PM, Benjamin Peterson <benjamin at python.org> wrote:
>> 2012/10/12 Chris Jerdonek <chris.jerdonek at gmail.com>:
>>> On Fri, Oct 12, 2012 at 9:05 AM, benjamin.peterson
>>> <python-checkins at python.org> wrote:
>>>> http://hg.python.org/cpython/rev/de8787029fe4
>>>> changeset:   79702:de8787029fe4
>>>> branch:      3.3
>>>> parent:      79701:d4ab5859721e
>>>> parent:      79690:0cddf0bd19f8
>>>
>>> It looks like this 3.3 check-in also reverted previous valid work.
>>> The following command shows the diff from the "mainline", which isn't
>>> visible in the e-mail contents below:
>>>
>>> hg diff -r 0cddf0bd19f8 -r de8787029fe4
>>>
>>> It's hard for me to piece together exactly what happened, or else I
>>> would try to fix it myself.
>>
>> Sorry about that. I merged 3.2 -> default instead of 3.2 -> 3.3 ->
>> default and these were my attempts to fix it.
>
> Are you already fixing it?  If not, it looks like I could fix it.
> I'll start on that..

Okay, I believe it's fixed now.  I also did a sanity check on the
default branch by comparing the tip with the default branch prior to
those merges.

The only thing I didn't check is whether you have any changes in 3.2
or 3.3 that need to be merged to default.

--Chris

From swamidass at gmail.com  Sat Oct 13 05:24:57 2012
From: swamidass at gmail.com (S Joshua Swamidass)
Date: Sat, 13 Oct 2012 12:24:57 +0900
Subject: [Python-Dev] ConfigParser argparse integration module
Message-ID: <CAJzP+m+bbtRO2Hi_cRJWxvH2c7pMPdfuhvzDM7=CMOeyyE=42Q@mail.gmail.com>

Hello,

It seemed like there were several requests to enable ini file parsing with
argparse. I wrote module to do this.

I'd be curious to know if this is useful and anyone has a better approach.
See the code here:

http://pypi.python.org/pypi/ConfArgParse

Thanks!

S Joshua Swamidass
http://swami.wustl.edu/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121013/781785a6/attachment-0001.html>

From benjamin at python.org  Sat Oct 13 07:00:30 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 13 Oct 2012 01:00:30 -0400
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> 3.3):
	merge heads
In-Reply-To: <CAOTb1weBTTd27e5LzRODwsqyPWkZoB4z0TXy+0NiB7pfKqDkww@mail.gmail.com>
References: <3XdYmG6jxSzR3X@mail.python.org>
	<CAOTb1wc0S-sJ4+UF3B8LZn4Uj=DkT0nCWazAbJs-bmWwfdD5RA@mail.gmail.com>
	<CAPZV6o_yFeZrWRr_fBtj3OOsqG-Uhqgs4W1ZkNvbH9s74kY44w@mail.gmail.com>
	<CAOTb1weEKpzLLDKuHq8A_VQLfn4yO1wKEkqQDpH8epv+ZW_6Nw@mail.gmail.com>
	<CAOTb1weBTTd27e5LzRODwsqyPWkZoB4z0TXy+0NiB7pfKqDkww@mail.gmail.com>
Message-ID: <CAPZV6o9zXLcZxV4_MQq0__t47UL4qi_JG=-S=0Nqmgkpekt--A@mail.gmail.com>

2012/10/12 Chris Jerdonek <chris.jerdonek at gmail.com>:
> On Fri, Oct 12, 2012 at 8:22 PM, Chris Jerdonek
> <chris.jerdonek at gmail.com> wrote:
>> On Fri, Oct 12, 2012 at 7:40 PM, Benjamin Peterson <benjamin at python.org> wrote:
>>> 2012/10/12 Chris Jerdonek <chris.jerdonek at gmail.com>:
>>>> On Fri, Oct 12, 2012 at 9:05 AM, benjamin.peterson
>>>> <python-checkins at python.org> wrote:
>>>>> http://hg.python.org/cpython/rev/de8787029fe4
>>>>> changeset:   79702:de8787029fe4
>>>>> branch:      3.3
>>>>> parent:      79701:d4ab5859721e
>>>>> parent:      79690:0cddf0bd19f8
>>>>
>>>> It looks like this 3.3 check-in also reverted previous valid work.
>>>> The following command shows the diff from the "mainline", which isn't
>>>> visible in the e-mail contents below:
>>>>
>>>> hg diff -r 0cddf0bd19f8 -r de8787029fe4
>>>>
>>>> It's hard for me to piece together exactly what happened, or else I
>>>> would try to fix it myself.
>>>
>>> Sorry about that. I merged 3.2 -> default instead of 3.2 -> 3.3 ->
>>> default and these were my attempts to fix it.
>>
>> Are you already fixing it?  If not, it looks like I could fix it.
>> I'll start on that..
>
> Okay, I believe it's fixed now.  I also did a sanity check on the
> default branch by comparing the tip with the default branch prior to
> those merges.

Thanks for looking into that.

>
> The only thing I didn't check is whether you have any changes in 3.2
> or 3.3 that need to be merged to default.

Everything looks okay to me.



-- 
Regards,
Benjamin

From lukasz at langa.pl  Sat Oct 13 10:06:55 2012
From: lukasz at langa.pl (=?iso-8859-2?Q?=A3ukasz_Langa?=)
Date: Sat, 13 Oct 2012 10:06:55 +0200
Subject: [Python-Dev] ConfigParser argparse integration module
In-Reply-To: <CAJzP+m+bbtRO2Hi_cRJWxvH2c7pMPdfuhvzDM7=CMOeyyE=42Q@mail.gmail.com>
References: <CAJzP+m+bbtRO2Hi_cRJWxvH2c7pMPdfuhvzDM7=CMOeyyE=42Q@mail.gmail.com>
Message-ID: <17E1F2E3-73A4-4C6A-BEF0-5CCD7EBFE9B1@langa.pl>

Wiadomo?? napisana przez S Joshua Swamidass <swamidass at gmail.com> w dniu 13 pa? 2012, o godz. 05:24:

> Hello,
> 
> It seemed like there were several requests to enable ini file parsing with argparse. I wrote module to do this.
> I'd be curious to know if this is useful and anyone has a better approach.

This mailing list focuses on the development *of* Python so your message if off-topic for this list. I suggest python-announce-list at python.org or the generic python-list at python.org.

> http://pypi.python.org/pypi/ConfArgParse

That being said, a couple of remarks:

1. See configglue which does what your module tries to do.
2. A "Production/Stable" trove classifier without any unit tests does not look very convincing.
3. Your code is 2.x only.
4. ConfArgParse.args2config uses RawConfigParser whereas ConfArgParse._parse_config uses SafeConfigParser which will lead to inconsistent content handling.
5. Your code is not PEP8 compliant.
6. Long option names traditionally don't use underscores but dashes. E.g. not --conf_file but --conf-file.
7. There doesn't seem to be a homepage. The one you listed in setup.py is your homepage, not the project's homepage.
8. There doesn't seem to be a code repository for the project.

All in all: nice approach, promising project but definitely needs more work. Good luck :)

-- 
Best regards,
?ukasz Langa
Senior Systems Architecture Engineer

IT Infrastructure Department
Grupa Allegro Sp. z o.o.

http://lukasz.langa.pl/
+48 791 080 144

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121013/eba071c8/attachment.html>

From sdl.web at gmail.com  Sat Oct 13 08:40:50 2012
From: sdl.web at gmail.com (Leo)
Date: Sat, 13 Oct 2012 14:40:50 +0800
Subject: [Python-Dev] [BUG] Trailing spaces in pretty-printed JSON
Message-ID: <m2k3uuri19.fsf@gmail.com>

Use this script on a json file and observe all the trailing spaces
generated. (screenshot attached.)

#!/usr/bin/env python

"""
Pretty print json file.
"""

if __name__ == '__main__':
    import sys
    import json

    if '-h' in sys.argv or '--help' in sys.argv:
        print "Usage: ppjson <FILE>"
        exit(0)

    assert sys.argv[1], "No file provided"
    with open(sys.argv[1]) as f:
        print json.dumps(json.load(f), indent=4)


-------------- next part --------------
A non-text attachment was scrubbed...
Name: py-json-bug.png
Type: image/png
Size: 37869 bytes
Desc: py-json-bug.png
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121013/9a9267fb/attachment.png>

From python-dev at masklinn.net  Sat Oct 13 20:06:20 2012
From: python-dev at masklinn.net (Xavier Morel)
Date: Sat, 13 Oct 2012 20:06:20 +0200
Subject: [Python-Dev] [BUG] Trailing spaces in pretty-printed JSON
In-Reply-To: <m2k3uuri19.fsf@gmail.com>
References: <m2k3uuri19.fsf@gmail.com>
Message-ID: <33E0EAD5-8BEE-4574-ABEC-A342BDADA34D@masklinn.net>

On 2012-10-13, at 08:40 , Leo wrote:

> Use this script on a json file and observe all the trailing spaces
> generated. (screenshot attached.)

1. Why didn't you report that on the tracker?
2. Why are you rewriting json.tool?

From rosuav at gmail.com  Sat Oct 13 20:08:31 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 14 Oct 2012 05:08:31 +1100
Subject: [Python-Dev] [BUG] Trailing spaces in pretty-printed JSON
In-Reply-To: <m2k3uuri19.fsf@gmail.com>
References: <m2k3uuri19.fsf@gmail.com>
Message-ID: <CAPTjJmqUhPjfvMEuQOyLvrNnyFO0xTwKz1ZNiAjh-qDs1LGTiQ@mail.gmail.com>

On Sat, Oct 13, 2012 at 5:40 PM, Leo <sdl.web at gmail.com> wrote:
> Use this script on a json file and observe all the trailing spaces
> generated. (screenshot attached.)

Confirmed as still the case in Python 3 (specifically, with an early
alpha of 3.3 and with 3.2 for Windows). It's because the item
separator is '; ' and the newline and indent are appended to that.
That can be overridden with:

json.dumps({"asdf":"123","qwer":"234","zxcv":"345"},indent=4,separators=(',',':
'))

but that compacts everything, not sure if that's what you want.

The code in question is in Lib/json/encoder.py in the source tree; I'm
sure you could post on the tracker (hint) with a patch (hint hint) if
you want it to strip spaces followed by newlines.

ChrisA

From solipsis at pitrou.net  Sun Oct 14 12:36:18 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 14 Oct 2012 12:36:18 +0200
Subject: [Python-Dev] cpython: Force 32bit Keccak implementation on
 SPARC. It look like the Solaris CC
References: <3XfPRg5BPZzR8T@mail.python.org>
Message-ID: <20121014123618.730bf9df@pitrou.net>

On Sun, 14 Oct 2012 02:53:51 +0200 (CEST)
christian.heimes <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/b397b0c08f69
> changeset:   79717:b397b0c08f69
> parent:      79715:3465ef48d0f5
> user:        Christian Heimes <christian at cheimes.de>
> date:        Sun Oct 14 02:52:01 2012 +0200
> summary:
>   Force 32bit Keccak implementation on SPARC. It look like the Solaris CC compiler doesn't like the address alignment.

I think SPARC (the architecture) disallows unaligned accesses to
memory. I don't think it is specific to the Solaris CC compiler.

Regards

Antoine.



From martin at v.loewis.de  Sun Oct 14 16:04:10 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Sun, 14 Oct 2012 16:04:10 +0200
Subject: [Python-Dev] cpython: Force 32bit Keccak implementation on
 SPARC. It look like the Solaris CC
In-Reply-To: <20121014123618.730bf9df@pitrou.net>
References: <3XfPRg5BPZzR8T@mail.python.org>
	<20121014123618.730bf9df@pitrou.net>
Message-ID: <20121014160410.Horde.GZ2lUcL8999QesZay0W13vA@webmail.df.eu>


Zitat von Antoine Pitrou <solipsis at pitrou.net>:

> I think SPARC (the architecture) disallows unaligned accesses to
> memory. I don't think it is specific to the Solaris CC compiler.

That's correct. Unaligned memory operations need to use memcpy.

Regards,
Martin



From christian at python.org  Sun Oct 14 17:16:15 2012
From: christian at python.org (Christian Heimes)
Date: Sun, 14 Oct 2012 17:16:15 +0200
Subject: [Python-Dev] cpython: Force 32bit Keccak implementation on
 SPARC. It look like the Solaris CC
In-Reply-To: <20121014123618.730bf9df@pitrou.net>
References: <3XfPRg5BPZzR8T@mail.python.org>
	<20121014123618.730bf9df@pitrou.net>
Message-ID: <507AD73F.20702@python.org>

Am 14.10.2012 12:36, schrieb Antoine Pitrou:
> I think SPARC (the architecture) disallows unaligned accesses to
> memory. I don't think it is specific to the Solaris CC compiler.

I've updated the comment accordingly. Unaligned access is just used by
the opt64 implementation of Keccak. opt32 is more conservative and works
on 32 and 64bit platforms.


From maker.py at gmail.com  Sun Oct 14 19:38:50 2012
From: maker.py at gmail.com (=?UTF-8?Q?Michele_Orr=C3=B9?=)
Date: Sun, 14 Oct 2012 19:38:50 +0200
Subject: [Python-Dev] Issues Pending Review
Message-ID: <CAMfKjGVhDUS9YMyO40joy46xw1WP4SW0ZtkAMAq7SYpP_toYaw@mail.gmail.com>

Hello, I have some issues pending 'patch review' hanging for more than
two weeks, could somebody please check them out?

HTTPServer does not correctly handle bad headers /
http://bugs.python.org/issue16083
rlcompleter adds builtins when custom dict is used /
http://bugs.python.org/issue5256
a -= b should be fast if a is a small set and b is a large set /
http://bugs.python.org/issue8425 (note: hettinger said to wait for his
-noisy before applying)
Add support for IEEE 754 contexts to decimal module. /
http://bugs.python.org/issue8786

-- 
?

From martin at v.loewis.de  Sun Oct 14 20:38:01 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Sun, 14 Oct 2012 20:38:01 +0200
Subject: [Python-Dev] Issues Pending Review
In-Reply-To: <CAMfKjGVhDUS9YMyO40joy46xw1WP4SW0ZtkAMAq7SYpP_toYaw@mail.gmail.com>
References: <CAMfKjGVhDUS9YMyO40joy46xw1WP4SW0ZtkAMAq7SYpP_toYaw@mail.gmail.com>
Message-ID: <20121014203801.Horde.Kr3CfcL8999QewaJi39AZAA@webmail.df.eu>


Zitat von Michele Orr? <maker.py at gmail.com>:

> Hello, I have some issues pending 'patch review' hanging for more than
> two weeks, could somebody please check them out?

In case nobody picks it up, my 5-for-1 offer still stands: if you review
five issues, I'll review one of yours.

Regards,
Martin



From maker.py at gmail.com  Sun Oct 14 23:43:37 2012
From: maker.py at gmail.com (=?UTF-8?Q?Michele_Orr=C3=B9?=)
Date: Sun, 14 Oct 2012 23:43:37 +0200
Subject: [Python-Dev] [Core-mentorship] Issues Pending Review
In-Reply-To: <507B0A24.7090807@udel.edu>
References: <CAMfKjGVhDUS9YMyO40joy46xw1WP4SW0ZtkAMAq7SYpP_toYaw@mail.gmail.com>
	<507B0A24.7090807@udel.edu>
Message-ID: <CAMfKjGX3T-ah1R8suHs_AdGVgJxgADgRm9UKHgSgpH4anRbatg@mail.gmail.com>

> In case nobody picks it up, my 5-for-1 offer still stands: if you review
> five issues, I'll review one of yours.
:D merwork saves?




>> HTTPServer does not correctly handle bad headers /
>> http://bugs.python.org/issue16083
> It does not seem clear what should be done and which versions to change.
If there is something more than than the "security" flag, may you
explain further what is unclear on the bugtracker?

> And it was reviewed 2 days after posting. So why list it here?
Sorry, it was not meant to be there. I probably picked the wrong one,
http://bugs.python.org/issue16081 / Fix compile warnings in thread_pthread.h

>> Add support for IEEE 754 contexts to decimal module. /
>> http://bugs.python.org/issue8786
> reviewed
Thanks.

-- 
?

From techtonik at gmail.com  Mon Oct 15 09:43:14 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 15 Oct 2012 10:43:14 +0300
Subject: [Python-Dev] PEP or formal description of Python module structure
Message-ID: <CAPkN8x+TaG8R5NTbvL_GCtttj3covuW9Z-WGVu4bMBYMxBDW6w@mail.gmail.com>

Hi,

I am trying to figure out what Python module is internally (WIP
http://wiki.python.org/moin/techtonik)? Is there already a good piece
of documentation that I missed that can answer all these questions
already?

...what properties do you get in empty Python module (__doc__, __name__, ...)?
...what of those properties are set by the language standard
(required) and what are just optional helpers from/for the
interpreter?
...what is the proper way to inspect modules and access their
properties at runtime?
...what optional properties are defined by language standard that can
you set implicitly?
...how each internal property is used by the interpreter and when it appeared?
...what is the proper way to create module namespace at run-time?

I'd like to see some kind of reference of properties with
classification required/optional, purpose, when set, who uses, how to
approach etc. in one place.

--
anatoly t.

From techtonik at gmail.com  Mon Oct 15 13:07:07 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 15 Oct 2012 14:07:07 +0300
Subject: [Python-Dev] urlretrieve regression in Python 3
Message-ID: <CAPkN8xL-wqmJNLp74370f3a0Z0uWSw=mg6fQqqf-ErRACK3hTg@mail.gmail.com>

Can anybody raise the priority of this issue to make it visible during
the next bug hunting day?
http://bugs.python.org/issue10836

From xavier.combelle at free.fr  Mon Oct 15 14:15:02 2012
From: xavier.combelle at free.fr (Xavier Combelle)
Date: Mon, 15 Oct 2012 14:15:02 +0200
Subject: [Python-Dev] PEP or formal description of Python module
	structure
In-Reply-To: <CAPkN8x+TaG8R5NTbvL_GCtttj3covuW9Z-WGVu4bMBYMxBDW6w@mail.gmail.com>
References: <CAPkN8x+TaG8R5NTbvL_GCtttj3covuW9Z-WGVu4bMBYMxBDW6w@mail.gmail.com>
Message-ID: <507BFE46.8000407@free.fr>

Le 15/10/2012 09:43, anatoly techtonik a ?crit :
> Hi,
>
> I am trying to figure out what Python module is internally (WIP
> http://wiki.python.org/moin/techtonik)? Is there already a good piece
> of documentation that I missed that can answer all these questions
> already?
>
> ...what properties do you get in empty Python module (__doc__, __name__, ...)?
> ...what of those properties are set by the language standard
> (required) and what are just optional helpers from/for the
> interpreter?
> ...what is the proper way to inspect modules and access their
> properties at runtime?
> ...what optional properties are defined by language standard that can
> you set implicitly?
> ...how each internal property is used by the interpreter and when it appeared?
> ...what is the proper way to create module namespace at run-time?
>
> I'd like to see some kind of reference of properties with
> classification required/optional, purpose, when set, who uses, how to
> approach etc. in one place.
>
> --
> anatoly t.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/xavier.combelle%40free.fr
from http://docs.python.org/reference/datamodel.html


Modules

    Modules are imported by the import
    <http://docs.python.org/reference/simple_stmts.html#import>
    statement (see section /The import statement/
    <http://docs.python.org/reference/simple_stmts.html#import>). A
    module object has a namespace implemented by a dictionary object
    (this is the dictionary referenced by the func_globals attribute of
    functions defined in the module). Attribute references are
    translated to lookups in this dictionary, e.g., m.x is equivalent to
    m.__dict__["x"]. A module object does not contain the code object
    used to initialize the module (since it isn't needed once the
    initialization is done).

    Attribute assignment updates the module's namespace dictionary,
    e.g., m.x = 1 is equivalent to m.__dict__["x"] = 1.

    Special read-only attribute: __dict__ is the module's namespace as a
    dictionary object.

    *CPython implementation detail:* Because of the way CPython clears
    module dictionaries, the module dictionary will be cleared when the
    module falls out of scope even if the dictionary still has live
    references. To avoid this, copy the dictionary or keep the module
    around while using its dictionary directly.

    Predefined (writable) attributes: __name__ is the module's name;
    __doc__ is the module's documentation string, or None if
    unavailable; __file__ is the pathname of the file from which the
    module was loaded, if it was loaded from a file. The __file__
    attribute is not present for C modules that are statically linked
    into the interpreter; for extension modules loaded dynamically from
    a shared library, it is the pathname of the shared library file.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121015/dd187a6f/attachment.html>

From peck at us.ibm.com  Mon Oct 15 18:04:31 2012
From: peck at us.ibm.com (Jon K Peck)
Date: Mon, 15 Oct 2012 10:04:31 -0600
Subject: [Python-Dev] AUTO: Jon K Peck is out of the office
Message-ID: <OFD12DA50A.1E88A30D-ON87257A98.00584E26-87257A98.00584E26@us.ibm.com>



I am out of the office until 10/16/2012.

I will be out of the office Monday 10/15/12.  I will not have email access
during this time.


Note: This is an automated response to your message  "Python-Dev Digest,
Vol 111, Issue 31" sent on 10/15/2012 4:00:04.

This is the only notification you will receive while this person is away.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121015/1d4ce383/attachment.html>

From brian at python.org  Mon Oct 15 18:40:06 2012
From: brian at python.org (Brian Curtin)
Date: Mon, 15 Oct 2012 11:40:06 -0500
Subject: [Python-Dev] AUTO: Jon K Peck is out of the office
In-Reply-To: <OFD12DA50A.1E88A30D-ON87257A98.00584E26-87257A98.00584E26@us.ibm.com>
References: <OFD12DA50A.1E88A30D-ON87257A98.00584E26-87257A98.00584E26@us.ibm.com>
Message-ID: <CAD+XWwr6_XT=ntWdSn5JrbohgVptyn+aMmAAA2CumAeEYN_Y-w@mail.gmail.com>

On Mon, Oct 15, 2012 at 11:04 AM, Jon K Peck <peck at us.ibm.com> wrote:
> I am out of the office until 10/16/2012.
>
> I will be out of the office Monday 10/15/12.  I will not have email access
> during this time.
>
>
> Note: This is an automated response to your message  "Python-Dev Digest, Vol
> 111, Issue 31" sent on 10/15/2012 4:00:04.
>
> This is the only notification you will receive while this person is away.

Enjoy your day off, John K Peck.

From barry at python.org  Mon Oct 15 18:45:05 2012
From: barry at python.org (Barry Warsaw)
Date: Mon, 15 Oct 2012 12:45:05 -0400
Subject: [Python-Dev] AUTO: Jon K Peck is out of the office
In-Reply-To: <CAD+XWwr6_XT=ntWdSn5JrbohgVptyn+aMmAAA2CumAeEYN_Y-w@mail.gmail.com>
References: <OFD12DA50A.1E88A30D-ON87257A98.00584E26-87257A98.00584E26@us.ibm.com>
	<CAD+XWwr6_XT=ntWdSn5JrbohgVptyn+aMmAAA2CumAeEYN_Y-w@mail.gmail.com>
Message-ID: <20121015124505.40db78d1@resist.wooz.org>

On Oct 15, 2012, at 11:40 AM, Brian Curtin wrote:

>On Mon, Oct 15, 2012 at 11:04 AM, Jon K Peck <peck at us.ibm.com> wrote:
>> I am out of the office until 10/16/2012.
>>
>> I will be out of the office Monday 10/15/12.  I will not have email access
>> during this time.
>>
>>
>> Note: This is an automated response to your message  "Python-Dev Digest, Vol
>> 111, Issue 31" sent on 10/15/2012 4:00:04.
>>
>> This is the only notification you will receive while this person is away.
>
>Enjoy your day off, John K Peck.

Indeed.  And when you get back, please fix your vacation program. :)

-Barry

From chris.jerdonek at gmail.com  Tue Oct 16 03:09:53 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Mon, 15 Oct 2012 18:09:53 -0700
Subject: [Python-Dev] [Python-checkins] cpython (2.7): don't expect
 warnings from doctests if they can't run
In-Reply-To: <3Xgd0y4fQxzNmb@mail.python.org>
References: <3Xgd0y4fQxzNmb@mail.python.org>
Message-ID: <CAOTb1wc-OxJMDh4qAizeVOHb4qMa8A1_5ssAKOQB7__-QS-kZw@mail.gmail.com>

On Mon, Oct 15, 2012 at 5:38 PM, benjamin.peterson
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/d39f4a92d823
> changeset:   79725:d39f4a92d823
> branch:      2.7
> user:        Benjamin Peterson <benjamin at python.org>
> date:        Mon Oct 15 20:38:21 2012 -0400
> summary:
>   don't expect warnings from doctests if they can't run
> -                ("class Tester is deprecated", DeprecationWarning)]
> +                depcreations.append(("class Tester is deprecated", DeprecationWarning))

There is a typo here.

--Chris

From trent at snakebite.org  Tue Oct 16 06:33:53 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 16 Oct 2012 00:33:53 -0400
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
Message-ID: <20121016043352.GA21441@snakebite.org>

    Any objections to regenerating configure with autoconf 2.69?  The
    current version is based off 2.68, which was release on the 22nd
    of September 2010.  2.69 was released on the 24th of April, 2012.

    (There are some fixes for the more esoteric UNIX platforms that
     Snakebite will benefit from.  Also, I struggled to find a box
     with 2.68 even installed when I needed to regenerate configure
     this weekend; it seems all package management/ports have bumped
     to 2.69.)

    If there are no objections, can it be applied across the board?
    2.7, 3.2, 3.3 and 3.x?

        Trent.

From nad at acm.org  Tue Oct 16 08:18:04 2012
From: nad at acm.org (Ned Deily)
Date: Mon, 15 Oct 2012 23:18:04 -0700
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
References: <20121016043352.GA21441@snakebite.org>
Message-ID: <nad-5C62E0.23180415102012@news.gmane.org>

In article <20121016043352.GA21441 at snakebite.org>,
 Trent Nelson <trent at snakebite.org> wrote:
>     Any objections to regenerating configure with autoconf 2.69?  The
>     current version is based off 2.68, which was release on the 22nd
>     of September 2010.  2.69 was released on the 24th of April, 2012.
> 
>     (There are some fixes for the more esoteric UNIX platforms that
>      Snakebite will benefit from.  Also, I struggled to find a box
>      with 2.68 even installed when I needed to regenerate configure
>      this weekend; it seems all package management/ports have bumped
>      to 2.69.)
> 
>     If there are no objections, can it be applied across the board?
>     2.7, 3.2, 3.3 and 3.x?

I believe it is the case that we don't currently require a specific 
version of autoconf.  autoconf is only run when making a committer makes 
a change to configure.ac et al and then it is run on the committer's 
machine.  So whatever is checked in for those files reflects the version 
of autoconf that the committer used.

-- 
 Ned Deily,
 nad at acm.org


From rdmurray at bitdance.com  Tue Oct 16 09:12:35 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 16 Oct 2012 03:12:35 -0400
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <nad-5C62E0.23180415102012@news.gmane.org>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
Message-ID: <20121016071236.0792D2500FA@webabinitio.net>

On Mon, 15 Oct 2012 23:18:04 -0700, Ned Deily <nad at acm.org> wrote:
> In article <20121016043352.GA21441 at snakebite.org>,
>  Trent Nelson <trent at snakebite.org> wrote:
> >     Any objections to regenerating configure with autoconf 2.69?  The
> >     current version is based off 2.68, which was release on the 22nd
> >     of September 2010.  2.69 was released on the 24th of April, 2012.
> > 
> >     (There are some fixes for the more esoteric UNIX platforms that
> >      Snakebite will benefit from.  Also, I struggled to find a box
> >      with 2.68 even installed when I needed to regenerate configure
> >      this weekend; it seems all package management/ports have bumped
> >      to 2.69.)
> > 
> >     If there are no objections, can it be applied across the board?
> >     2.7, 3.2, 3.3 and 3.x?
> 
> I believe it is the case that we don't currently require a specific 
> version of autoconf.  autoconf is only run when making a committer makes 
> a change to configure.ac et al and then it is run on the committer's 
> machine.  So whatever is checked in for those files reflects the version 
> of autoconf that the committer used.

My understanding is that we use a specific version of autoconf.
The reason is that otherwise we end up with useless churn in the repo
as the generated file changes when different committers use different
versions.  In the past we have had issues with a new autoconf version
actually breaking the Python build, so we also need to test a new version
before switching to it.

--David

From trent at snakebite.org  Tue Oct 16 10:04:36 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 16 Oct 2012 04:04:36 -0400
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <20121016071236.0792D2500FA@webabinitio.net>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
Message-ID: <20121016080435.GB21441@snakebite.org>

On Tue, Oct 16, 2012 at 12:12:35AM -0700, R. David Murray wrote:
> On Mon, 15 Oct 2012 23:18:04 -0700, Ned Deily <nad at acm.org> wrote:
> > In article <20121016043352.GA21441 at snakebite.org>,
> >  Trent Nelson <trent at snakebite.org> wrote:
> > >     Any objections to regenerating configure with autoconf 2.69?  The
> > >     current version is based off 2.68, which was release on the 22nd
> > >     of September 2010.  2.69 was released on the 24th of April, 2012.
> > > 
> > >     (There are some fixes for the more esoteric UNIX platforms that
> > >      Snakebite will benefit from.  Also, I struggled to find a box
> > >      with 2.68 even installed when I needed to regenerate configure
> > >      this weekend; it seems all package management/ports have bumped
> > >      to 2.69.)
> > > 
> > >     If there are no objections, can it be applied across the board?
> > >     2.7, 3.2, 3.3 and 3.x?
> > 
> > I believe it is the case that we don't currently require a specific 
> > version of autoconf.  autoconf is only run when making a committer makes 
> > a change to configure.ac et al and then it is run on the committer's 
> > machine.  So whatever is checked in for those files reflects the version 
> > of autoconf that the committer used.
> 
> My understanding is that we use a specific version of autoconf.
> The reason is that otherwise we end up with useless churn in the repo
> as the generated file changes when different committers use different
> versions.

    Indeed; the 2.68->2.69 bump results in a 746 line diff for
    configure.

> In the past we have had issues with a new autoconf version
> actually breaking the Python build, so we also need to test a new version
> before switching to it.

    If there are no objections in principle to the bump (i.e. nobody is
    aware of any existing 2.69 incompatibilities), my plan is to bump,
    then make sure all the build slaves and Snakebite machines work with
    the new version.

    If 2.69 introduces insurmountable build issues, I'll simply revert
    back to 2.68 (with some notes at the top of configure.ac mentioning
    the issues with 2.69).


        Trent.

From nad at acm.org  Tue Oct 16 10:04:46 2012
From: nad at acm.org (Ned Deily)
Date: Tue, 16 Oct 2012 01:04:46 -0700
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
Message-ID: <nad-EE1DCA.01044616102012@news.gmane.org>

In article <20121016071236.0792D2500FA at webabinitio.net>,
 "R. David Murray" <rdmurray at bitdance.com> wrote:
> My understanding is that we use a specific version of autoconf.
> The reason is that otherwise we end up with useless churn in the repo
> as the generated file changes when different committers use different
> versions.  In the past we have had issues with a new autoconf version
> actually breaking the Python build, so we also need to test a new version
> before switching to it.

Well, there was a test in the past for a specific version of autoconf 
but I believe that was removed.  I see that the default branch 
configure.ac contains AC_PREREQ(2.65) but the most recently checked-in 
version of configure was generated with 2.68.  I understand about the 
desirability of avoiding a lot of churn although it seems to me to not 
be a huge thing; build breaking is another matter, of course.  If we are 
going to mandate a specific version again, that should be documented and 
checked for.

-- 
 Ned Deily,
 nad at acm.org


From storchaka at gmail.com  Tue Oct 16 10:21:39 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 16 Oct 2012 11:21:39 +0300
Subject: [Python-Dev] Issues Pending Review
In-Reply-To: <20121014203801.Horde.Kr3CfcL8999QewaJi39AZAA@webmail.df.eu>
References: <CAMfKjGVhDUS9YMyO40joy46xw1WP4SW0ZtkAMAq7SYpP_toYaw@mail.gmail.com>
	<20121014203801.Horde.Kr3CfcL8999QewaJi39AZAA@webmail.df.eu>
Message-ID: <k5j5ek$je6$1@ger.gmane.org>

On 14.10.12 21:38, martin at v.loewis.de wrote:
> In case nobody picks it up, my 5-for-1 offer still stands: if you review
> five issues, I'll review one of yours.

What 50 issues you want I'll review.



From neologix at free.fr  Tue Oct 16 10:43:37 2012
From: neologix at free.fr (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Tue, 16 Oct 2012 10:43:37 +0200
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <20121016071236.0792D2500FA@webabinitio.net>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
Message-ID: <CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>

> My understanding is that we use a specific version of autoconf.
> The reason is that otherwise we end up with useless churn in the repo
> as the generated file changes when different committers use different
> versions.  In the past we have had issues with a new autoconf version
> actually breaking the Python build, so we also need to test a new version
> before switching to it.

Well, so I guess all committers will have to use the same
Linux/FreeBSD/whatever distribution then?
AFAICT there's no requirement regarding the mercurial version used by
committers either.

Charles

From trent at snakebite.org  Tue Oct 16 10:59:18 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 16 Oct 2012 04:59:18 -0400
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <nad-EE1DCA.01044616102012@news.gmane.org>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<nad-EE1DCA.01044616102012@news.gmane.org>
Message-ID: <20121016085918.GC21441@snakebite.org>

On Tue, Oct 16, 2012 at 01:04:46AM -0700, Ned Deily wrote:
> In article <20121016071236.0792D2500FA at webabinitio.net>,
>  "R. David Murray" <rdmurray at bitdance.com> wrote:
> > My understanding is that we use a specific version of autoconf.
> > The reason is that otherwise we end up with useless churn in the repo
> > as the generated file changes when different committers use different
> > versions.  In the past we have had issues with a new autoconf version
> > actually breaking the Python build, so we also need to test a new version
> > before switching to it.
> 
> Well, there was a test in the past for a specific version of autoconf 
> but I believe that was removed.  I see that the default branch 
> configure.ac contains AC_PREREQ(2.65) but the most recently checked-in 
> version of configure was generated with 2.68.

    That's a good point regarding AC_PREREQ.  I think there's been a
    general unspoken rule to try and minimize 'configure' churn, which
    means using the same version of autoconf that the previous person
    used.

> I understand about the 
> desirability of avoiding a lot of churn although it seems to me to not 
> be a huge thing;

    I don't know, the churn is usually offensive enough to warrant using
    the same autoconf version.  It's impossible to vet a configure.ac
    change if you're regenerating configure with a different version of
    autoconf.

> build breaking is another matter, of course.  If we are 
> going to mandate a specific version again, that should be documented and 
> checked for.

    My preference: bump to 2.69 and set AC_PREREQ(2.69).  If 2.69 proves
    unworkable, revert back to 2.68 and AC_PREREQ(2.68).

    I definitely like the idea of explicitly setting the version being
    used via AC_PREREQ, as this will prevent accidental version churn
    being checked in by a committer inadvertently.

> -- 
>  Ned Deily,
>  nad at acm.org

        Trent.

From trent at snakebite.org  Tue Oct 16 11:05:23 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 16 Oct 2012 05:05:23 -0400
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>
Message-ID: <20121016090523.GD21441@snakebite.org>

On Tue, Oct 16, 2012 at 01:43:37AM -0700, Charles-Fran?ois Natali wrote:
> > My understanding is that we use a specific version of autoconf.
> > The reason is that otherwise we end up with useless churn in the repo
> > as the generated file changes when different committers use different
> > versions.  In the past we have had issues with a new autoconf version
> > actually breaking the Python build, so we also need to test a new version
> > before switching to it.
> 
> Well, so I guess all committers will have to use the same
> Linux/FreeBSD/whatever distribution then?
> AFAICT there's no requirement regarding the mercurial version used by
> committers either.

    Autoconf is a special case though.  Different versions of autoconf
    produce wildly different outputs for 'configure', making it impossible
    to vet configure.ac changes by reviewing the configure diff.

    It doesn't matter which OS or Mercurial version a developer uses as
    they don't implicitly affect any versioned resources; autoconf does.

        Trent.

From stefan at bytereef.org  Tue Oct 16 10:59:34 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Tue, 16 Oct 2012 10:59:34 +0200
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>
Message-ID: <20121016085934.GA2995@sleipnir.bytereef.org>

Charles-Fran?ois Natali <neologix at free.fr> wrote:
> Well, so I guess all committers will have to use the same
> Linux/FreeBSD/whatever distribution then?
> AFAICT there's no requirement regarding the mercurial version used by
> committers either.

It should be sufficient to install autoconf-x.y into /home/user/bin or
something similar. Installing autoconf from source really takes about
3 minutes.


Stefan Krah




From neologix at free.fr  Tue Oct 16 11:17:35 2012
From: neologix at free.fr (=?ISO-8859-1?Q?Charles=2DFran=E7ois_Natali?=)
Date: Tue, 16 Oct 2012 11:17:35 +0200
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <20121016085934.GA2995@sleipnir.bytereef.org>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>
	<20121016085934.GA2995@sleipnir.bytereef.org>
Message-ID: <CAH_1eM3PH1iL7hXDa9kd_7iaJLiqc+BQZLRkt5sHqakZhsNfzA@mail.gmail.com>

> It should be sufficient to install autoconf-x.y into /home/user/bin or
> something similar. Installing autoconf from source really takes about
> 3 minutes.

Well, maybe, maybe not.
autoconf depends on a least m4 and Perl, and you may very well have a
compatibility issue here.
That's why most distributions have package managers, and in 2012 we're
past the './configure && make && make install".

> It doesn't matter which OS or Mercurial version a developer uses as
> they don't implicitly affect any versioned resources; autoconf does.

If you're worried about the noise in diff, it's never been a problem
at least to me (just don't post a configure diff for review, the
configure.ac is enough).

If you're worried about runtime compatibility, then autoconf is not
your only worry. Proper build also depends on the target shell, target
toolchain (gcc, libc, etc).

From trent at snakebite.org  Tue Oct 16 11:20:26 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 16 Oct 2012 05:20:26 -0400
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <20121016071236.0792D2500FA@webabinitio.net>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
Message-ID: <20121016092026.GE21441@snakebite.org>

On Tue, Oct 16, 2012 at 12:12:35AM -0700, R. David Murray wrote:
> On Mon, 15 Oct 2012 23:18:04 -0700, Ned Deily <nad at acm.org> wrote:
> > In article <20121016043352.GA21441 at snakebite.org>,
> >  Trent Nelson <trent at snakebite.org> wrote:
> > >     Any objections to regenerating configure with autoconf 2.69?  The
> > >     current version is based off 2.68, which was release on the 22nd
> > >     of September 2010.  2.69 was released on the 24th of April, 2012.
> > > 
> > >     (There are some fixes for the more esoteric UNIX platforms that
> > >      Snakebite will benefit from.  Also, I struggled to find a box
> > >      with 2.68 even installed when I needed to regenerate configure
> > >      this weekend; it seems all package management/ports have bumped
> > >      to 2.69.)
> > > 
> > >     If there are no objections, can it be applied across the board?
> > >     2.7, 3.2, 3.3 and 3.x?
> > 
> > I believe it is the case that we don't currently require a specific 
> > version of autoconf.  autoconf is only run when making a committer makes 
> > a change to configure.ac et al and then it is run on the committer's 
> > machine.  So whatever is checked in for those files reflects the version 
> > of autoconf that the committer used.
> 
> My understanding is that we use a specific version of autoconf.
> The reason is that otherwise we end up with useless churn in the repo
> as the generated file changes when different committers use different
> versions.  In the past we have had issues with a new autoconf version
> actually breaking the Python build, so we also need to test a new version
> before switching to it.

    Just to put this in perspective, here's a change I made to
    configure.ac over the weekend as part of #15819:

% hg diff configure.ac                               
diff -r 51ce9830d85a configure.ac
--- a/configure.ac      Sat Oct 13 11:58:23 2012 -0400
+++ b/configure.ac      Tue Oct 16 09:12:56 2012 +0000
@@ -9,6 +9,9 @@
 
 AC_INIT(python, PYTHON_VERSION, http://bugs.python.org/)
 
+BUILDDIR="`pwd`"
+AC_SUBST(BUILDDIR)
+
 AC_SUBST(HGVERSION)
 AC_SUBST(HGTAG)
 AC_SUBST(HGBRANCH)

    With the correct version of autoconf (2.68), this generates the
    following diff to configure:

% hg diff configure
diff -r 51ce9830d85a configure
--- a/configure Sat Oct 13 11:58:23 2012 -0400
+++ b/configure Tue Oct 16 09:15:04 2012 +0000
@@ -714,6 +714,7 @@
 HGBRANCH
 HGTAG
 HGVERSION
+BUILDDIR
 target_alias
 host_alias
 build_alias
@@ -2719,6 +2720,9 @@
 
 
 
+BUILDDIR="`pwd`"
+
+
 

    However, the 2.69 version of autoconf generates this monster of a
    configure diff:

        http://hydrogen.snakebite.net/~cpython/configure.diff

    As you can see, it's impossible to figure out what my configure.ac
    change was from that diff.  Which is why I'm proposing we leverage
    AC_PREREQ to prevent inadvertent autoconf version churn.

        Trent.

From solipsis at pitrou.net  Tue Oct 16 11:27:24 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 16 Oct 2012 11:27:24 +0200
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>
	<20121016090523.GD21441@snakebite.org>
Message-ID: <20121016112724.04201f60@pitrou.net>

On Tue, 16 Oct 2012 05:05:23 -0400
Trent Nelson <trent at snakebite.org> wrote:
> On Tue, Oct 16, 2012 at 01:43:37AM -0700, Charles-Fran?ois Natali wrote:
> > > My understanding is that we use a specific version of autoconf.
> > > The reason is that otherwise we end up with useless churn in the repo
> > > as the generated file changes when different committers use different
> > > versions.  In the past we have had issues with a new autoconf version
> > > actually breaking the Python build, so we also need to test a new version
> > > before switching to it.
> > 
> > Well, so I guess all committers will have to use the same
> > Linux/FreeBSD/whatever distribution then?
> > AFAICT there's no requirement regarding the mercurial version used by
> > committers either.
> 
>     Autoconf is a special case though.  Different versions of autoconf
>     produce wildly different outputs for 'configure', making it impossible
>     to vet configure.ac changes by reviewing the configure diff.

Isn't it enough to review the configure.ac diff?

Regards

Antoine.



From trent at snakebite.org  Tue Oct 16 11:32:25 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 16 Oct 2012 05:32:25 -0400
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <CAH_1eM3PH1iL7hXDa9kd_7iaJLiqc+BQZLRkt5sHqakZhsNfzA@mail.gmail.com>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>
	<20121016085934.GA2995@sleipnir.bytereef.org>
	<CAH_1eM3PH1iL7hXDa9kd_7iaJLiqc+BQZLRkt5sHqakZhsNfzA@mail.gmail.com>
Message-ID: <20121016093220.GF21441@snakebite.org>

On Tue, Oct 16, 2012 at 02:17:35AM -0700, Charles-Fran?ois Natali wrote:
> > It should be sufficient to install autoconf-x.y into /home/user/bin or
> > something similar. Installing autoconf from source really takes about
> > 3 minutes.
> 
> Well, maybe, maybe not.
> autoconf depends on a least m4 and Perl, and you may very well have a
> compatibility issue here.
> That's why most distributions have package managers, and in 2012 we're
> past the './configure && make && make install".

    If you're unable to get 2.69 (or whatever) installed for whatever
    reason, you can always just commit configure.ac and ask another
    committer to regenerate configure with the correct version.

    (In my case, I couldn't find a single Snakebite box that had 2.68
     installed; out of 20-something hosts, everything had 2.69, which
     is partly why I'd like to bump to 2.69.)

> > It doesn't matter which OS or Mercurial version a developer uses as
> > they don't implicitly affect any versioned resources; autoconf does.
> 
> If you're worried about the noise in diff, it's never been a problem
> at least to me (just don't post a configure diff for review, the
> configure.ac is enough).
> 
> If you're worried about runtime compatibility, then autoconf is not
> your only worry. Proper build also depends on the target shell, target
> toolchain (gcc, libc, etc).

    Ignoring diff noise, what I'm suggesting is that we formalize the
    version used to build configure, as opposed to just accepting that
    "configure was built with whatever arbitrary version of autoconf
    the last developer that modified configure.ac had installed".

    If you look at configure's history, we pretty much already follow
    this informally.

    Anyway, back to the original question: does anyone know of reasons
    we shouldn't bump to 2.69?  Any known incompatibilities?

        Trent.

From petri at digip.org  Tue Oct 16 11:53:55 2012
From: petri at digip.org (Petri Lehtinen)
Date: Tue, 16 Oct 2012 12:53:55 +0300
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <20121016085918.GC21441@snakebite.org>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<nad-EE1DCA.01044616102012@news.gmane.org>
	<20121016085918.GC21441@snakebite.org>
Message-ID: <20121016095355.GA14341@p29>

Trent Nelson wrote:
> > build breaking is another matter, of course.  If we are
> > going to mandate a specific version again, that should be documented and
> > checked for.
>
>     My preference: bump to 2.69 and set AC_PREREQ(2.69).  If 2.69 proves
>     unworkable, revert back to 2.68 and AC_PREREQ(2.68).
>
>     I definitely like the idea of explicitly setting the version being
>     used via AC_PREREQ, as this will prevent accidental version churn
>     being checked in by a committer inadvertently.

Bear in mind that AC_PREREQ sets the minimum required version. It
cannot be used to prevent from running newer autoconf in the future.

Petri

From hs at ox.cx  Tue Oct 16 12:23:37 2012
From: hs at ox.cx (Hynek Schlawack)
Date: Tue, 16 Oct 2012 12:23:37 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default):
	Merge issue #15936: Add link from os.urandom to random.SystemRandom
In-Reply-To: <3XgsqD0X10zR5R@mail.python.org>
References: <3XgsqD0X10zR5R@mail.python.org>
Message-ID: <BF549EE4-C7C8-441F-8FDC-9F3A4A10DDD7@ox.cx>

Am 16.10.2012 um 12:15 schrieb andrew.svetlov <python-checkins at python.org>:

> +   For easy to use interface to system randomness please see
> +   :class:`random.SystemRandom`.

Is it just my non-native speaker ears, or should there be an ?an? before ?easy??

From andrew.svetlov at gmail.com  Tue Oct 16 12:28:13 2012
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Tue, 16 Oct 2012 13:28:13 +0300
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default):
 Merge issue #15936: Add link from os.urandom to random.SystemRandom
In-Reply-To: <BF549EE4-C7C8-441F-8FDC-9F3A4A10DDD7@ox.cx>
References: <3XgsqD0X10zR5R@mail.python.org>
	<BF549EE4-C7C8-441F-8FDC-9F3A4A10DDD7@ox.cx>
Message-ID: <CAL3CFcXyPJ5Wvt+EctsFo9Sb15HG_eBXCf1sZLTjsWGUsShSvg@mail.gmail.com>

I dont feel anything wrong with it, but I'm also not native speaker.

On Tue, Oct 16, 2012 at 1:23 PM, Hynek Schlawack <hs at ox.cx> wrote:
> Am 16.10.2012 um 12:15 schrieb andrew.svetlov <python-checkins at python.org>:
>
>> +   For easy to use interface to system randomness please see
>> +   :class:`random.SystemRandom`.
>
> Is it just my non-native speaker ears, or should there be an ?an? before ?easy??
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins



-- 
Thanks,
Andrew Svetlov

From larry at hastings.org  Tue Oct 16 12:43:51 2012
From: larry at hastings.org (Larry Hastings)
Date: Tue, 16 Oct 2012 11:43:51 +0100
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default):
 Merge issue #15936: Add link from os.urandom to random.SystemRandom
In-Reply-To: <BF549EE4-C7C8-441F-8FDC-9F3A4A10DDD7@ox.cx>
References: <3XgsqD0X10zR5R@mail.python.org>
	<BF549EE4-C7C8-441F-8FDC-9F3A4A10DDD7@ox.cx>
Message-ID: <507D3A67.6070701@hastings.org>

On 10/16/2012 11:23 AM, Hynek Schlawack wrote:
> Am 16.10.2012 um 12:15 schrieb andrew.svetlov <python-checkins at python.org>:
>
>> +   For easy to use interface to system randomness please see
>> +   :class:`random.SystemRandom`.
> Is it just my non-native speaker ears, or should there be an ?an? before ?easy??

There should.  And "for system randomness" is hard to follow.  I would 
have written it as

    For an easy-to-use interface to the random number generator provided
    by your platform, please see :class:`random.SystemRandom`.



//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121016/8cf6f711/attachment.html>

From christian at python.org  Tue Oct 16 12:52:25 2012
From: christian at python.org (Christian Heimes)
Date: Tue, 16 Oct 2012 12:52:25 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default):
 Merge issue #15936: Add link from os.urandom to random.SystemRandom
In-Reply-To: <507D3A67.6070701@hastings.org>
References: <3XgsqD0X10zR5R@mail.python.org>
	<BF549EE4-C7C8-441F-8FDC-9F3A4A10DDD7@ox.cx>
	<507D3A67.6070701@hastings.org>
Message-ID: <507D3C69.8050408@python.org>

Am 16.10.2012 12:43, schrieb Larry Hastings:
> On 10/16/2012 11:23 AM, Hynek Schlawack wrote:
>> Am 16.10.2012 um 12:15 schrieb andrew.svetlov <python-checkins at python.org>:
>>
>>> +   For easy to use interface to system randomness please see
>>> +   :class:`random.SystemRandom`.
>> Is it just my non-native speaker ears, or should there be an ?an? before ?easy??
> 
> There should.  And "for system randomness" is hard to follow.  I would
> have written it as
> 
>     For an easy-to-use interface to the random number generator provided
>     by your platform, please see :class:`random.SystemRandom`.

I don't like the phrase "easy-to-use" at all. IMO it can't get any
easier than "give my X random bytes". random.SystemRandom is simply a
high level interface that is mostly compatible with the random module.


From andrew.svetlov at gmail.com  Tue Oct 16 12:56:20 2012
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Tue, 16 Oct 2012 13:56:20 +0300
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default):
 Merge issue #15936: Add link from os.urandom to random.SystemRandom
In-Reply-To: <507D3C69.8050408@python.org>
References: <3XgsqD0X10zR5R@mail.python.org>
	<BF549EE4-C7C8-441F-8FDC-9F3A4A10DDD7@ox.cx>
	<507D3A67.6070701@hastings.org> <507D3C69.8050408@python.org>
Message-ID: <CAL3CFcWD0K=MyWnQB=bcWC9+p5Tk0o_B3GjNbB6o=C0FxR-eKQ@mail.gmail.com>

Well. I rephrased text following suggestion from Larry Hastings.

If there are need more clarification please reopen  #15936

On Tue, Oct 16, 2012 at 1:52 PM, Christian Heimes <christian at python.org> wrote:
> Am 16.10.2012 12:43, schrieb Larry Hastings:
>> On 10/16/2012 11:23 AM, Hynek Schlawack wrote:
>>> Am 16.10.2012 um 12:15 schrieb andrew.svetlov <python-checkins at python.org>:
>>>
>>>> +   For easy to use interface to system randomness please see
>>>> +   :class:`random.SystemRandom`.
>>> Is it just my non-native speaker ears, or should there be an ?an? before ?easy??
>>
>> There should.  And "for system randomness" is hard to follow.  I would
>> have written it as
>>
>>     For an easy-to-use interface to the random number generator provided
>>     by your platform, please see :class:`random.SystemRandom`.
>
> I don't like the phrase "easy-to-use" at all. IMO it can't get any
> easier than "give my X random bytes". random.SystemRandom is simply a
> high level interface that is mostly compatible with the random module.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com



-- 
Thanks,
Andrew Svetlov

From solipsis at pitrou.net  Tue Oct 16 13:14:27 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 16 Oct 2012 13:14:27 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default):
 Merge issue #15936: Add link from os.urandom to random.SystemRandom
References: <3XgsqD0X10zR5R@mail.python.org>
	<BF549EE4-C7C8-441F-8FDC-9F3A4A10DDD7@ox.cx>
	<507D3A67.6070701@hastings.org> <507D3C69.8050408@python.org>
Message-ID: <20121016131427.02cf9007@pitrou.net>

On Tue, 16 Oct 2012 12:52:25 +0200
Christian Heimes <christian at python.org> wrote:
> Am 16.10.2012 12:43, schrieb Larry Hastings:
> > On 10/16/2012 11:23 AM, Hynek Schlawack wrote:
> >> Am 16.10.2012 um 12:15 schrieb andrew.svetlov <python-checkins at python.org>:
> >>
> >>> +   For easy to use interface to system randomness please see
> >>> +   :class:`random.SystemRandom`.
> >> Is it just my non-native speaker ears, or should there be an ?an? before ?easy??
> > 
> > There should.  And "for system randomness" is hard to follow.  I would
> > have written it as
> > 
> >     For an easy-to-use interface to the random number generator provided
> >     by your platform, please see :class:`random.SystemRandom`.
> 
> I don't like the phrase "easy-to-use" at all. IMO it can't get any
> easier than "give my X random bytes". random.SystemRandom is simply a
> high level interface that is mostly compatible with the random module.

I guess it's easy to use if you need something other than raw bytes.
But, yes, I would call it "higher level" rather than "easy to use"
(I don't think there's a need for hyphens, by the way).

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From stephen at xemacs.org  Tue Oct 16 13:59:41 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 16 Oct 2012 20:59:41 +0900
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default):
 Merge issue #15936: Add link from os.urandom to random.SystemRandom
In-Reply-To: <20121016131427.02cf9007@pitrou.net>
References: <3XgsqD0X10zR5R@mail.python.org>
	<BF549EE4-C7C8-441F-8FDC-9F3A4A10DDD7@ox.cx>
	<507D3A67.6070701@hastings.org> <507D3C69.8050408@python.org>
	<20121016131427.02cf9007@pitrou.net>
Message-ID: <87zk3m630y.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:

 > But, yes, I would call it "higher level" rather than "easy to use"
 > (I don't think there's a need for hyphens, by the way).

You don't need them to get the point across, but elderly grammar
pedants will think better of you if you use the hyphens.

Personally, I hope they'll be used.  I'm tired of the assumption that
"illiterate techie" is a redundant phrase.


From solipsis at pitrou.net  Tue Oct 16 14:58:18 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 16 Oct 2012 14:58:18 +0200
Subject: [Python-Dev] cpython (3.3): Tweak the threaded example in
	concurrent.futures
References: <3XgxGd4q4yzR5q@mail.python.org>
Message-ID: <20121016145818.590989e2@pitrou.net>

On Tue, 16 Oct 2012 14:51:21 +0200 (CEST)
nick.coghlan <python-checkins at python.org> wrote:
>  
> +   # We can use a with statement to ensure threads are cleaned up promptly
>     with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
> -       future_to_url = dict((executor.submit(load_url, url, 60), url)
> -                            for url in URLS)
> -
> -       for future in concurrent.futures.as_completed(future_to_url):
> -           url = future_to_url[future]
> -           if future.exception() is not None:
> -               print('%r generated an exception: %s' % (url,
> -                                                        future.exception()))
> +       # Start the load operations and mark each future with its URL
> +       load_urls = [executor.submit(load_url, url, 60) for url in URLS]
> +       for future, url in zip(load_urls, URLS):
> +           future.url = url

Adding an "url" attribute here looks a bit ugly to me. Why not use a
dict comprehension for future_to_url?

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From ncoghlan at gmail.com  Tue Oct 16 15:15:43 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 16 Oct 2012 23:15:43 +1000
Subject: [Python-Dev] cpython (3.3): Tweak the threaded example in
	concurrent.futures
In-Reply-To: <20121016145818.590989e2@pitrou.net>
References: <3XgxGd4q4yzR5q@mail.python.org>
	<20121016145818.590989e2@pitrou.net>
Message-ID: <CADiSq7fTiycn7hzKPfMxTvq+-mSWRid2DvVy0isHa_jw-r8cEA@mail.gmail.com>

On Tue, Oct 16, 2012 at 10:58 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Adding an "url" attribute here looks a bit ugly to me. Why not use a
> dict comprehension for future_to_url?

No reason other than the fact it didn't occur to me to do so. Fixed :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From eliben at gmail.com  Tue Oct 16 15:35:46 2012
From: eliben at gmail.com (Eli Bendersky)
Date: Tue, 16 Oct 2012 06:35:46 -0700
Subject: [Python-Dev] Issues Pending Review
In-Reply-To: <20121014203801.Horde.Kr3CfcL8999QewaJi39AZAA@webmail.df.eu>
References: <CAMfKjGVhDUS9YMyO40joy46xw1WP4SW0ZtkAMAq7SYpP_toYaw@mail.gmail.com>
	<20121014203801.Horde.Kr3CfcL8999QewaJi39AZAA@webmail.df.eu>
Message-ID: <CAF-Rda-gyr0i-JamOeamO6XnK8t0hEQRRZYUYTtCohFdmMa7Ew@mail.gmail.com>

> Hello, I have some issues pending 'patch review' hanging for more than
>> two weeks, could somebody please check them out?
>>
>
> In case nobody picks it up, my 5-for-1 offer still stands: if you review
> five issues, I'll review one of yours.
>

Is that a general offer or just for the OP? :-)
Seriously, getting an issue reviewed by Martin is something worth fighting
for, and reviewing other 5 issues seems like a small price. This is not to
imply that Martin doesn't review others' issues often enough, just that his
reviews come from a high authority and can almost always put a final
resolution on the problem.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121016/cba98bcf/attachment.html>

From barry at python.org  Tue Oct 16 15:47:18 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 16 Oct 2012 09:47:18 -0400
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <20121016093220.GF21441@snakebite.org>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>
	<20121016085934.GA2995@sleipnir.bytereef.org>
	<CAH_1eM3PH1iL7hXDa9kd_7iaJLiqc+BQZLRkt5sHqakZhsNfzA@mail.gmail.com>
	<20121016093220.GF21441@snakebite.org>
Message-ID: <20121016094718.2ff3a84b@limelight.wooz.org>

On Oct 16, 2012, at 05:32 AM, Trent Nelson wrote:

> Anyway, back to the original question: does anyone know of reasons
> we shouldn't bump to 2.69?  Any known incompatibilities?

There will be problems building with 2.69 on Ubuntus older than 12.10,
and Debians older than wheezy.

% rmadison autoconf
  autoconf |     2.61-4 |         hardy | source, all
  autoconf | 2.65-3ubuntu1 |         lucid | source, all
  autoconf | 2.67-2ubuntu1 |         natty | source, all
  autoconf | 2.68-1ubuntu1 |       oneiric | source, all
  autoconf | 2.68-1ubuntu2 |       precise | source, all
  autoconf | 2.69-1ubuntu1 |       quantal | source, all
% rmadison -u debian autoconf
 autoconf | 2.67-2 | squeeze | source, all
 autoconf | 2.69-1 | wheezy  | source, all
 autoconf | 2.69-1 | sid     | source, all

FWIW, precise is Ubuntu 12.04 LTS, so it carries Python 2.7 and 3.2.  I think
it would be fine to update the default branch (i.e. 3.4), but I'm not sure
what benefit you gain from making this change to stable branches, and you
could potentially cause build problems, which you may not find out about for a
while, e.g. when 2.7.4 is released and all the distros go to update.

Cheers,
-Barry

From a.badger at gmail.com  Tue Oct 16 16:28:49 2012
From: a.badger at gmail.com (Toshio Kuratomi)
Date: Tue, 16 Oct 2012 07:28:49 -0700
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <20121016112724.04201f60@pitrou.net>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>
	<20121016090523.GD21441@snakebite.org>
	<20121016112724.04201f60@pitrou.net>
Message-ID: <20121016142849.GF2974@unaka.lan>

On Tue, Oct 16, 2012 at 11:27:24AM +0200, Antoine Pitrou wrote:
> On Tue, 16 Oct 2012 05:05:23 -0400
> Trent Nelson <trent at snakebite.org> wrote:
> > On Tue, Oct 16, 2012 at 01:43:37AM -0700, Charles-Fran?ois Natali wrote:
> > > > My understanding is that we use a specific version of autoconf.
> > > > The reason is that otherwise we end up with useless churn in the repo
> > > > as the generated file changes when different committers use different
> > > > versions.  In the past we have had issues with a new autoconf version
> > > > actually breaking the Python build, so we also need to test a new version
> > > > before switching to it.
> > > 
> > > Well, so I guess all committers will have to use the same
> > > Linux/FreeBSD/whatever distribution then?
> > > AFAICT there's no requirement regarding the mercurial version used by
> > > committers either.
> > 
> >     Autoconf is a special case though.  Different versions of autoconf
> >     produce wildly different outputs for 'configure', making it impossible
> >     to vet configure.ac changes by reviewing the configure diff.
> 
> Isn't it enough to review the configure.ac diff?
> 
That's the ideal but it's been wrong in the past and may possibly be wrong
in the future as well.

Anecdotally, in the Linux distribution I package for we have a conversation
about whether we should apply patches to configure.ac and then run
autoreconf (or equivalent) or include the patches to ocnfigure about once
a year.  Although the latter has been pretty stable for several autoconf
version updates enough people have bad memories of those times when bumping
autoconf revisions that there's always a vocal contingent who advocate
shipping patches to the actual configure scripts (they're under the
impression the package maintainer will actually audit the configure patch
that autoconf generated to see if there's breakage that way).

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121016/1eef72f5/attachment.pgp>

From chris.jerdonek at gmail.com  Tue Oct 16 16:35:25 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Tue, 16 Oct 2012 07:35:25 -0700
Subject: [Python-Dev] [Python-checkins] cpython (3.3): Tweak the
	threaded example in concurrent.futures
In-Reply-To: <3XgxGd4q4yzR5q@mail.python.org>
References: <3XgxGd4q4yzR5q@mail.python.org>
Message-ID: <CAOTb1wdAxcoT5qfpJxk-H5VWZjPYnMNdbdjisCb=Y5szypZUnA@mail.gmail.com>

On Tue, Oct 16, 2012 at 5:51 AM, nick.coghlan
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/a3f27289ec68
> changeset:   79746:a3f27289ec68
> branch:      3.3
> +           except Exception as exc:
> +               print('%r generated an exception: %s' % (url, exc))
>             else:
> -               print('%r page is %d bytes' % (url, len(future.result())))
> +               print('%r page is %d bytes' % (url, len(data)))

I'm not suggesting this particular example be changed, but going
forward should we prefer the newer str.format() approach to
formatting?

http://docs.python.org/dev/library/stdtypes.html#printf-style-string-formatting

I've also seen references in the docs to the "old %-style formatting,"
but maybe old is meant more in the sense of having been around longer
than discouraged.

--Chris

From tseaver at palladion.com  Tue Oct 16 17:18:07 2012
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 16 Oct 2012 11:18:07 -0400
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <20121016094718.2ff3a84b@limelight.wooz.org>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>
	<20121016085934.GA2995@sleipnir.bytereef.org>
	<CAH_1eM3PH1iL7hXDa9kd_7iaJLiqc+BQZLRkt5sHqakZhsNfzA@mail.gmail.com>
	<20121016093220.GF21441@snakebite.org>
	<20121016094718.2ff3a84b@limelight.wooz.org>
Message-ID: <k5jtrr$guu$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/16/2012 09:47 AM, Barry Warsaw wrote:
> On Oct 16, 2012, at 05:32 AM, Trent Nelson wrote:
> 
>> Anyway, back to the original question: does anyone know of reasons 
>> we shouldn't bump to 2.69?  Any known incompatibilities?
> 
> There will be problems building with 2.69 on Ubuntus older than
> 12.10, and Debians older than wheezy.
> 
> % rmadison autoconf autoconf |     2.61-4 |         hardy | source,
> all autoconf | 2.65-3ubuntu1 |         lucid | source, all autoconf |
> 2.67-2ubuntu1 |         natty | source, all autoconf | 2.68-1ubuntu1 |
> oneiric | source, all autoconf | 2.68-1ubuntu2 |       precise |
> source, all autoconf | 2.69-1ubuntu1 |       quantal | source, all %
> rmadison -u debian autoconf autoconf | 2.67-2 | squeeze | source, all 
> autoconf | 2.69-1 | wheezy  | source, all autoconf | 2.69-1 | sid
> | source, all
> 
> FWIW, precise is Ubuntu 12.04 LTS, so it carries Python 2.7 and 3.2.
> I think it would be fine to update the default branch (i.e. 3.4), but
> I'm not sure what benefit you gain from making this change to stable
> branches, and you could potentially cause build problems, which you
> may not find out about for a while, e.g. when 2.7.4 is released and
> all the distros go to update.

Agreed:  this is really the same issue as bumping the VisualStudio
version (or any other build tooling) inside a release line:  too much
potential for breakage for little gain.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlB9eq8ACgkQ+gerLs4ltQ5gWQCcDDjMRWcUDagb6dhZCK+5pSVs
dVIAoLH+S205plmi22RS0aBaF5GxnTmn
=PrEy
-----END PGP SIGNATURE-----


From brett at python.org  Tue Oct 16 17:23:00 2012
From: brett at python.org (Brett Cannon)
Date: Tue, 16 Oct 2012 11:23:00 -0400
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <k5jtrr$guu$1@ger.gmane.org>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>
	<20121016085934.GA2995@sleipnir.bytereef.org>
	<CAH_1eM3PH1iL7hXDa9kd_7iaJLiqc+BQZLRkt5sHqakZhsNfzA@mail.gmail.com>
	<20121016093220.GF21441@snakebite.org>
	<20121016094718.2ff3a84b@limelight.wooz.org>
	<k5jtrr$guu$1@ger.gmane.org>
Message-ID: <CAP1=2W5Wkzs0OF2KwUJP8nL1gj8dHJo6Y-3T=hvcBLPD=aOyeQ@mail.gmail.com>

On Tue, Oct 16, 2012 at 11:18 AM, Tres Seaver <tseaver at palladion.com> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 10/16/2012 09:47 AM, Barry Warsaw wrote:
> > On Oct 16, 2012, at 05:32 AM, Trent Nelson wrote:
> >
> >> Anyway, back to the original question: does anyone know of reasons
> >> we shouldn't bump to 2.69?  Any known incompatibilities?
> >
> > There will be problems building with 2.69 on Ubuntus older than
> > 12.10, and Debians older than wheezy.
> >
> > % rmadison autoconf autoconf |     2.61-4 |         hardy | source,
> > all autoconf | 2.65-3ubuntu1 |         lucid | source, all autoconf |
> > 2.67-2ubuntu1 |         natty | source, all autoconf | 2.68-1ubuntu1 |
> > oneiric | source, all autoconf | 2.68-1ubuntu2 |       precise |
> > source, all autoconf | 2.69-1ubuntu1 |       quantal | source, all %
> > rmadison -u debian autoconf autoconf | 2.67-2 | squeeze | source, all
> > autoconf | 2.69-1 | wheezy  | source, all autoconf | 2.69-1 | sid
> > | source, all
> >
> > FWIW, precise is Ubuntu 12.04 LTS, so it carries Python 2.7 and 3.2.
> > I think it would be fine to update the default branch (i.e. 3.4), but
> > I'm not sure what benefit you gain from making this change to stable
> > branches, and you could potentially cause build problems, which you
> > may not find out about for a while, e.g. when 2.7.4 is released and
> > all the distros go to update.
>
> Agreed:  this is really the same issue as bumping the VisualStudio
> version (or any other build tooling) inside a release line:  too much
> potential for breakage for little gain.
>

I think Barry's suggestion of updating default and leaving stable versions
alone is a good one.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121016/4ed3cd2d/attachment.html>

From trent at snakebite.org  Tue Oct 16 17:55:01 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 16 Oct 2012 11:55:01 -0400
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <CAP1=2W5Wkzs0OF2KwUJP8nL1gj8dHJo6Y-3T=hvcBLPD=aOyeQ@mail.gmail.com>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>
	<20121016085934.GA2995@sleipnir.bytereef.org>
	<CAH_1eM3PH1iL7hXDa9kd_7iaJLiqc+BQZLRkt5sHqakZhsNfzA@mail.gmail.com>
	<20121016093220.GF21441@snakebite.org>
	<20121016094718.2ff3a84b@limelight.wooz.org>
	<k5jtrr$guu$1@ger.gmane.org>
	<CAP1=2W5Wkzs0OF2KwUJP8nL1gj8dHJo6Y-3T=hvcBLPD=aOyeQ@mail.gmail.com>
Message-ID: <20121016155500.GA32904@snakebite.org>

On Tue, Oct 16, 2012 at 08:23:00AM -0700, Brett Cannon wrote:
>    On Tue, Oct 16, 2012 at 11:18 AM, Tres Seaver <tseaver at palladion.com>
>    wrote:
> 
>      -----BEGIN PGP SIGNED MESSAGE-----
>      Hash: SHA1
>      On 10/16/2012 09:47 AM, Barry Warsaw wrote:
>      > On Oct 16, 2012, at 05:32 AM, Trent Nelson wrote:
>      >
>      >> Anyway, back to the original question: does anyone know of reasons
>      >> we shouldn't bump to 2.69? A Any known incompatibilities?
>      >
>      > There will be problems building with 2.69 on Ubuntus older than
>      > 12.10, and Debians older than wheezy.
>      >
>      > % rmadison autoconf autoconf | A  A  2.61-4 | A  A  A  A  hardy |
>      source,
>      > all autoconf | 2.65-3ubuntu1 | A  A  A  A  lucid | source, all
>      autoconf |
>      > 2.67-2ubuntu1 | A  A  A  A  natty | source, all autoconf |
>      2.68-1ubuntu1 |
>      > oneiric | source, all autoconf | 2.68-1ubuntu2 | A  A  A  precise |
>      > source, all autoconf | 2.69-1ubuntu1 | A  A  A  quantal | source, all
>      %
>      > rmadison -u debian autoconf autoconf | 2.67-2 | squeeze | source, all
>      > autoconf | 2.69-1 | wheezy A | source, all autoconf | 2.69-1 | sid
>      > | source, all
>      >
>      > FWIW, precise is Ubuntu 12.04 LTS, so it carries Python 2.7 and 3.2.
>      > I think it would be fine to update the default branch (i.e. 3.4), but
>      > I'm not sure what benefit you gain from making this change to stable
>      > branches, and you could potentially cause build problems, which you
>      > may not find out about for a while, e.g. when 2.7.4 is released and
>      > all the distros go to update.
> 
>      Agreed: A this is really the same issue as bumping the VisualStudio
>      version (or any other build tooling) inside a release line: A too much
>      potential for breakage for little gain.
> 
>    I think Barry's suggestion of updating default and leaving stable versions
>    alone is a good one.A 

    Bumped 3.4 to 2.69 in 79763:74b95194ba86!

        Trent.

From dmalcolm at redhat.com  Tue Oct 16 17:58:33 2012
From: dmalcolm at redhat.com (David Malcolm)
Date: Tue, 16 Oct 2012 11:58:33 -0400
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <20121016085934.GA2995@sleipnir.bytereef.org>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>
	<20121016085934.GA2995@sleipnir.bytereef.org>
Message-ID: <1350403113.28646.11.camel@surprise>

On Tue, 2012-10-16 at 10:59 +0200, Stefan Krah wrote:
> Charles-Fran?ois Natali <neologix at free.fr> wrote:
> > Well, so I guess all committers will have to use the same
> > Linux/FreeBSD/whatever distribution then?
> > AFAICT there's no requirement regarding the mercurial version used by
> > committers either.
> 
> It should be sufficient to install autoconf-x.y into /home/user/bin or
> something similar. Installing autoconf from source really takes about
> 3 minutes.
FWIW this was also documented in the devguide, as of:
http://hg.python.org/devguide/rev/5432be4d4e1a

though that description was then removed in:
http://hg.python.org/devguide/rev/27be97280cff




From python at mrabarnett.plus.com  Tue Oct 16 18:38:48 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 16 Oct 2012 17:38:48 +0100
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default):
 Merge issue #15936: Add link from os.urandom to random.SystemRandom
In-Reply-To: <87zk3m630y.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <3XgsqD0X10zR5R@mail.python.org>
	<BF549EE4-C7C8-441F-8FDC-9F3A4A10DDD7@ox.cx>
	<507D3A67.6070701@hastings.org> <507D3C69.8050408@python.org>
	<20121016131427.02cf9007@pitrou.net>
	<87zk3m630y.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <507D8D98.6080000@mrabarnett.plus.com>

On 2012-10-16 12:59, Stephen J. Turnbull wrote:
> Antoine Pitrou writes:
>
>   > But, yes, I would call it "higher level" rather than "easy to use"
>   > (I don't think there's a need for hyphens, by the way).
>
> You don't need them to get the point across, but elderly grammar
> pedants will think better of you if you use the hyphens.
>
> Personally, I hope they'll be used.
+1

 >  I'm tired of the assumption that "illiterate techie" is a redundant 
phrase.
>


From bugtrack at roumenpetrov.info  Tue Oct 16 20:50:27 2012
From: bugtrack at roumenpetrov.info (Roumen Petrov)
Date: Tue, 16 Oct 2012 21:50:27 +0300
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <20121016092026.GE21441@snakebite.org>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<20121016092026.GE21441@snakebite.org>
Message-ID: <507DAC73.8060703@roumenpetrov.info>

Trent Nelson wrote:
[SNIP]

> diff -r 51ce9830d85a configure.ac
> --- a/configure.ac      Sat Oct 13 11:58:23 2012 -0400
> +++ b/configure.ac      Tue Oct 16 09:12:56 2012 +0000
> @@ -9,6 +9,9 @@
>
>   AC_INIT(python, PYTHON_VERSION, http://bugs.python.org/)
>
> +BUILDDIR="`pwd`"
        ^^^^^

http://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/Preset-Output-Variables.html


[SNIP]
>      However, the 2.69 version of autoconf generates this monster of a
>      configure diff:

It is python project rule to keep generated files into repository.



Roumen


From larry at hastings.org  Wed Oct 17 17:12:50 2012
From: larry at hastings.org (Larry Hastings)
Date: Wed, 17 Oct 2012 16:12:50 +0100
Subject: [Python-Dev] PEP 429, Release Schedule for Python 3.4, posted
Message-ID: <507ECAF2.8020400@hastings.org>



It's essentially identical to the draft I posted last week--nothing new 
there.  Looking forward to February 2014!

And, to quote Barry Warsaw, let the wild rumpus begin!,


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121017/63f1f820/attachment.html>

From larry at hastings.org  Wed Oct 17 17:13:46 2012
From: larry at hastings.org (Larry Hastings)
Date: Wed, 17 Oct 2012 16:13:46 +0100
Subject: [Python-Dev] PEP 429, Release Schedule for Python 3.4, posted
In-Reply-To: <507ECAF2.8020400@hastings.org>
References: <507ECAF2.8020400@hastings.org>
Message-ID: <507ECB2A.7070107@hastings.org>


Should have posted a link, sorry.  Here you go:

    http://www.python.org/dev/peps/pep-0429/


//arry/

On 10/17/2012 04:12 PM, Larry Hastings wrote:
>
>
> It's essentially identical to the draft I posted last week--nothing 
> new there.  Looking forward to February 2014!
>
> And, to quote Barry Warsaw, let the wild rumpus begin!,
>
>
> //arry/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121017/65a47d76/attachment.html>

From storchaka at gmail.com  Wed Oct 17 18:51:31 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Wed, 17 Oct 2012 19:51:31 +0300
Subject: [Python-Dev] cpython (3.3): Tweak the threaded example in
	concurrent.futures
In-Reply-To: <CADiSq7fTiycn7hzKPfMxTvq+-mSWRid2DvVy0isHa_jw-r8cEA@mail.gmail.com>
References: <3XgxGd4q4yzR5q@mail.python.org>
	<20121016145818.590989e2@pitrou.net>
	<CADiSq7fTiycn7hzKPfMxTvq+-mSWRid2DvVy0isHa_jw-r8cEA@mail.gmail.com>
Message-ID: <k5mnmq$aff$1@ger.gmane.org>

 > +           url = future_to_url[url]

             url = future_to_url[future]



From storchaka at gmail.com  Wed Oct 17 18:55:15 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Wed, 17 Oct 2012 19:55:15 +0300
Subject: [Python-Dev] cpython (3.3): Tweak the threaded example in
	concurrent.futures
In-Reply-To: <20121016145818.590989e2@pitrou.net>
References: <3XgxGd4q4yzR5q@mail.python.org>
	<20121016145818.590989e2@pitrou.net>
Message-ID: <k5mntq$cig$1@ger.gmane.org>

On 16.10.12 15:58, Antoine Pitrou wrote:
> Adding an "url" attribute here looks a bit ugly to me. Why not use a
> dict comprehension for future_to_url?

This is especially ugly after implementing PEP 412 (Key-Sharing Dictionary).



From andrew.svetlov at gmail.com  Wed Oct 17 20:34:02 2012
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Wed, 17 Oct 2012 21:34:02 +0300
Subject: [Python-Dev] cpython (3.3): Tweak the threaded example in
	concurrent.futures
In-Reply-To: <k5mntq$cig$1@ger.gmane.org>
References: <3XgxGd4q4yzR5q@mail.python.org>
	<20121016145818.590989e2@pitrou.net> <k5mntq$cig$1@ger.gmane.org>
Message-ID: <CAL3CFcWjxU8yrroNss-Yf6mYqe67Jz0vUUtDefS8NpqAf_4G7A@mail.gmail.com>

Good note!

On Wed, Oct 17, 2012 at 7:55 PM, Serhiy Storchaka <storchaka at gmail.com> wrote:
> On 16.10.12 15:58, Antoine Pitrou wrote:
>>
>> Adding an "url" attribute here looks a bit ugly to me. Why not use a
>> dict comprehension for future_to_url?
>
>
> This is especially ugly after implementing PEP 412 (Key-Sharing Dictionary).
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com



-- 
Thanks,
Andrew Svetlov

From doko at ubuntu.com  Thu Oct 18 00:07:16 2012
From: doko at ubuntu.com (Matthias Klose)
Date: Thu, 18 Oct 2012 00:07:16 +0200
Subject: [Python-Dev] Bumping autoconf from 2.68 to 2.69
In-Reply-To: <1350403113.28646.11.camel@surprise>
References: <20121016043352.GA21441@snakebite.org>
	<nad-5C62E0.23180415102012@news.gmane.org>
	<20121016071236.0792D2500FA@webabinitio.net>
	<CAH_1eM288ysijFnmPh3ra=ioWvChDbrh5p9TN+myf=UxJeXFaQ@mail.gmail.com>
	<20121016085934.GA2995@sleipnir.bytereef.org>
	<1350403113.28646.11.camel@surprise>
Message-ID: <507F2C14.3050206@ubuntu.com>

On 16.10.2012 17:58, David Malcolm wrote:
> On Tue, 2012-10-16 at 10:59 +0200, Stefan Krah wrote:
>> Charles-Fran?ois Natali <neologix at free.fr> wrote:
>>> Well, so I guess all committers will have to use the same
>>> Linux/FreeBSD/whatever distribution then?
>>> AFAICT there's no requirement regarding the mercurial version used by
>>> committers either.
>>
>> It should be sufficient to install autoconf-x.y into /home/user/bin or
>> something similar. Installing autoconf from source really takes about
>> 3 minutes.
> FWIW this was also documented in the devguide, as of:
> http://hg.python.org/devguide/rev/5432be4d4e1a
> 
> though that description was then removed in:
> http://hg.python.org/devguide/rev/27be97280cf

so maybe this could be restored.

the python configure.ac doesn't use that internal and specific autoconf
features, so it might be ok to live with the diffs.  I didn't see any issues
using the system supplied autoconf with any python version on not that recent
distro releases.

However the effort to use a specific autoconf version for each branch should be
minimal (other projects do require this).

  Matthias


From dholth at gmail.com  Thu Oct 18 15:16:25 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 18 Oct 2012 09:16:25 -0400
Subject: [Python-Dev] Improved evaluator added to ast module
In-Reply-To: <loom.20121011T192849-311@post.gmane.org>
References: <loom.20121011T175200-792@post.gmane.org>
	<CAPZV6o9H4NHT3v_n2uTLaf-aaes1LkAN161++j9MiyhmH9Z7EQ@mail.gmail.com>
	<CAG8k2+7H=hAeKpZ_BmN7hRx0qQjDQ=AQEuest2viRRteA3ua=g@mail.gmail.com>
	<loom.20121011T192849-311@post.gmane.org>
Message-ID: <CAG8k2+66ND=Ucmt=9xfiYDUhQjAFSsXQC-xvE4VAb9L9h+RPgA@mail.gmail.com>

On Thu, Oct 11, 2012 at 1:36 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Daniel Holth <dholth <at> gmail.com> writes:
>
>> How does this compare to the markerlib approach? In markerlib you just
>> make sure all the AST nodes are in a set of allowed nodes, currently
>> (Compare, BoolOp, Attribute, Name, Load, Str, cmpop, boolop), and then
>> use the normal eval(). Is one way more secure / fast / flexible than
>> the other?
>
> I don't think performance is an issue, and the markerlib approach seems just
> as reasonable as the one I've taken, except that it calls eval(), whereas my
> approach doesn't. It boils down to what should be allowed in expressions, and
> what shouldn't be.
>
> ISTM there is a space for a limited evaluator that's less limiting than
> literal_eval(). I do realise that this type of sandboxing is not easy to achieve,
> and I'm not aiming to advance the state of the art here - I just want to close
> the issue in the best way I can.

I bet the literal_eval approach simply predates compile(ast) which is
a Python 2.6 feature. It is also probably slightly faster on CPython
to avoid compile(ast) if you are only evaluating the code once.

From g.brandl at gmx.net  Thu Oct 18 17:41:58 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 18 Oct 2012 17:41:58 +0200
Subject: [Python-Dev] Improved evaluator added to ast module
In-Reply-To: <CAG8k2+66ND=Ucmt=9xfiYDUhQjAFSsXQC-xvE4VAb9L9h+RPgA@mail.gmail.com>
References: <loom.20121011T175200-792@post.gmane.org>
	<CAPZV6o9H4NHT3v_n2uTLaf-aaes1LkAN161++j9MiyhmH9Z7EQ@mail.gmail.com>
	<CAG8k2+7H=hAeKpZ_BmN7hRx0qQjDQ=AQEuest2viRRteA3ua=g@mail.gmail.com>
	<loom.20121011T192849-311@post.gmane.org>
	<CAG8k2+66ND=Ucmt=9xfiYDUhQjAFSsXQC-xvE4VAb9L9h+RPgA@mail.gmail.com>
Message-ID: <k5p801$3bf$1@ger.gmane.org>

On 10/18/2012 03:16 PM, Daniel Holth wrote:
> On Thu, Oct 11, 2012 at 1:36 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> Daniel Holth <dholth <at> gmail.com> writes:
>>
>>> How does this compare to the markerlib approach? In markerlib you just
>>> make sure all the AST nodes are in a set of allowed nodes, currently
>>> (Compare, BoolOp, Attribute, Name, Load, Str, cmpop, boolop), and then
>>> use the normal eval(). Is one way more secure / fast / flexible than
>>> the other?
>>
>> I don't think performance is an issue, and the markerlib approach seems just
>> as reasonable as the one I've taken, except that it calls eval(), whereas my
>> approach doesn't. It boils down to what should be allowed in expressions, and
>> what shouldn't be.
>>
>> ISTM there is a space for a limited evaluator that's less limiting than
>> literal_eval(). I do realise that this type of sandboxing is not easy to achieve,
>> and I'm not aiming to advance the state of the art here - I just want to close
>> the issue in the best way I can.
> 
> I bet the literal_eval approach simply predates compile(ast) which is
> a Python 2.6 feature.

Nope. All of ast (in contrast to _ast) is new in 2.6.

> It is also probably slightly faster on CPython
> to avoid compile(ast) if you are only evaluating the code once.

Yes; if you inspect the nodes anyway you can just as well evaluate them
on the way.

Georg


From dholth at gmail.com  Thu Oct 18 19:29:34 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 18 Oct 2012 13:29:34 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
Message-ID: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>

I'd like to submit the Wheel PEPs 425 (filename metadata), 426
(Metadata 1.3), and 427 (wheel itself) for acceptance. The format has
been stable since May and we are preparing a patch to support it in
pip, but we need to earn consensus before including it in the most
widely used installer.

Wheel is a binary packaging format that is mostly based on the
phenomenal work done by 'packaging' as PEP 376 et al. The resulting
packages solve packaging problems by being installable without
setup.py or any variation of distutils, lxml can be installed in 0.7
seconds, and a simple installer is just "unzip" inside site-packages.
Wheel installers know about the major version number of the spec
itself, and will refuse to install version 2.0 wheels if they do not
understand them, so the format can be changed down the line.

Let me know what I need to do to get it accepted, if anything needs to
be added or revised, or, preferably, that it is awesome and you want
to use it ASAP.

Thanks,

Daniel Holth

From martin at v.loewis.de  Thu Oct 18 20:21:14 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Thu, 18 Oct 2012 20:21:14 +0200
Subject: [Python-Dev] PEP 427 comment: code signing
Message-ID: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>

I'm -1 on the usage of ed25519 in PEP 427. While the PEP proposes to use JSON
Web signatures, this algorithm is not supported by the current JWS draft [1].

Instead, I suggest to use the ES256 algorithm from JWS, i.e. ECDSA with the
NIST P-256 curve and SHA-256. This has the advantage of using standard
algorithms [2].

I don't know what the rationale for suggesting ed25519 is; I suppose that
existence of a pure-Python implementation played a role. However:
- ECDSA also has a pure-Python implementation
- ECDSA is well-supported by OpenSSL, i.e. a signature generator may also
   invoke the OpenSSL command line for efficient implementation. I believe
   M2Crypto also exposes enough of OpenSSL tp perform ECDSA signing and
   verification.

I'm -0 on the use of JWS; I would prefer a signature format that is already
an established internet standard (such a PGP or S/MIME). However, it does look
that this may become a proper internet standard in the near future, so it's
an ok choice.

If it really must be ed25519, I request that this is registered with IANA
once the PEP is accepted, the RFC is accepted, and the JWS algorithm
registry is open.

Regards,
Martin

[1] http://tools.ietf.org/html/draft-ietf-jose-json-web-signature-06
[2] http://tools.ietf.org/html/draft-ietf-jose-json-web-algorithms-06


From benjamin at python.org  Thu Oct 18 20:35:19 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 18 Oct 2012 14:35:19 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
Message-ID: <CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>

2012/10/18 Daniel Holth <dholth at gmail.com>:
> Let me know what I need to do to get it accepted, if anything needs to
> be added or revised, or, preferably, that it is awesome and you want
> to use it ASAP.

Traditionally, you send the peps to python-dev, so people can bikeshed inline.

-- 
Regards,
Benjamin

From martin at v.loewis.de  Thu Oct 18 20:36:27 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Thu, 18 Oct 2012 20:36:27 +0200
Subject: [Python-Dev] PEP 425 comment: package names
Message-ID: <20121018203627.Horde.hL9XC0lCcOxQgEwr1qV276A@webmail.df.eu>

ISTM that some important information and some elaboration is missing  
from PEP 425.

The PEP is currently silent on how exactly these "compatibility tags"  
are combined
with the package name, and the file extension. This should be  
specified, and preferably
some examples be given.

Regards,
Martin




From martin at v.loewis.de  Thu Oct 18 20:45:44 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Thu, 18 Oct 2012 20:45:44 +0200
Subject: [Python-Dev] PEP 426 comment: field order
Message-ID: <20121018204544.Horde.oJXcHklCcOxQgE5YhVcnAsA@webmail.df.eu>

I'd like to request that PEP 426 is extended to talk about the order  
of fields.
In particular, for the Extension field, is it necessary that all  
"additional tags"
follow immediately the respective Extension field?

I also request that RFC 2119 terminology is followed strictly. In particular,
the phrasing "Additional tags defined by the extension should be of  
the form string/Name:"
is unclear - under what "particular circumstances" can I deviate from that
requirement, i.e. use some form other than string/Name?

Regards,
Martin



From dholth at gmail.com  Thu Oct 18 21:07:10 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 18 Oct 2012 15:07:10 -0400
Subject: [Python-Dev] PEP 426 comment: field order
In-Reply-To: <20121018204544.Horde.oJXcHklCcOxQgE5YhVcnAsA@webmail.df.eu>
References: <20121018204544.Horde.oJXcHklCcOxQgE5YhVcnAsA@webmail.df.eu>
Message-ID: <CAG8k2+6Nyac+PjUkiW8GngtAVG0i8_9c3BjX5YoZE=V8P1r-Hw@mail.gmail.com>

Will add that the order is not significant. It is essentially a multidict.

On Thu, Oct 18, 2012 at 2:45 PM,  <martin at v.loewis.de> wrote:
> I'd like to request that PEP 426 is extended to talk about the order of
> fields.
> In particular, for the Extension field, is it necessary that all "additional
> tags"
> follow immediately the respective Extension field?
>
> I also request that RFC 2119 terminology is followed strictly. In
> particular,
> the phrasing "Additional tags defined by the extension should be of the form
> string/Name:"
> is unclear - under what "particular circumstances" can I deviate from that
> requirement, i.e. use some form other than string/Name?
>
> Regards,
> Martin
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/dholth%40gmail.com

From solipsis at pitrou.net  Thu Oct 18 21:10:14 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 18 Oct 2012 21:10:14 +0200
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
Message-ID: <20121018211014.73d56057@pitrou.net>

On Thu, 18 Oct 2012 14:35:19 -0400
Benjamin Peterson <benjamin at python.org> wrote:
> 2012/10/18 Daniel Holth <dholth at gmail.com>:
> > Let me know what I need to do to get it accepted, if anything needs to
> > be added or revised, or, preferably, that it is awesome and you want
> > to use it ASAP.
> 
> Traditionally, you send the peps to python-dev, so people can bikeshed inline.

Or at least send the URLs, it's helpful.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From dholth at gmail.com  Thu Oct 18 21:15:59 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 18 Oct 2012 15:15:59 -0400
Subject: [Python-Dev] PEP 425 comment: package names
In-Reply-To: <20121018203627.Horde.hL9XC0lCcOxQgEwr1qV276A@webmail.df.eu>
References: <20121018203627.Horde.hL9XC0lCcOxQgEwr1qV276A@webmail.df.eu>
Message-ID: <CAG8k2+5UaPUa3zDX0CMC9uWVbTZ+VT+csa62gkd4j7qsG9Nv7w@mail.gmail.com>

On Thu, Oct 18, 2012 at 2:36 PM,  <martin at v.loewis.de> wrote:
> ISTM that some important information and some elaboration is missing from
> PEP 425.
>
> The PEP is currently silent on how exactly these "compatibility tags" are
> combined
> with the package name, and the file extension. This should be specified, and
> preferably
> some examples be given.

Wheel specifies how it uses the tags. You have to strip the known
extension from the filename. I can include its example "this is how a
particular file format uses the tags" in the pep. It is
name-version-tag-tag-tag.ext with all - folded to _

From martin at v.loewis.de  Thu Oct 18 21:23:31 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Thu, 18 Oct 2012 21:23:31 +0200
Subject: [Python-Dev] PEP 425 comment: package names
In-Reply-To: <CAG8k2+5UaPUa3zDX0CMC9uWVbTZ+VT+csa62gkd4j7qsG9Nv7w@mail.gmail.com>
References: <20121018203627.Horde.hL9XC0lCcOxQgEwr1qV276A@webmail.df.eu>
	<CAG8k2+5UaPUa3zDX0CMC9uWVbTZ+VT+csa62gkd4j7qsG9Nv7w@mail.gmail.com>
Message-ID: <20121018212331.Horde.XAKqaUlCcOxQgFczJ8AHXlA@webmail.df.eu>


Zitat von Daniel Holth <dholth at gmail.com>:

> On Thu, Oct 18, 2012 at 2:36 PM,  <martin at v.loewis.de> wrote:
>> ISTM that some important information and some elaboration is missing from
>> PEP 425.
>>
>> The PEP is currently silent on how exactly these "compatibility tags" are
>> combined
>> with the package name, and the file extension. This should be specified, and
>> preferably
>> some examples be given.
>
> Wheel specifies how it uses the tags. You have to strip the known
> extension from the filename.

Hmm. The word "extension" doesn't even appear in connection with
file names in the PEP (only in relation to "C extensions").

Does the PEP, or does it not, specify that a dash must be used between
the package-name-and-version? Neither the words "hyphen" nor "dash"
appear in the PEP, except that hyphens are used inside the tag, and that
hyphens and dots in get_platform() results are replaced.

Regards,
Martin



From dholth at gmail.com  Thu Oct 18 21:37:15 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 18 Oct 2012 15:37:15 -0400
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
Message-ID: <CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>

On Thu, Oct 18, 2012 at 2:21 PM,  <martin at v.loewis.de> wrote:
> I'm -1 on the usage of ed25519 in PEP 427. While the PEP proposes to use
> JSON
> Web signatures, this algorithm is not supported by the current JWS draft
> [1].
>
> Instead, I suggest to use the ES256 algorithm from JWS, i.e. ECDSA with the
> NIST P-256 curve and SHA-256. This has the advantage of using standard
> algorithms [2].
>
> I don't know what the rationale for suggesting ed25519 is; I suppose that
> existence of a pure-Python implementation played a role. However:
> - ECDSA also has a pure-Python implementation
> - ECDSA is well-supported by OpenSSL, i.e. a signature generator may also
>   invoke the OpenSSL command line for efficient implementation. I believe
>   M2Crypto also exposes enough of OpenSSL tp perform ECDSA signing and
>   verification.
>
> I'm -0 on the use of JWS; I would prefer a signature format that is already
> an established internet standard (such a PGP or S/MIME). However, it does
> look
> that this may become a proper internet standard in the near future, so it's
> an ok choice.
>
> If it really must be ed25519, I request that this is registered with IANA
> once the PEP is accepted, the RFC is accepted, and the JWS algorithm
> registry is open.

I expected ed25519 to be somewhat controversial. I will register it
with IANA when possible. I chose it because ed25519 is fast enough
that you need never consider dis-using it for performance reasons. The
wheel reference implementation includes a reasonably performant
~250-line pure-Python version*. Unlike ECDSA, signature generation
does not consume entropy; this feature of ECDSA broke the Playstation
3's code signing system.

JWS is likewise tiny to implement, so the wheel reference installer
always can and does verify the internal consistency of every signed
wheel.

S/MIME signatures are allowed as a courtesy to a government contractor
friend but are not implemented.

* https://bitbucket.org/dholth/wheel/src/tip/wheel/signatures
* http://ed25519.cr.yp.to

From dholth at gmail.com  Thu Oct 18 21:38:30 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 18 Oct 2012 15:38:30 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <20121018211014.73d56057@pitrou.net>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
Message-ID: <CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>

On Thu, Oct 18, 2012 at 3:10 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Thu, 18 Oct 2012 14:35:19 -0400
> Benjamin Peterson <benjamin at python.org> wrote:
>> 2012/10/18 Daniel Holth <dholth at gmail.com>:
>> > Let me know what I need to do to get it accepted, if anything needs to
>> > be added or revised, or, preferably, that it is awesome and you want
>> > to use it ASAP.
>>
>> Traditionally, you send the peps to python-dev, so people can bikeshed inline.
>
> Or at least send the URLs, it's helpful.

The texts seemed a bit long for direct inclusion.

http://www.python.org/dev/peps/pep-0425/
http://www.python.org/dev/peps/pep-0426/
http://www.python.org/dev/peps/pep-0427/

From dholth at gmail.com  Thu Oct 18 21:41:02 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 18 Oct 2012 15:41:02 -0400
Subject: [Python-Dev] PEP 425 comment: package names
In-Reply-To: <20121018212331.Horde.XAKqaUlCcOxQgFczJ8AHXlA@webmail.df.eu>
References: <20121018203627.Horde.hL9XC0lCcOxQgEwr1qV276A@webmail.df.eu>
	<CAG8k2+5UaPUa3zDX0CMC9uWVbTZ+VT+csa62gkd4j7qsG9Nv7w@mail.gmail.com>
	<20121018212331.Horde.XAKqaUlCcOxQgFczJ8AHXlA@webmail.df.eu>
Message-ID: <CAG8k2+46Bj0yeiwkhdOzA2QwztE8vM9zdLz=XVuuh-yWrjSVMg@mail.gmail.com>

On Thu, Oct 18, 2012 at 3:23 PM,  <martin at v.loewis.de> wrote:
>
> Zitat von Daniel Holth <dholth at gmail.com>:
>
>
>> On Thu, Oct 18, 2012 at 2:36 PM,  <martin at v.loewis.de> wrote:
>>>
>>> ISTM that some important information and some elaboration is missing from
>>> PEP 425.
>>>
>>> The PEP is currently silent on how exactly these "compatibility tags" are
>>> combined
>>> with the package name, and the file extension. This should be specified,
>>> and
>>> preferably
>>> some examples be given.
>>
>>
>> Wheel specifies how it uses the tags. You have to strip the known
>> extension from the filename.
>
>
> Hmm. The word "extension" doesn't even appear in connection with
> file names in the PEP (only in relation to "C extensions").
>
> Does the PEP, or does it not, specify that a dash must be used between
> the package-name-and-version? Neither the words "hyphen" nor "dash"
> appear in the PEP, except that hyphens are used inside the tag, and that
> hyphens and dots in get_platform() results are replaced.

The pep was only meant to specify the tag format which is a-b-c or
a.z-b-c , and some typical values of those tags. I agree that it is
confusing not to include an example of a tag being used in the context
of an actual file name.

From benjamin at python.org  Thu Oct 18 21:44:45 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 18 Oct 2012 15:44:45 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
Message-ID: <CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>

2012/10/18 Daniel Holth <dholth at gmail.com>:
> On Thu, Oct 18, 2012 at 3:10 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> On Thu, 18 Oct 2012 14:35:19 -0400
>> Benjamin Peterson <benjamin at python.org> wrote:
>>> 2012/10/18 Daniel Holth <dholth at gmail.com>:
>>> > Let me know what I need to do to get it accepted, if anything needs to
>>> > be added or revised, or, preferably, that it is awesome and you want
>>> > to use it ASAP.
>>>
>>> Traditionally, you send the peps to python-dev, so people can bikeshed inline.
>>
>> Or at least send the URLs, it's helpful.
>
> The texts seemed a bit long for direct inclusion.

Those are pretty short as PEPs go. :)


-- 
Regards,
Benjamin

From dholth at gmail.com  Thu Oct 18 21:48:37 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 18 Oct 2012 15:48:37 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
Message-ID: <CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>

PEP: 425
Title: Compatibility Tags for Built Distributions
Version: $Revision$
Last-Modified: 07-Aug-2012
Author: Daniel Holth <dholth at fastmail.fm>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 27-Jul-2012
Python-Version: 3.4
Post-History: 8-Aug-2012


Abstract
========

This PEP specifies a tagging system to indicate with which versions of
Python a built or binary distribution is compatible.  A set of three
tags indicate which Python implementation and language version, ABI,
and platform a built distribution requires.  The tags are terse because
they will be included in filenames.


PEP Editor's Note
=================

While the naming scheme described in this PEP will not be supported directly
in the standard library until Python 3.4 at the earliest, draft
implementations may be made available in third party projects.


Rationale
=========

Today "python setup.py bdist" generates the same filename on PyPy
and CPython, but an incompatible archive, making it inconvenient to
share built distributions in the same folder or index.  Instead, built
distributions should have a file naming convention that includes enough
information to decide whether or not a particular archive is compatible
with a particular implementation.

Previous efforts come from a time where CPython was the only important
implementation and the ABI was the same as the Python language release.
This specification improves upon the older schemes by including the Python
implementation, language version, ABI, and platform as a set of tags.

By comparing the tags it supports with the tags listed by the
distribution, an installer can make an educated decision about whether
to download a particular built distribution without having to read its
full metadata.

Overview
========

The tag format is {python tag}-{abi tag}-{platform tag}

python tag
    ?py27?, ?cp33?
abi tag
    ?cp32dmu?, ?none?
platform tag
    ?linux_x86_64?, ?any?

For example, the tag py27-none-any indicates compatible with Python 2.7
(any Python 2.7 implementation) with no abi requirement, on any platform.

Details
=======

Python Tag
----------

The Python tag indicates the implementation and version required by
a distribution.  Major implementations have abbreviated codes, initially:

* py: Generic Python (does not require implementation-specific features)
* cp: CPython
* ip: IronPython
* pp: PyPy
* jy: Jython

Other Python implementations should use `sys.implementation.name`.

The version is `py_version_nodot`.  CPython gets away with no dot,
but if one is needed the underscore `_` is used instead.  PyPy should
probably use its own versions here `pp18`, `pp19`.

The version can be just the major version `2` or `3` `py2`, `py3` for
many pure-Python distributions.

Importantly, major-version-only tags like `py2` and `py3` are not
shorthand for `py20` and `py30`.  Instead, these tags mean the packager
intentionally released a cross-version-compatible distribution.

A single-source Python 2/3 compatible distribution can use the compound
tag `py2.py3`.  See `Compressed Tag Sets`, below.

ABI Tag
-------

The ABI tag indicates which Python ABI is required by any included
extension modules.  For implementation-specific ABIs, the implementation
is abbreviated in the same way as the Python Tag, e.g. `cp33d` would be
the CPython 3.3 ABI with debugging.

The CPython stable ABI is `abi3` as in the shared library suffix, and
is available starting with Python 3.2.

Implementations with a very unstable ABI may use the first 6 bytes (as
8 base64-encoded characters) of the SHA-256 hash of ther source code
revision and compiler flags, etc, but will probably not have a great need
to distribute binary distributions. Each implementation's community may
decide how to best use the ABI tag.

Platform Tag
------------

The platform tag is simply `distutils.util.get_platform()` with all
hyphens `-` and periods `.` replaced with underscore `_`.

* win32
* linux_i386
* linux_x86_64

Use
===

The tags are used by installers to decide which built distribution
(if any) to download from a list of potential built distributions.
The installer maintains a list of (pyver, abi, arch) tuples that it
will support.  If the built distribution's tag is `in` the list, then
it can be installed.

For example, an installer running under CPython 3.3 on a linux_x86_64
system might support::

 1. cp33-cp33m-linux_x86_64
 2. cp33-none-linux_x86_64
 3. cp3-abi3-linux_x86_64
 4. cp33-none-any
 5. cp3-none-any
 6. py33-none-any
 7. py3-none-any

A user could instruct their installer to fall back to building from an
sdist more or less often by configuring this list of tags.

Rarely there will be more than one supported built distribution for a
particular version of a package.  For example, a packager could release
a package tagged `cp3-abi3-linux_x86_64` that contains an optional C
extension and the same distribution tagged `py3-none-any` that does not.
The index of the tag in the supported tags list breaks the tie, and the
package with the C extension is installed in preference to the package
without because that tag appears first in the list.

Compressed Tag Sets
===================

To allow for compact filenames of bdists that work with more than
one compatibility tag triple, each tag in a filename can instead be a
'.'-separated, sorted, set of tags.  For example, pip, a pure-Python
package that is written to run under Python 2 and 3 with the same source
code, could distribute a bdist with the tag `py2.py3-none-any`.
The full list of simple tags is::

    for x in pytag.split('.'):
        for y in abitag.split('.'):
            for z in archtag.split('.'):
                yield '-'.join((x, y, z))

A bdist format that implements this scheme should include the expanded
tags in bdist-specific metadata.  This compression scheme can generate
large numbers of unsupported tags and "impossible" tags that are supported
by no Python implementation e.g. "cp33-cp31u-win64", so use it sparingly.

FAQ
===

Can I have a tag `py32+` to indicate a minimum Python minor release?
    No.  Inspect the Trove classifiers to determine this level of
    cross-release compatibility.  Similar to the announcements "beaglevote
    versions 3.2 and above no longer supports Python 1.52", you will
    have to manually keep track of the maximum (PEP-386) release that
    still supports your version of Python.

Why isn't there a `.` in the Python version number?
    CPython has lasted 20+ years without a 3-digit major release. This
    should continue for some time.  Other implementations may use _ as
    a delimeter, since both - and . delimit the surrounding filename.

Who will maintain the registry of abbreviated implementations?
    New two-letter abbreviations can be requested on the python-dev
    mailing list.  As a rule of thumb, abbreviations are reserved for
    the current 4 most prominent implementations.

Does the compatibility tag go into METADATA or PKG-INFO?
    No.  The compatibility tag is part of the built distribution's
    metadata.  METADATA / PKG-INFO should be valid for an entire
    distribution, not a single build of that distribution.

Why didn't you mention my favorite Python implementation?
    The abbreviated tags facilitate sharing compiled Python code in a
    public index.  Your Python implementation can use this specification
    too, but with longer tags.
    Recall that all "pure Python" built distributions just use 'py'.

References
==========

.. [1] Egg Filename-Embedded Metadata
   (http://peak.telecommunity.com/DevCenter/EggFormats#filename-embedded-metadata)

.. [2] Creating Built Distributions
   (http://docs.python.org/distutils/builtdist.html)

.. [3] PEP 3147 -- PYC Repository Directories
   (http://www.python.org/dev/peps/pep-3147/)

Acknowledgements
================

The author thanks Paul Moore, Nick Coghlan, Mark Abramowitz, and
Mr. Michele Lacchia for their valuable advice and help with this effort.

Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

From dholth at gmail.com  Thu Oct 18 21:49:13 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 18 Oct 2012 15:49:13 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
Message-ID: <CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>

PEP: 426
Title: Metadata for Python Software Packages 1.3
Version: $Revision$
Last-Modified: $Date$
Author: Daniel Holth <dholth at fastmail.fm>
Discussions-To: Distutils SIG
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 30 Aug 2012


Abstract
========

This PEP describes a mechanism for adding metadata to Python distributions.
It includes specifics of the field names, and their semantics and
usage.

This document specifies version 1.3 of the metadata format.
Version 1.0 is specified in PEP 241.
Version 1.1 is specified in PEP 314.
Version 1.2 is specified in PEP 345.

Version 1.3 of the metadata format adds fields designed to make
third-party packaging of Python Software easier and defines a
formal extension mechanism.  The fields are "Setup-Requires-Dist"
"Provides-Extra", and "Extension".  This version also adds the `extra`
variable to the `environment markers` specification.

Metadata Files
==============

The syntax defined in this PEP is for use with Python distribution
metadata files. The file format is a simple UTF-8 encoded Key: value
format with no maximum line length. It is parsable by the ``email``
module with an appropriate ``email.policy.Policy()``.  The field names
listed in the `Fields`_ section are used as the header names.

There are two standard locations for these metadata files:

* the ``PKG-INFO`` file included in the base directory of Python
  source distribution archives (as created by the distutils ``sdist``
  command)
* the ``dist-info/METADATA`` files in a Python installation database, as
  described in PEP 376.

Other tools involved in Python distribution may choose to record this
metadata in additional tool-specific locations (e.g. as part of a
binary distribution archive format).

Encoding
========

Metadata 1.3 files are UTF-8 with the restriction that keys must be
ASCII. Parser implementations should be aware that older versions of
the Metadata specification do not specify an encoding.

Fields
======

This section specifies the names and semantics of each of the
supported metadata fields.

Fields marked with "(Multiple use)" may be specified multiple
times in a single metadata file.  Other fields may only occur
once in a metadata file.  Fields marked with "(optional)" are
not required to appear in a valid metadata file; all other
fields must be present.

Metadata-Version
::::::::::::::::

Version of the file format; "1.3" is the only legal value.

Example::

    Metadata-Version: 1.3


Name
::::

The name of the distribution.

Example::

    Name: BeagleVote


Version
:::::::

A string containing the distribution's version number.  This
field  must be in the format specified in PEP 386.

Example::

    Version: 1.0a2


Platform (multiple use)
:::::::::::::::::::::::

A Platform specification describing an operating system supported by
the distribution which is not listed in the "Operating System" Trove
classifiers.
See "Classifier" below.

Examples::

    Platform: ObscureUnix
    Platform: RareDOS


Supported-Platform (multiple use)
:::::::::::::::::::::::::::::::::

Binary distributions containing a metadata file will use the
Supported-Platform field in their metadata to specify the OS and
CPU for which the binary distribution was compiled.  The semantics of
the Supported-Platform field are not specified in this PEP.

Example::

    Supported-Platform: RedHat 7.2
    Supported-Platform: i386-win32-2791


Summary
:::::::

A one-line summary of what the distribution does.

Example::

    Summary: A module for collecting votes from beagles.


Description (optional)
::::::::::::::::::::::

A longer description of the distribution that can run to several
paragraphs.  Software that deals with metadata should not assume
any maximum size for this field, though people shouldn't include
their instruction manual as the description.

The contents of this field can be written using reStructuredText
markup [1]_.  For programs that work with the metadata, supporting
markup is optional; programs can also display the contents of the
field as-is.  This means that authors should be conservative in
the markup they use.

To support empty lines and lines with indentation with respect to
the RFC 822 format, any CRLF character has to be suffixed by 7 spaces
followed by a pipe ("|") char. As a result, the Description field is
encoded into a folded field that can be interpreted by RFC822
parser [2]_.

Example::

    Description: This project provides powerful math functions
            |For example, you can use `sum()` to sum numbers:
            |
            |Example::
            |
            |    >>> sum(1, 2)
            |    3
            |

This encoding implies that any occurences of a CRLF followed by 7 spaces
and a pipe char have to be replaced by a single CRLF when the field is unfolded
using a RFC822 reader.


Keywords (optional)
:::::::::::::::::::

A list of additional keywords to be used to assist searching
for the distribution in a larger catalog.

Example::

    Keywords: dog puppy voting election


Home-page (optional)
::::::::::::::::::::

A string containing the URL for the distribution's home page.

Example::

    Home-page: http://www.example.com/~cschultz/bvote/


Download-URL
::::::::::::

A string containing the URL from which this version of the distribution
can be downloaded.  (This means that the URL can't be something like
".../BeagleVote-latest.tgz", but instead must be ".../BeagleVote-0.45.tgz".)


Author (optional)
:::::::::::::::::

A string containing the author's name at a minimum; additional
contact information may be provided.

Example::

    Author: C. Schultz, Universal Features Syndicate,
            Los Angeles, CA <cschultz at peanuts.example.com>


Author-email (optional)
:::::::::::::::::::::::

A string containing the author's e-mail address.  It can contain
a name and e-mail address in the legal forms for a RFC-822
``From:`` header.

Example::

    Author-email: "C. Schultz" <cschultz at example.com>


Maintainer (optional)
:::::::::::::::::::::

A string containing the maintainer's name at a minimum; additional
contact information may be provided.

Note that this field is intended for use when a project is being
maintained by someone other than the original author:  it should be
omitted if it is identical to ``Author``.

Example::

    Maintainer: C. Schultz, Universal Features Syndicate,
            Los Angeles, CA <cschultz at peanuts.example.com>


Maintainer-email (optional)
:::::::::::::::::::::::::::

A string containing the maintainer's e-mail address.  It can contain
a name and e-mail address in the legal forms for a RFC-822
``From:`` header.

Note that this field is intended for use when a project is being
maintained by someone other than the original author:  it should be
omitted if it is identical to ``Author-email``.

Example::

    Maintainer-email: "C. Schultz" <cschultz at example.com>


License (optional)
::::::::::::::::::

Text indicating the license covering the distribution where the license
is not a selection from the "License" Trove classifiers. See
"Classifier" below.  This field may also be used to specify a
particular version of a licencse which is named via the ``Classifier``
field, or to indicate a variation or exception to such a license.

Examples::

    License: This software may only be obtained by sending the
            author a postcard, and then the user promises not
            to redistribute it.

    License: GPL version 3, excluding DRM provisions


Classifier (multiple use)
:::::::::::::::::::::::::

Each entry is a string giving a single classification value
for the distribution.  Classifiers are described in PEP 301 [2].

Examples::

    Classifier: Development Status :: 4 - Beta
    Classifier: Environment :: Console (Text Based)


Requires-Dist (multiple use)
::::::::::::::::::::::::::::

Each entry contains a string naming some other distutils
project required by this distribution.

The format of a requirement string is identical to that of a
distutils project name (e.g., as found in the ``Name:`` field.
optionally followed by a version declaration within parentheses.

The distutils project names should correspond to names as found
on the `Python Package Index`_.

Version declarations must follow the rules described in
`Version Specifiers`_

Examples::

    Requires-Dist: pkginfo
    Requires-Dist: PasteDeploy
    Requires-Dist: zope.interface (>3.5.0)


Setup-Requires-Dist (multiple use)
::::::::::::::::::::::::::::::::::

Like Requires-Dist, but names dependencies needed while the
distributions's distutils / packaging `setup.py` / `setup.cfg` is run.
Commonly used to generate a manifest from version control.

Examples::

    Setup-Requires-Dist: custom_setup_command

Dependencies mentioned in `Setup-Requires-Dist` may be installed exclusively
for setup and are not guaranteed to be available at run time.


Provides-Dist (multiple use)
::::::::::::::::::::::::::::

Each entry contains a string naming a Distutils project which
is contained within this distribution.  This field *must* include
the project identified in the ``Name`` field, followed by the
version : Name (Version).

A distribution may provide additional names, e.g. to indicate that
multiple projects have been bundled together.  For instance, source
distributions of the ``ZODB`` project have historically included
the ``transaction`` project, which is now available as a separate
distribution.  Installing such a source distribution satisfies
requirements for both ``ZODB`` and ``transaction``.

A distribution may also provide a "virtual" project name, which does
not correspond to any separately-distributed project:  such a name
might be used to indicate an abstract capability which could be supplied
by one of multiple projects.  E.g., multiple projects might supply
RDBMS bindings for use by a given ORM:  each project might declare
that it provides ``ORM-bindings``, allowing other projects to depend
only on having at most one of them installed.

A version declaration may be supplied and must follow the rules described
in `Version Specifiers`_. The distribution's version number will be implied
if none is specified.

Examples::

    Provides-Dist: OtherProject
    Provides-Dist: AnotherProject (3.4)
    Provides-Dist: virtual_package


Obsoletes-Dist (multiple use)
:::::::::::::::::::::::::::::

Each entry contains a string describing a distutils project's distribution
which this distribution renders obsolete, meaning that the two projects
should not be installed at the same time.

Version declarations can be supplied.  Version numbers must be in the
format specified in `Version Specifiers`_.

The most common use of this field will be in case a project name
changes, e.g. Gorgon 2.3 gets subsumed into Torqued Python 1.0.
When you install Torqued Python, the Gorgon distribution should be
removed.

Examples::

    Obsoletes-Dist: Gorgon
    Obsoletes-Dist: OtherProject (<3.0)


Requires-Python
:::::::::::::::

This field specifies the Python version(s) that the distribution is
guaranteed to be compatible with.

Version numbers must be in the format specified in `Version Specifiers`_.

Examples::

    Requires-Python: 2.5
    Requires-Python: >2.1
    Requires-Python: >=2.3.4
    Requires-Python: >=2.5,<2.7


Requires-External (multiple use)
::::::::::::::::::::::::::::::::

Each entry contains a string describing some dependency in the
system that the distribution is to be used.  This field is intended to
serve as a hint to downstream project maintainers, and has no
semantics which are meaningful to the ``distutils`` distribution.

The format of a requirement string is a name of an external
dependency, optionally followed by a version declaration within
parentheses.

Because they refer to non-Python software releases, version numbers
for this field are **not** required to conform to the format
specified in PEP 386:  they should correspond to the
version scheme used by the external dependency.

Notice that there's is no particular rule on the strings to be used.

Examples::

    Requires-External: C
    Requires-External: libpng (>=1.5)


Project-URL (multiple-use)
::::::::::::::::::::::::::

A string containing a browsable URL for the project and a label for it,
separated by a comma.

Example::

    Bug Tracker, http://bitbucket.org/tarek/distribute/issues/

The label is a free text limited to 32 signs.


Provides-Extra (multiple use)
:::::::::::::::::::::::::::::

A string containing the name of an optional feature. Must be printable
ASCII, not containing whitespace, comma (,), or square brackets [].
May be used to make a dependency conditional on whether the optional
feature has been requested.

Example::

    Name: beaglevote
    Provides-Extra: pdf
    Requires-Dist: reportlab; extra == 'pdf'
    Requires-Dist: nose; extra == 'test'
    Requires-Dist: sphinx; extra == 'doc'

A second distribution requires an optional dependency by placing it
inside square brackets and can request multiple features by separating
them with a comma (,).

The full set of requirements is the union of the `Requires-Dist` sets
evaluated with `extra` set to `None` and then to the name of each
requested feature.

Example::

    Requires-Dist: beaglevote[pdf]
        -> requires beaglevote, reportlab

    Requires-Dist: beaglevote[test, doc]
        -> requires beaglevote, sphinx, nose

Two feature names `test` and `doc` are reserved to mark dependencies that
are needed for running automated tests and generating documentation,
respectively.

It is legal to specify `Provides-Extra` without referencing it in any
`Requires-Dist`. It is an error to request a feature name that has
not been declared with `Provides-Extra`.


Extension (multiple-use)
::::::::::::::::::::::::

An ASCII string, not containing whitespace or the / character, that
indicates the presence of extended metadata. Additional tags defined by
the extension should be of the form string/Name::

    Extension: Chili
    Chili/Type: Poblano
    Chili/Heat: Mild


Version Specifiers
==================

Version specifiers are a series of conditional operators and
version numbers, separated by commas.  Conditional operators
must be one of "<", ">", "<=", ">=", "==" and "!=".

Any number of conditional operators can be specified, e.g.
the string ">1.0, !=1.3.4, <2.0" is a legal version declaration.
The comma (",") is equivalent to the **and** operator.

Each version number must be in the format specified in PEP 386.

When a version is provided, it always includes all versions that
starts with the same value. For example the "2.5" version of Python
will include versions like "2.5.2" or "2.5.3". Pre and post releases
in that case are excluded. So in our example, versions like "2.5a1" are
not included when "2.5" is used. If the first version of the range is
required, it has to be explicitly given. In our example, it will be
"2.5.0".

Notice that some projects might omit the ".0" prefix for the first release
of the "2.5.x" series:

- 2.5
- 2.5.1
- 2.5.2
- etc.

In that case, "2.5.0" will have to be explicitly used to avoid any confusion
between the "2.5" notation that represents the full range. It is a recommended
practice to use schemes of the same length for a series to completely avoid
this problem.

Some Examples:

- ``Requires-Dist: zope.interface (3.1)``: any version that starts with 3.1,
  excluding post or pre-releases.
- ``Requires-Dist: zope.interface (==3.1)``: equivalent to ``Requires-Dist:
  zope.interface (3.1)``.
- ``Requires-Dist: zope.interface (3.1.0)``: any version that starts with
  3.1.0, excluding post or pre-releases. Since that particular project doesn't
  use more than 3 digits, it also means "only the 3.1.0 release".
- ``Requires-Python: 3``: Any Python 3 version, no matter wich one, excluding
  post or pre-releases.
- ``Requires-Python: >=2.6,<3``: Any version of Python 2.6 or 2.7, including
  post releases of 2.6, pre and post releases of 2.7. It excludes pre releases
  of Python 3.
- ``Requires-Python: 2.6.2``: Equivalent to ">=2.6.2,<2.6.3". So this includes
  only Python 2.6.2. Of course, if Python was numbered with 4 digits, it would
  have include all versions of the 2.6.2 series.
- ``Requires-Python: 2.5.0``: Equivalent to ">=2.5.0,<2.5.1".
- ``Requires-Dist: zope.interface (3.1,!=3.1.3)``: any version that starts with
  3.1, excluding post or pre-releases of 3.1 *and* excluding any version that
  starts with "3.1.3". For this particular project, this means: "any version
  of the 3.1 series but not 3.1.3". This is equivalent to:
  ">=3.1,!=3.1.3,<3.2".

Environment markers
===================

An **environment marker** is a marker that can be added at the end of a
field after a semi-colon (";"), to add a condition about the execution
environment.

Here are some example of fields using such markers::

   Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
   Obsoletes-Dist: pywin31; sys.platform == 'win32'
   Requires-Dist: foo (1,!=1.3); platform.machine == 'i386'
   Requires-Dist: bar; python_version == '2.4' or python_version == '2.5'
   Requires-External: libxslt; 'linux' in sys.platform

The micro-language behind this is a simple subset of Python: it compares
only strings, with the ``==`` and ``in`` operators (and their opposites),
and with the ability to combine expressions. Parenthesis are supported
for grouping.

The pseudo-grammar is ::

    EXPR [in|==|!=|not in] EXPR [or|and] ...

where ``EXPR`` belongs to any of those:

- python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1])
- python_full_version = sys.version.split()[0]
- os.name = os.name
- sys.platform = sys.platform
- platform.version = platform.version()
- platform.machine = platform.machine()
- platform.python_implementation = platform.python_implementation()
- a free string, like ``'2.4'``, or ``'win32'``
- extra = (name of requested feature) or None

Notice that ``in`` is restricted to strings, meaning that it is not possible
to use other sequences like tuples or lists on the right side.

The fields that benefit from this marker are:

- Requires-Python
- Requires-External
- Requires-Dist
- Setup-Requires-Dist
- Provides-Dist
- Obsoletes-Dist
- Classifier

(The `extra` variable is only meaningful for Requires-Dist.)

Summary of Differences From PEP 345
===================================

* Values are now expected to be UTF-8

* Metadata-Version is now 1.3.

* Added `extra` to environment markers.

* Changed fields:

  - Requires-Dist

* Added fields:

  - Setup-Requires-Dist
  - Provides-Extra
  - Extension

References
==========

This document specifies version 1.3 of the metadata format.
Version 1.0 is specified in PEP 241.
Version 1.1 is specified in PEP 314.
Version 1.2 is specified in PEP 345.

.. [1] reStructuredText markup:
   http://docutils.sourceforge.net/

.. _`Python Package Index`: http://pypi.python.org/pypi/

.. [2] RFC 822 Long Header Fields:
   http://www.freesoft.org/CIE/RFC/822/7.htm

Copyright
=========

This document has been placed in the public domain.


..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:

From dholth at gmail.com  Thu Oct 18 21:49:40 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 18 Oct 2012 15:49:40 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
Message-ID: <CAG8k2+4Tk_tby0QF+NkvbFkBvQnANJBnyEnK5fu0O6xyDbF=CQ@mail.gmail.com>

PEP: 427
Title: The Wheel Binary Package Format 0.1
Version: $Revision$
Last-Modified: $Date$
Author: Daniel Holth <dholth at fastmail.fm>
Discussions-To: <distutils-sig at python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 20-Sep-2012
Post-History:


Abstract
========

This PEP describes a built-package format for Python called "wheel".

A wheel is a ZIP-format archive with a specially formatted file name
and the ``.whl`` extension.  It contains a single distribution nearly
as it would be installed according to PEP 376 with a particular
installation scheme.  A wheel file may be installed by simply
unpacking into site-packages with the standard 'unzip' tool, while
preserving enough information to spread its contents out onto their
final paths at any later time.


Note
====

This draft PEP describes version 0.1 of the "wheel" format. When the PEP
is accepted, the version will be changed to 1.0.  (The major version
is used to indicate potentially backwards-incompatible changes to the
format.)


Rationale
=========

Python needs a package format that is easier to install than sdist.
Python's sdist packages are defined by and require the distutils and
setuptools build systems, running arbitrary code to build-and-install,
and re-compile, code just so it can be installed into a new
virtualenv.  This system of conflating build-install is slow, hard to
maintain, and hinders innovation in both build systems and installers.

Wheel attempts to remedy these problems by providing a simpler
interface between the build system and the installer.  The wheel
binary package format frees installers from having to know about the
build system, saves time by amortizing compile time over many
installations, and removes the need to install a build system in the
target environment.


Details
=======

Installing a wheel 'distribution-1.0.py32.none.any.whl'
-------------------------------------------------------

Wheel installation notionally consists of two phases:

- Unpack.

  a. Parse ``distribution-1.0.dist-info/WHEEL``.
  b. Check that installer is compatible with Wheel-Version.  Warn if
     minor version is greater, abort if major version is greater.
  c. If Root-Is-Purelib == 'true', unpack archive into purelib
     (site-packages).
  d. Else unpack archive into platlib (site-packages).

- Spread.

  a. Unpacked archive includes ``distribution-1.0.dist-info/`` and (if
     there is data) ``distribution-1.0.data/``.
  b. Move each subtree of ``distribution-1.0.data/`` onto its
     destination path. Each subdirectory of ``distribution-1.0.data/``
     is a key into a dict of destination directories, such as
     ``distribution-1.0.data/(purelib|platlib|headers|scripts|data)``.
     The initially supported paths are taken from
     ``distutils.command.install``.
  c. If applicable, update scripts starting with ``#!python`` to point
     to the correct interpreter.
  d. Update ``distribution-1.0.dist.info/RECORD`` with the installed
     paths.
  e. Remove empty ``distribution-1.0.data`` directory.
  f. Compile any installed .py to .pyc. (Uninstallers should be smart
     enough to remove .pyc even if it is not mentioned in RECORD.)

Recommended installer features
''''''''''''''''''''''''''''''

Rewrite ``#!python``.
    In wheel, scripts are packaged in
    ``{distribution}-{version}.data/scripts/``.  If the first line of
    a file in ``scripts/`` starts with exactly b'#!python', rewrite to
    point to the correct interpreter.  Unix installers may need to add
    the +x bit to these files if the archive was created on Windows.

Generate script wrappers.
    In wheel, scripts packaged on Unix systems will certainly not have
    accompanying .exe wrappers.  Windows installers may want to add them
    during install.


File Format
-----------

File name convention
''''''''''''''''''''

The wheel filename is ``{distribution}-{version}(-{build
tag})?-{python tag}-{abi tag}-{platform tag}.whl``.

distribution
    Distribution name, e.g. 'django', 'pyramid'.

version
    PEP-386 compliant version, e.g. 1.0.

build tag
    Optional build number.  Must start with a digit.  A tie breaker if
    two wheels have the same version.  Sort as None if unspecified,
    else sort the initial digits as a number, and the remainder
    lexicographically.

language implementation and version tag
    E.g. 'py27', 'py2', 'py3'.

abi tag
    E.g. 'cp33m', 'abi3', 'none'.

platform tag
    E.g. 'linux_x86_64', 'any'.

For example, ``distribution-1.0-1-py27-none-any.whl`` is the first
build of a package called 'distribution', and is compatible with
Python 2.7 (any Python 2.7 implementation), with no ABI (pure Python),
on any CPU architecture.

The last three components of the filename before the extension are
called "compatibility tags."  The compatibility tags express the
package's basic interpreter requirements and are detailed in PEP 425.


File contents
'''''''''''''

#. Wheel files contain a folder {distribution}-{version}.dist-info/
   with the PEP 426 metadata (Metadata version 1.3 or greater) and an
   additional file WHEEL with metadata about the archive itself.
#. The root of a .whl is installed into one of purelib or platlib.
#. Wheel files contain metadata about the wheel format itself in
   ``{distribution}-{version}.dist-info/WHEEL``::

       Wheel-Version: 0.1
       Generator: bdist_wheel 0.7
       Root-Is-Purelib: true

#. Wheel-Version is the version number of the Wheel specification.
   Generator is the name and optionally the version of the software
   that produced the archive. Root-Is-Purelib is true if the top level
   directory of the archive should be installed into purelib;
   otherwise the root should be installed into platlib.
#. A wheel installer should warn if Wheel-Version is greater than the
   version it supports, and fail if Wheel-Version has a greater major
   version than the version it supports.
#. If a .whl contains scripts, both purelib and platlib, or any other
   files that are not installed on ``sys.path``, they are found in
   ``{distribution}-{version}.data/{key}``, where ``{key}`` is an
   index into a dictionary of install paths.
#. Wheel, being an installation format that is intended to work across
   multiple versions of Python, does not generally include .pyc files.
#. Wheel does not contain setup.py or setup.cfg.


The .dist-info directory
^^^^^^^^^^^^^^^^^^^^^^^^

#. Wheel .dist-info directories include at a minimum METADATA, WHEEL,
   and RECORD.
#. METADATA is the PEP 426 metadata (Metadata version 1.3 or greater)
#. WHEEL is the wheel metadata, specific to a build of the package.
#. RECORD is a list of (almost) all the files in the wheel and their
   secure hashes.  Unlike PEP 376, every file except RECORD, which
   cannot contain a hash of itself, must include its hash.  The hash
   algorithm must be sha256 or better; specifically, md5 and sha1 are
   not permitted, as signed wheel files rely on the strong hashes in
   RECORD to validate the integrity of the archive.
#. INSTALLER and REQUESTED are not included in the archive.
#. RECORD.jws is used for digital signatures.  It is not mentioned in
   RECORD.
#. RECORD.p7s is reserved as a courtesy to anyone who would prefer to
   use s/mime signatures to secure their wheel files.  It is not
   mentioned in RECORD.
#. During extraction, wheel installers verify all the hashes in RECORD
   against the file contents.  Apart from RECORD and its signatures,
   installation will fail if any file in the archive is not both
   mentioned and correctly hashed in RECORD.


The .data directory
^^^^^^^^^^^^^^^^^^^

Any file that is not normally installed inside site-packages goes into
the .data directory, named as the .dist-info directory but with the
.data/ extension::

    distribution-1.0.dist-info/

    distribution-1.0.data/

The .data directory contains subdirectories and those subdirectories
contain the scripts, headers, documentation and so forth from the
distribution.  During installation the contents of these subdirectories
are moved onto their destination paths.


Signed wheel files
------------------

Wheel files include an extended RECORD that enables digital
signatures.  PEP 376's RECORD is altered to include
``digestname=urlsafe_b64encode_nopad(digest)`` (urlsafe base64
encoding with no trailing = characters) as the second column instead
of an md5sum.  All possible entries are hashed, including any
generated files such as .pyc files, but not RECORD. For example::

    file.py,sha256=AVTFPZpEKzuHr7OvQZmhaU3LvwKz06AJw8mT\_pNh2yI,3144
    distribution-1.0.dist-info/RECORD,,

RECORD.jws is not mentioned in RECORD at all.  Every other file in the
archive must have a correct hash in RECORD, or the installation will
fail.

The signature is one or more JSON Web Signature JSON Serialization
(JWS-JS) signatures stored in a file RECORD.jws adjacent to RECORD.

A signature-aware installer can be instructed to check for a
particular Ed25519 public key by using an extended "extras" syntax.::

    # request a normal optional feature "extra", and indicate
    # the package should be signed by a particular
    # urlsafe-b64encode-nopad encoded ed25519 public key:
    package[extra, ed25519=ouBJlTJJ4SJXoy8Bi1KRlewWLU6JW7HUXTgvU1YRuiA]

An application could distribute a requires.txt file with many such
lines for all its dependencies and their public keys.  By installing
from this file an application's users would know whether the
application's dependencies came from the correct publishers.

Applications that wish to "fail open" for backwards compatibility with
non-signature-aware installers should specify that their package
provides the extra ``ed25519=(key)`` with no associated dependencies.

Key distribution is outside the scope of this spec.  Public wheel
signing keys could be signed with the packager's GPG key or stored at
an ``https://`` protected URL.


JSON Web Signatures Extensions
''''''''''''''''''''''''''''''

The Ed25519 algorithm is used as an extension to the JSON Web
Signatures specification.  Wheel uses ``alg="Ed25519"`` in the header.
The key attribute holds the signature's public JSON Web Key.  For JSON
Web Key / JSON Private Key the verifying (public) key is called vk and
the signing (private) key is called sk.

Example header::

    {
     "alg": "Ed25519",
     "typ": "JWT",
     "key": {
     "alg": "Ed25519",
     "vk": "tmAYCrSfj8gtJ10v3VkvW7jOndKmQIYE12hgnFu3cvk"
     }
    }

A future version of wheel may omit ``typ``.

Example payload::

    { "hash": "sha256=ADD-r2urObZHcxBW3Cr-vDCu5RJwT4CaRTHiFmbcIYY" }

A future version of wheel may include timestamps in the payload or in
the signature.

See

- http://self-issued.info/docs/draft-ietf-jose-json-web-signature.html
- http://self-issued.info/docs/draft-jones-json-web-signature-json-serialization-01.html
- http://self-issued.info/docs/draft-ietf-jose-json-web-key-05.html
- http://self-issued.info/docs/draft-jones-jose-json-private-key-00.html


Comparison to .egg
------------------

#. Wheel is an installation format; egg is importable.  Wheel archives
   do not need to include .pyc and are less tied to a specific Python
   version or implementation. Wheel can install (pure Python) packages
   built with previous versions of Python so you don't always have to
   wait for the packager to catch up.
#. Wheel uses .dist-info directories; egg uses .egg-info.  Wheel is
   compatible with the new world of Python packaging and the new
   concepts it brings.
#. Wheel has a richer file naming convention for today's
   multi-implementation world.  A single wheel archive can indicate
   its compatibility with a number of Python language versions and
   implementations, ABIs, and system architectures.  Historically the
   ABI has been specific to a CPython release, wheel is ready for the
   stable ABI.
#. Wheel is lossless.  The first wheel implementation bdist_wheel
   always generates egg-info, and then converts it to a .whl.  It is
   also possible to convert existing eggs and bdist_wininst
   distributions.
#. Wheel is versioned.  Every wheel file contains the version of the
   wheel specification and the implementation that packaged it.
   Hopefully the next migration can simply be to Wheel 2.0.
#. Wheel is a reference to the other Python.


FAQ
===

Wheel defines a .data directory.  Should I put all my data there?
    This specification does not have an opinion on how you should organize
    your code.  The .data directory is just a place for any files that are
    not normally installed inside ``site-packages`` or on the PYTHONPATH.


Copyright
=========

This document has been placed into the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

From victor.stinner at gmail.com  Fri Oct 19 03:03:53 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 19 Oct 2012 03:03:53 +0200
Subject: [Python-Dev] Why not using the hash when comparing strings?
Message-ID: <CAMpsgwYA3wL-KmkFh-wqi4aqLtrBqYzJ31CyTj48h9ng8B=mqA@mail.gmail.com>

Hi,

I would like to know if there a reason for not using the hash of
(bytes or unicode) strings when comparing two objects and the hash of
the two objects was already been computed. Using the hash would speed
up comparaison of long strings when the two strings are different.

Something like:

    if ((op == Py_EQ || op == Py_NE)
        && a->ob_shash != -1
        && b->ob_shash != -1
        && a->ob_shash != b->ob_shash) {
        /* strings are not equal */
    }

There are hash collision, so a->ob_shash == b->ob_shash doesn't mean
that the two strings are equal. But if the two hashs are different,
the two strings are different. Isn't it?

Victor

From steve at pearwood.info  Fri Oct 19 03:07:17 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 19 Oct 2012 12:07:17 +1100
Subject: [Python-Dev] Why not using the hash when comparing strings?
In-Reply-To: <CAMpsgwYA3wL-KmkFh-wqi4aqLtrBqYzJ31CyTj48h9ng8B=mqA@mail.gmail.com>
References: <CAMpsgwYA3wL-KmkFh-wqi4aqLtrBqYzJ31CyTj48h9ng8B=mqA@mail.gmail.com>
Message-ID: <5080A7C5.7050508@pearwood.info>

On 19/10/12 12:03, Victor Stinner wrote:
> Hi,
>
> I would like to know if there a reason for not using the hash of
> (bytes or unicode) strings when comparing two objects and the hash of
> the two objects was already been computed. Using the hash would speed
> up comparaison of long strings when the two strings are different.

Assuming the hash has already been compared, then I imagine it would be
faster.

> Something like:
>
>      if ((op == Py_EQ || op == Py_NE)
>          &&  a->ob_shash != -1
>          &&  b->ob_shash != -1
>          &&  a->ob_shash != b->ob_shash) {
>          /* strings are not equal */
>      }
>
> There are hash collision, so a->ob_shash == b->ob_shash doesn't mean
> that the two strings are equal. But if the two hashs are different,
> the two strings are different. Isn't it?

I would certainly hope so :)


-- 
Steven

From benjamin at python.org  Fri Oct 19 03:22:32 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 18 Oct 2012 21:22:32 -0400
Subject: [Python-Dev] Why not using the hash when comparing strings?
In-Reply-To: <CAMpsgwYA3wL-KmkFh-wqi4aqLtrBqYzJ31CyTj48h9ng8B=mqA@mail.gmail.com>
References: <CAMpsgwYA3wL-KmkFh-wqi4aqLtrBqYzJ31CyTj48h9ng8B=mqA@mail.gmail.com>
Message-ID: <CAPZV6o_N=N4VTKPs-i+NFiAnR+DRrBNvmC_z175q9F+fePz=fQ@mail.gmail.com>

2012/10/18 Victor Stinner <victor.stinner at gmail.com>:
> Hi,
>
> I would like to know if there a reason for not using the hash of
> (bytes or unicode) strings when comparing two objects and the hash of
> the two objects was already been computed. Using the hash would speed
> up comparaison of long strings when the two strings are different.
>
> Something like:
>
>     if ((op == Py_EQ || op == Py_NE)
>         && a->ob_shash != -1
>         && b->ob_shash != -1
>         && a->ob_shash != b->ob_shash) {
>         /* strings are not equal */
>     }
>
> There are hash collision, so a->ob_shash == b->ob_shash doesn't mean
> that the two strings are equal. But if the two hashs are different,
> the two strings are different. Isn't it?

It would be interesting to see how common it is for strings which have
their hash computed to be compared.



-- 
Regards,
Benjamin

From dholth at gmail.com  Fri Oct 19 03:23:37 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 18 Oct 2012 21:23:37 -0400
Subject: [Python-Dev] PEP 426 comment: field order
In-Reply-To: <CAG8k2+6Nyac+PjUkiW8GngtAVG0i8_9c3BjX5YoZE=V8P1r-Hw@mail.gmail.com>
References: <20121018204544.Horde.oJXcHklCcOxQgE5YhVcnAsA@webmail.df.eu>
	<CAG8k2+6Nyac+PjUkiW8GngtAVG0i8_9c3BjX5YoZE=V8P1r-Hw@mail.gmail.com>
Message-ID: <CAG8k2+407WW1+7RVOFnASUuOL2nJwGAMi+nAjOXb_SGMs4rDnQ@mail.gmail.com>

Added some notes about the (lack of) ordering.

The email module provides an ordered multidict interface to the data.
The first tag wins (if you improperly define Name: twice for example),
but the order of everything is preserved. We just don't need it,
except that it might be surprising to see your classifiers randomly
re-ordered.

It is also possible to textwrap.dedent(p['Description']) with p =
email.parser.Parser().parsestr(metadata).

I don't really expect anyone to use email.parser.Parser() so I'm
hesitant to mention it, but it seems to work. I say it's read-only
because the 3.2 Parser()/Generator() can't reserialize it due to
Unicode. The improved Python 3.3 email module would be able to under
the right conditions.


diff -r 79e95f487a33 -r 4773b6b3e8f2 pep-0426.txt
--- a/pep-0426.txt      Thu Oct 18 08:31:44 2012 +0100
+++ b/pep-0426.txt      Thu Oct 18 21:10:26 2012 -0400
@@ -33,10 +33,14 @@

 The syntax defined in this PEP is for use with Python distribution
 metadata files. The file format is a simple UTF-8 encoded Key: value
-format with no maximum line length. It is parsable by the ``email``
+format with no maximum line length.  It is parseable by the ``email``
 module with an appropriate ``email.policy.Policy()``.  The field names
 listed in the `Fields`_ section are used as the header names.

+In Python 3.2, a serviceable read-only parser is::
+
+    email.parser.Parser().parsestr(metadata)
+
 There are two standard locations for these metadata files:

 * the ``PKG-INFO`` file included in the base directory of Python
@@ -66,7 +70,8 @@
 times in a single metadata file.  Other fields may only occur
 once in a metadata file.  Fields marked with "(optional)" are
 not required to appear in a valid metadata file; all other
-fields must be present.
+fields must be present.  The fields may appear in any order within
+the file.

 Metadata-Version
 ::::::::::::::::
@@ -480,12 +485,17 @@

 An ASCII string, not containing whitespace or the / character, that
 indicates the presence of extended metadata. Additional tags defined by
-the extension should be of the form string/Name::
+an `Extension: Chili` should be of the form `Chili/Name`::

     Extension: Chili
     Chili/Type: Poblano
     Chili/Heat: Mild

+An implementation might iterate over all the declared `Extension:`
+fields to invoke the processors for those extensions.  As the order of
+the fields is not used, the `Extension: Chili` field may appear before
+or after its declared tags `Chili/Type:` etc.
+

From python at mrabarnett.plus.com  Fri Oct 19 03:30:41 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 19 Oct 2012 02:30:41 +0100
Subject: [Python-Dev] Why not using the hash when comparing strings?
In-Reply-To: <CAMpsgwYA3wL-KmkFh-wqi4aqLtrBqYzJ31CyTj48h9ng8B=mqA@mail.gmail.com>
References: <CAMpsgwYA3wL-KmkFh-wqi4aqLtrBqYzJ31CyTj48h9ng8B=mqA@mail.gmail.com>
Message-ID: <5080AD41.8030404@mrabarnett.plus.com>

On 2012-10-19 02:03, Victor Stinner wrote:
> Hi,
>
> I would like to know if there a reason for not using the hash of
> (bytes or unicode) strings when comparing two objects and the hash of
> the two objects was already been computed. Using the hash would speed
> up comparaison of long strings when the two strings are different.
>
> Something like:
>
>      if ((op == Py_EQ || op == Py_NE)
>          && a->ob_shash != -1
>          && b->ob_shash != -1
>          && a->ob_shash != b->ob_shash) {
>          /* strings are not equal */
>      }
>
> There are hash collision, so a->ob_shash == b->ob_shash doesn't mean
> that the two strings are equal. But if the two hashs are different,
> the two strings are different. Isn't it?
>
Correct. It's true for any hashable type.

From stephen at xemacs.org  Fri Oct 19 04:55:58 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 19 Oct 2012 11:55:58 +0900
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
Message-ID: <87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>

Executive summary:

You probably should include a full ABNF grammar....

Daniel Holth writes:

 > To support empty lines and lines with indentation with respect to
 > the RFC 822 format, any CRLF character has to be suffixed by 7 spaces
 > followed by a pipe ("|") char. [...]
 > This encoding implies that any occurences of a CRLF followed by 7 spaces
 > and a pipe char have to be replaced by a single CRLF when the field
 > is unfolded using a RFC822 reader.

This isn't RFC 822 unfolding at all.  An RFC 822 "reader" will simply
remove the CRLF and optionally "canonicalize" the spaces (the latter
is not allowed by RFC 822, but sometimes it's observed).  This implies
that if you use an RFC 822 reader, you need to replace instances of the
regexp r"\s+\|" with a newline.  (If you have a conforming reader, you
can use the regexp r"\s{7}\|" instead.)  And of course you have to
RFC-2047-encode non-ASCII in an RFC-822 field.

So please don't refer to the basic format ("field-name: field-body"
followed by optional continuation lines) as "RFC822".  "Inspired by
RFC 822" maybe.  Better "chosen to resemble the familiar RFC 822
header format used in email and netnews."  (Note that RFC 822 is
actually ambiguous even about the basic format; section 3.4.2 implies
that "name   :body" would be an acceptable field, although section
3.1.2 doesn't seem to allow space before the colon.  Referring to RFC
822 as a standard here is a bad idea.  There is a reason why that
standard gets revised/replaced periodically!)

I don't understand why you specify that the newline is represented by
CRLF *after* unfolding.  Once unfolded, these fields are all what
RFC822 would call "unstructured fields" (in that context of that RFC).
They will contain text followed by a terminating CRLF, but including
no others.  In fact that CRLF is redundant, and may as well be
stripped (and probably will be, in most implementations).

I don't understand why you specify newline as CRLF here, except to
pretend that you're respecting RFC 822.  But all you're using are the
division of a field into field-name and field-body by a colon, and the
convention that a newline followed by folding whitespace is a
continuation line.  These are both trivial to implement, and almost
all implementations will undoubtedly read the file as *text* in
universal newline mode.  I see no reason to specify a binary format.

 > Author-email (optional)
 > :::::::::::::::::::::::
 > 
 > A string containing the author's e-mail address.  It can contain
 > a name and e-mail address in the legal forms for a RFC-822
 > ``From:`` header.

Heavens above, no!  From RFC 822, this:

    Wilt . (the  Stilt) Chamberlain at NBA.US

is a legal email address, which probably would be represented
conventionally as

    "Wilt (the Stilt) Chamberlain" <Wilt.Chamberlain at NBA.US>

However, it's not at all clear that all mail clients, let alone just
plain folks, will interpret the first form correctly.  And there are
worse examples given in that RFC.  Is there a reason why you can't
require these to be in the form recommended by RFC 5322 (ie, the
"conventional representation" above)?  Or you could relax this so that
the quotes are prohibited.

 > License (optional)
 > ::::::::::::::::::
 > 
 > Text indicating the license covering the distribution where the license
 > is not a selection from the "License" Trove classifiers. See
 > "Classifier" below.  This field may also be used to specify a
 > particular version of a licencse which is named via the ``Classifier``
                                A
typo----------------------------+

 > field, or to indicate a variation or exception to such a license.

This won't do as is.  It doesn't exclude the possibility of including
a complete license, and if that is intentional, this field needs to be
in the same format as "Distribution".  Licenses are complex documents,
needing at least some of the power of something like ReST.  You may as
well give them all of it.

 > Project-URL (multiple-use)
 > Provides-Extra (multiple use)

Hyphen or no hyphen?  Consistency is good.


From ncoghlan at gmail.com  Fri Oct 19 05:02:56 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 19 Oct 2012 13:02:56 +1000
Subject: [Python-Dev] Rejecting PEPs 407 and 413?
Message-ID: <CADiSq7f_X84+sxj7k2A+iTBJ64Zu=VAKfJyMeb3T80LRmDbW3A@mail.gmail.com>

With the 3.4 release PEP published using a traditional schedule,
perhaps MvL would care to do the honours as BDFL-Delegate in rejecting
the two "faster release cycle for the standard library" PEPs?

(I know I asked to hold off on that when MvL last brought it up, but
I've since realised that "do the first alpha early" isn't a
stand-alone PEP, it's just a matter of convincing Larry it's
worthwhile and negotiating timing with the release team after there
are some release-worthy features on trunk)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From dholth at gmail.com  Fri Oct 19 05:21:53 2012
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 18 Oct 2012 23:21:53 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAG8k2+5zLb=Dw5LfAJmJgntV8pCyq49nNv8KuzYGo1Z58UyCGw@mail.gmail.com>

On Thu, Oct 18, 2012 at 10:55 PM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
> Executive summary:
>
> You probably should include a full ABNF grammar....
>
> Daniel Holth writes:
>
>  > To support empty lines and lines with indentation with respect to
>  > the RFC 822 format, any CRLF character has to be suffixed by 7 spaces
>  > followed by a pipe ("|") char. [...]
>  > This encoding implies that any occurences of a CRLF followed by 7 spaces
>  > and a pipe char have to be replaced by a single CRLF when the field
>  > is unfolded using a RFC822 reader.
>
> This isn't RFC 822 unfolding at all.  An RFC 822 "reader" will simply
> remove the CRLF and optionally "canonicalize" the spaces (the latter
> is not allowed by RFC 822, but sometimes it's observed).  This implies
> that if you use an RFC 822 reader, you need to replace instances of the
> regexp r"\s+\|" with a newline.  (If you have a conforming reader, you
> can use the regexp r"\s{7}\|" instead.)  And of course you have to
> RFC-2047-encode non-ASCII in an RFC-822 field.
>
> So please don't refer to the basic format ("field-name: field-body"
> followed by optional continuation lines) as "RFC822".  "Inspired by
> RFC 822" maybe.  Better "chosen to resemble the familiar RFC 822
> header format used in email and netnews."  (Note that RFC 822 is
> actually ambiguous even about the basic format; section 3.4.2 implies
> that "name   :body" would be an acceptable field, although section
> 3.1.2 doesn't seem to allow space before the colon.  Referring to RFC
> 822 as a standard here is a bad idea.  There is a reason why that
> standard gets revised/replaced periodically!)
>
> I don't understand why you specify that the newline is represented by
> CRLF *after* unfolding.  Once unfolded, these fields are all what
> RFC822 would call "unstructured fields" (in that context of that RFC).
> They will contain text followed by a terminating CRLF, but including
> no others.  In fact that CRLF is redundant, and may as well be
> stripped (and probably will be, in most implementations).
>
> I don't understand why you specify newline as CRLF here, except to
> pretend that you're respecting RFC 822.  But all you're using are the
> division of a field into field-name and field-body by a colon, and the
> convention that a newline followed by folding whitespace is a
> continuation line.  These are both trivial to implement, and almost
> all implementations will undoubtedly read the file as *text* in
> universal newline mode.  I see no reason to specify a binary format.
>
>  > Author-email (optional)
>  > :::::::::::::::::::::::
>  >
>  > A string containing the author's e-mail address.  It can contain
>  > a name and e-mail address in the legal forms for a RFC-822
>  > ``From:`` header.
>
> Heavens above, no!  From RFC 822, this:
>
>     Wilt . (the  Stilt) Chamberlain at NBA.US
>
> is a legal email address, which probably would be represented
> conventionally as
>
>     "Wilt (the Stilt) Chamberlain" <Wilt.Chamberlain at NBA.US>
>
> However, it's not at all clear that all mail clients, let alone just
> plain folks, will interpret the first form correctly.  And there are
> worse examples given in that RFC.  Is there a reason why you can't
> require these to be in the form recommended by RFC 5322 (ie, the
> "conventional representation" above)?  Or you could relax this so that
> the quotes are prohibited.
>
>  > License (optional)
>  > ::::::::::::::::::
>  >
>  > Text indicating the license covering the distribution where the license
>  > is not a selection from the "License" Trove classifiers. See
>  > "Classifier" below.  This field may also be used to specify a
>  > particular version of a licencse which is named via the ``Classifier``
>                                 A
> typo----------------------------+
>
>  > field, or to indicate a variation or exception to such a license.
>
> This won't do as is.  It doesn't exclude the possibility of including
> a complete license, and if that is intentional, this field needs to be
> in the same format as "Distribution".  Licenses are complex documents,
> needing at least some of the power of something like ReST.  You may as
> well give them all of it.
>
>  > Project-URL (multiple-use)
>  > Provides-Extra (multiple use)
>
> Hyphen or no hyphen?  Consistency is good.

I will include or remove the hyphen.

Your other comments are also true of the predecessor Metadata 1.2.

The | folding discussion could probably die. Personally I do not
respect RFC822 at all (in this format). I rather expect the pragmatic
implementer to more or less [line.split(':', 1) for line in
open('METADATA') if line[0].isalpha()]. The fields that matter at
runtime (Name, Version, Requires-Dist, Provides-Extra) are all
single-line only. Basically everything else is a curiosity for the
human reader.

The .dist-info (PEP 376) or the wheel spec should gain a well-known
file package-1.0.dist-info/LICENSE. Many open source licenses require
that you include the license with every copy of the program.

Thanks,

Daniel Holth

From hrvoje.niksic at avl.com  Fri Oct 19 11:22:09 2012
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Fri, 19 Oct 2012 11:22:09 +0200
Subject: [Python-Dev] Why not using the hash when comparing strings?
In-Reply-To: <CAPZV6o_N=N4VTKPs-i+NFiAnR+DRrBNvmC_z175q9F+fePz=fQ@mail.gmail.com>
References: <CAMpsgwYA3wL-KmkFh-wqi4aqLtrBqYzJ31CyTj48h9ng8B=mqA@mail.gmail.com>
	<CAPZV6o_N=N4VTKPs-i+NFiAnR+DRrBNvmC_z175q9F+fePz=fQ@mail.gmail.com>
Message-ID: <50811BC1.7090102@avl.com>

On 10/19/2012 03:22 AM, Benjamin Peterson wrote:
> It would be interesting to see how common it is for strings which have
> their hash computed to be compared.

Since all identifier-like strings mentioned in Python are interned, and 
therefore have had their hash computed, I would imagine comparing them 
to be fairly common. After all, strings are often used as makeshift 
enums in Python.

On the flip side, those strings are typically small, so a measurable 
overall speed improvement brought by such a change seems unlikely.

From duncan.booth at suttoncourtenay.org.uk  Fri Oct 19 12:02:19 2012
From: duncan.booth at suttoncourtenay.org.uk (Duncan Booth)
Date: Fri, 19 Oct 2012 10:02:19 +0000 (UTC)
Subject: [Python-Dev] Why not using the hash when comparing strings?
References: <CAMpsgwYA3wL-KmkFh-wqi4aqLtrBqYzJ31CyTj48h9ng8B=mqA@mail.gmail.com>
	<CAPZV6o_N=N4VTKPs-i+NFiAnR+DRrBNvmC_z175q9F+fePz=fQ@mail.gmail.com>
	<50811BC1.7090102@avl.com>
Message-ID: <XnsA0F170450281Aduncanrcpcouk@127.0.0.1>

Hrvoje Niksic <hrvoje.niksic at avl.com> wrote:

> On 10/19/2012 03:22 AM, Benjamin Peterson wrote:
>> It would be interesting to see how common it is for strings which have
>> their hash computed to be compared.
> 
> Since all identifier-like strings mentioned in Python are interned, and 
> therefore have had their hash computed, I would imagine comparing them 
> to be fairly common. After all, strings are often used as makeshift 
> enums in Python.
> 
> On the flip side, those strings are typically small, so a measurable 
> overall speed improvement brought by such a change seems unlikely.

I'm pretty sure it would result in a small slowdown.

Many (most?) of the comparisons against interned identifiers will be done 
by dictionary lookups and the dictionary lookup code only tries the string 
comparison after it has determined that the hashes match. The only time 
dictionary key strings contents are actually compared is when the hash 
matches but the pointers are different; it is already the case that if the 
hashes don't match the strings are never compared.


From larry at hastings.org  Fri Oct 19 13:01:09 2012
From: larry at hastings.org (Larry Hastings)
Date: Fri, 19 Oct 2012 04:01:09 -0700
Subject: [Python-Dev] Rejecting PEPs 407 and 413?
In-Reply-To: <CADiSq7f_X84+sxj7k2A+iTBJ64Zu=VAKfJyMeb3T80LRmDbW3A@mail.gmail.com>
References: <CADiSq7f_X84+sxj7k2A+iTBJ64Zu=VAKfJyMeb3T80LRmDbW3A@mail.gmail.com>
Message-ID: <508132F5.5030809@hastings.org>

On 10/18/2012 08:02 PM, Nick Coghlan wrote:
> With the 3.4 release PEP published using a traditional schedule,
> perhaps MvL would care to do the honours as BDFL-Delegate in rejecting
> the two "faster release cycle for the standard library" PEPs?
>
> (I know I asked to hold off on that when MvL last brought it up, but
> I've since realised that "do the first alpha early" isn't a
> stand-alone PEP, it's just a matter of convincing Larry it's
> worthwhile and negotiating timing with the release team after there
> are some release-worthy features on trunk)

FWIW I don't think those peps should be rejected simply because I didn't 
follow either for the 3.4 release schedule.  I think they should both 
have their day in the court of public opinion.  (Of course, maybe that 
day has already passed.)


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121019/ad994db5/attachment.html>

From ncoghlan at gmail.com  Fri Oct 19 13:33:50 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 19 Oct 2012 21:33:50 +1000
Subject: [Python-Dev] Rejecting PEPs 407 and 413?
In-Reply-To: <508132F5.5030809@hastings.org>
References: <CADiSq7f_X84+sxj7k2A+iTBJ64Zu=VAKfJyMeb3T80LRmDbW3A@mail.gmail.com>
	<508132F5.5030809@hastings.org>
Message-ID: <CADiSq7epDiY9W+SC0napxbuP2jsQsbyJo9mrSHEBVuJkA4qbgg@mail.gmail.com>

On Fri, Oct 19, 2012 at 9:01 PM, Larry Hastings <larry at hastings.org> wrote:
> FWIW I don't think those peps should be rejected simply because I didn't
> follow either for the 3.4 release schedule.  I think they should both have
> their day in the court of public opinion.  (Of course, maybe that day has
> already passed.)

Martin wanted to mark them rejected a while ago - reaction was
decidedly mixed, and the burden of proof to justify the extra workload
and complexity certainly wasn't met. I asked him to hold off because I
was planning to update 413 to the simple "early alphas" idea, but:
1. That's up to the RM rather than really needing a PEP
2. Even if it was a PEP level suggestion, a new PEP would be better
for a new idea anyway

At the moment, with the "3.4" used throughout the examples in both
PEPs, it's a little confusing w.r.t the actual 3.4 release PEP. I
could live with "Deferred" instead of "Rejected", but one or the other
should happen.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From victor.stinner at gmail.com  Fri Oct 19 14:36:39 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 19 Oct 2012 14:36:39 +0200
Subject: [Python-Dev] Why not using the hash when comparing strings?
In-Reply-To: <CAPZV6o_N=N4VTKPs-i+NFiAnR+DRrBNvmC_z175q9F+fePz=fQ@mail.gmail.com>
References: <CAMpsgwYA3wL-KmkFh-wqi4aqLtrBqYzJ31CyTj48h9ng8B=mqA@mail.gmail.com>
	<CAPZV6o_N=N4VTKPs-i+NFiAnR+DRrBNvmC_z175q9F+fePz=fQ@mail.gmail.com>
Message-ID: <CAMpsgwakwKnWH2tyNQJi-_KLPu=Q8AgWgb1e9UrrYz3s17pr7g@mail.gmail.com>

2012/10/19 Benjamin Peterson <benjamin at python.org>:
> It would be interesting to see how common it is for strings which have
> their hash computed to be compared.

I implemented a quick hack. When running "./python -m test test_os":
Python calls PyUnicode_RichCompare() 15206 times with Py_EQ or Py_NE
operator. In 41.4% (6295 calls), the hash of the two operands is
known. In 41.2% (6262 times on 15206), the hash of the two operands
are known *and are different*!

The hit rate may depend since when the process was started. For
example, in a fresh interpreter: the hit rate is only 7% (189 hit /
2703 calls).

When running the test suite, the hit rate is around 80% (hashs are
known in 90%) after running 70 tests. At the same time, the average of
string length is 4.1 characters and quite all strings are pure ASCII.

I create the issue http://bugs.python.org/issue16286 to discuss this
optimization.

Victor

From benjamin at python.org  Fri Oct 19 14:56:42 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 19 Oct 2012 08:56:42 -0400
Subject: [Python-Dev] Why not using the hash when comparing strings?
In-Reply-To: <XnsA0F170450281Aduncanrcpcouk@127.0.0.1>
References: <CAMpsgwYA3wL-KmkFh-wqi4aqLtrBqYzJ31CyTj48h9ng8B=mqA@mail.gmail.com>
	<CAPZV6o_N=N4VTKPs-i+NFiAnR+DRrBNvmC_z175q9F+fePz=fQ@mail.gmail.com>
	<50811BC1.7090102@avl.com>
	<XnsA0F170450281Aduncanrcpcouk@127.0.0.1>
Message-ID: <CAPZV6o_GR_J-crDSddLtYP+7-71+DK8gcSmdb7_Y7rYp=WhYcg@mail.gmail.com>

2012/10/19 Duncan Booth <duncan.booth at suttoncourtenay.org.uk>:
> Hrvoje Niksic <hrvoje.niksic at avl.com> wrote:
>
>> On 10/19/2012 03:22 AM, Benjamin Peterson wrote:
>>> It would be interesting to see how common it is for strings which have
>>> their hash computed to be compared.
>>
>> Since all identifier-like strings mentioned in Python are interned, and
>> therefore have had their hash computed, I would imagine comparing them
>> to be fairly common. After all, strings are often used as makeshift
>> enums in Python.
>>
>> On the flip side, those strings are typically small, so a measurable
>> overall speed improvement brought by such a change seems unlikely.
>
> I'm pretty sure it would result in a small slowdown.

Dictionary code has its own special path for string comparisons, so
this is not an issue.



-- 
Regards,
Benjamin

From oscar.j.benjamin at gmail.com  Fri Oct 19 15:09:05 2012
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Fri, 19 Oct 2012 14:09:05 +0100
Subject: [Python-Dev] Why not using the hash when comparing strings?
In-Reply-To: <XnsA0F170450281Aduncanrcpcouk@127.0.0.1>
References: <CAMpsgwYA3wL-KmkFh-wqi4aqLtrBqYzJ31CyTj48h9ng8B=mqA@mail.gmail.com>
	<CAPZV6o_N=N4VTKPs-i+NFiAnR+DRrBNvmC_z175q9F+fePz=fQ@mail.gmail.com>
	<50811BC1.7090102@avl.com>
	<XnsA0F170450281Aduncanrcpcouk@127.0.0.1>
Message-ID: <CAHVvXxQK4e-uF7q+omB9w5BmBKir4NqcxRh-srEVa_=eAVypSw@mail.gmail.com>

On 19 October 2012 11:02, Duncan Booth
<duncan.booth at suttoncourtenay.org.uk> wrote:
> Hrvoje Niksic <hrvoje.niksic at avl.com> wrote:
>
>> On 10/19/2012 03:22 AM, Benjamin Peterson wrote:
>>> It would be interesting to see how common it is for strings which have
>>> their hash computed to be compared.
>>
>> Since all identifier-like strings mentioned in Python are interned, and
>> therefore have had their hash computed, I would imagine comparing them
>> to be fairly common. After all, strings are often used as makeshift
>> enums in Python.
>>
>> On the flip side, those strings are typically small, so a measurable
>> overall speed improvement brought by such a change seems unlikely.
>
> I'm pretty sure it would result in a small slowdown.
>
> Many (most?) of the comparisons against interned identifiers will be done
> by dictionary lookups and the dictionary lookup code only tries the string
> comparison after it has determined that the hashes match. The only time
> dictionary key strings contents are actually compared is when the hash
> matches but the pointers are different; it is already the case that if the
> hashes don't match the strings are never compared.

My understanding is that:

The first part of string comparison is an identity check (pointer
comparison) which catches the case of comparing two equal interned
strings. What is not checked for is comparing two unequal interned
strings (check both strings are interned and then conclude that
unequal pointers means unequal strings). The hashes are also not
checked during a standard string comparison. The lengths are also not
checked since the unicode_compare function is a generic cmp() function
for a rich comparison rather than a specific function for
equal/unequal comparison:
http://hg.python.org/cpython/file/321414874b26/Objects/unicodeobject.c#l6114

Most string comparisons occur during dict lookup in which case the
hash is (implicitly?) checked before the strings are compared. A
second hash comparison in this case would often be redundant.


My *opinion* is that:

On average character by character string comparison stops very quickly
after comparing 1 or 2 characters. If this is assumed to be the case
then it might be wasteful to check whether the strings have hashes and
then compare the hashes (likewise for checking whether they are
interned).

This was the subject of lengthy debate in the recent python-list
thread "comparing strings from the back":
http://mail.python.org/pipermail/python-list/2012-September/629866.html

Consider the case where the strings are not interned, non-empty, and
differ in the first character:

    def current(s1, s2):
        if s1 is s2:
            return True
        elif s1.len > 0  and s2.len > 0 and s1[0] != s2[0]:
            return False
        # Most of the time we don't reach this point
        ...

    def proposed_hashcheck(s1, s2):
        if s1 is s2:
            return True
        elif s1.hash is not None and s2.hash is not None and s1.hash != s2.hash:
            return False
        # I think we often reach this point when not comparing within a dict
        elif s1.len > 0  and s2.len > 0 and s1[0] != s2[0]:
            return False
        ...

When comparing within a dict the hash check (modulo dict size) is
already implicitly likely-affirmative and I assume (I'm not sure) that
the full hashes are explicitly checked before comparing the strings.
Either way I think that the additional hash check within the string
comparison will normally be redundant.

When not comparing within a dict, it's hard to say how often the
strings have hashes but if they usually don't then the extra
hash-check will normally be redundant in that case as well. If the
character by character comparison fails at the first character it may
be quicker to optimise for that case than checking if both hashes are
present and then comparing the hashes.

Other potential optimisations include comparing length before
comparing characters:

    def proposed_lencheck(s1, s2):
        if s1 is s2:
            return True
        elif s1.len != s2.len:
            return False
        elif s1.len > 0  and s2.len > 0 and s1[0] != s2[0]:
            return False
        ...

In the above I'm trying to represent the fast path when comparing
strings that differ in the first character. One case that isn't helped
by either of length or hash checking is comparing equal strings: you
always need to check all the characters (unless the identity check
succeeds).

Depending on how often unequal interned strings are compared another
method could be:

    def proposed_interncheck(s1, s2):
        if s1 is s2:
            return True
        elif s1.interned and s2.interned:
            return False
        elif s1.len > 0  and s2.len > 0 and s1[0] != s2[0]:
            return False
        ...

All of these solutions add complexity over the current method and
require the string comparison functions to be refactored from the
current generic rich compare method. None of them (I think) would help
much when comparing strings whose first character differs (the common
case). The one case where they may help significantly is that
discussed in the other thread where you are comparing many strings
that for whatever reason have a long common prefix:

    '/usr/share/lib/prog/file1' == '/usr/share/lib/prog/file2'


Oscar

From storchaka at gmail.com  Fri Oct 19 15:36:03 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Fri, 19 Oct 2012 16:36:03 +0300
Subject: [Python-Dev] cpython (2.7): Issue #6074: Restore the
 long-broken support for running with read-only source
In-Reply-To: <3XjlyF3G9LzMdF@mail.python.org>
References: <3XjlyF3G9LzMdF@mail.python.org>
Message-ID: <50815743.4010608@gmail.com>

On 19.10.12 14:58, nick.coghlan wrote:
> http://hg.python.org/cpython/rev/321414874b26
> changeset:   79827:321414874b26
> branch:      2.7
> parent:      79814:2f0770cc6d3f
> user:        Nick Coghlan <ncoghlan at gmail.com>
> date:        Fri Oct 19 21:58:18 2012 +1000
> summary:
>    Issue #6074: Restore the long-broken support for running with read-only source files on Windows

> +def _iter_files(name):
>       for f in (name + os.extsep + "py",
>                 name + os.extsep + "pyc",
>                 name + os.extsep + "pyo",
>                 name + os.extsep + "pyw",
>                 name + "$py.class"):
> +        yield f

Why not just

     return (name + os.extsep + "py",
             name + os.extsep + "pyc",
             name + os.extsep + "pyo",
             name + os.extsep + "pyw",
             name + "$py.class")

?



From brett at python.org  Fri Oct 19 15:56:15 2012
From: brett at python.org (Brett Cannon)
Date: Fri, 19 Oct 2012 09:56:15 -0400
Subject: [Python-Dev] Why not using the hash when comparing strings?
In-Reply-To: <CAMpsgwakwKnWH2tyNQJi-_KLPu=Q8AgWgb1e9UrrYz3s17pr7g@mail.gmail.com>
References: <CAMpsgwYA3wL-KmkFh-wqi4aqLtrBqYzJ31CyTj48h9ng8B=mqA@mail.gmail.com>
	<CAPZV6o_N=N4VTKPs-i+NFiAnR+DRrBNvmC_z175q9F+fePz=fQ@mail.gmail.com>
	<CAMpsgwakwKnWH2tyNQJi-_KLPu=Q8AgWgb1e9UrrYz3s17pr7g@mail.gmail.com>
Message-ID: <CAP1=2W5JVuVPEQmoYW_MX3RH5Stnv2tLs=O9k0uqG_6OmgfG-Q@mail.gmail.com>

On Fri, Oct 19, 2012 at 8:36 AM, Victor Stinner <victor.stinner at gmail.com>wrote:

> 2012/10/19 Benjamin Peterson <benjamin at python.org>:
> > It would be interesting to see how common it is for strings which have
> > their hash computed to be compared.
>
> I implemented a quick hack. When running "./python -m test test_os":
> Python calls PyUnicode_RichCompare() 15206 times with Py_EQ or Py_NE
> operator. In 41.4% (6295 calls), the hash of the two operands is
> known. In 41.2% (6262 times on 15206), the hash of the two operands
> are known *and are different*!
>
> The hit rate may depend since when the process was started. For
> example, in a fresh interpreter: the hit rate is only 7% (189 hit /
> 2703 calls).
>
> When running the test suite, the hit rate is around 80% (hashs are
> known in 90%) after running 70 tests. At the same time, the average of
> string length is 4.1 characters and quite all strings are pure ASCII.
>
> I create the issue http://bugs.python.org/issue16286 to discuss this
> optimization.
>

If you want to measure the performance impact compared to a clean build
then you can use the unladen benchmarks as it contains several Python
3-compatible benchmarks now.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121019/345830f3/attachment.html>

From anto.cuni at gmail.com  Fri Oct 19 15:08:40 2012
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 19 Oct 2012 15:08:40 +0200
Subject: [Python-Dev] return type of __complex__
Message-ID: <508150D8.3060108@gmail.com>

Hi,

while fixing pypy to pass CPython 3.2 tests, I found what I think it's a
inconsistency in how CPython (both 2.7 and 3.2) handles __complex__:

>>> class Obj:
...     def __complex__(self):
...         return 2.0
...
>>> obj = Obj()
>>> complex(obj)
(2+0j)
>>>
>>> import cmath
>>> cmath.acos(obj)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __complex__ should return a complex object


i.e., the complex constructor does not check that __complex__ returns an
actual complex, while the cmath functions do.
To me it looks like a bug in complex_new which should do the check as well;
however, there is a test in test_complex.test_constructor which checks that
returning a float actually works.

Is that the real intended behavior?

ciao,
Anto

From ncoghlan at gmail.com  Fri Oct 19 15:57:50 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 19 Oct 2012 23:57:50 +1000
Subject: [Python-Dev] cpython (2.7): Issue #6074: Restore the
 long-broken support for running with read-only source
In-Reply-To: <50815743.4010608@gmail.com>
References: <3XjlyF3G9LzMdF@mail.python.org>
	<50815743.4010608@gmail.com>
Message-ID: <CADiSq7esndozDsiXzX2Y2csQxU7dpUscgRQ3c_d7nBL1dHMjNg@mail.gmail.com>

On Fri, Oct 19, 2012 at 11:36 PM, Serhiy Storchaka <storchaka at gmail.com> wrote:
> Why not just
>
>     return (name + os.extsep + "py",
>
>             name + os.extsep + "pyc",
>             name + os.extsep + "pyo",
>             name + os.extsep + "pyw",
>             name + "$py.class")

No good reason - I copied the shape of the existing code without
really thinking about it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Fri Oct 19 16:13:51 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 20 Oct 2012 00:13:51 +1000
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <508150D8.3060108@gmail.com>
References: <508150D8.3060108@gmail.com>
Message-ID: <CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>

On Fri, Oct 19, 2012 at 11:08 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> Is that the real intended behavior?

Given the way complex numbers interact with floats generally,
returning a complex number with no imaginary component as a floating
point value seems legitimate and the checks in cmath overly strict.
Otherwise you would get redundancy like:

    def __complex__(self):
        return complex(value)

or

    def __complex__(self):
        return value + 0j

More importantly, relaxing the checks in cmath is backwards
compatible. while tightening up the checks in complex_new is not.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From barry at python.org  Fri Oct 19 16:14:36 2012
From: barry at python.org (Barry Warsaw)
Date: Fri, 19 Oct 2012 10:14:36 -0400
Subject: [Python-Dev] Rejecting PEPs 407 and 413?
In-Reply-To: <CADiSq7epDiY9W+SC0napxbuP2jsQsbyJo9mrSHEBVuJkA4qbgg@mail.gmail.com>
References: <CADiSq7f_X84+sxj7k2A+iTBJ64Zu=VAKfJyMeb3T80LRmDbW3A@mail.gmail.com>
	<508132F5.5030809@hastings.org>
	<CADiSq7epDiY9W+SC0napxbuP2jsQsbyJo9mrSHEBVuJkA4qbgg@mail.gmail.com>
Message-ID: <20121019101436.67242aa7@resist.wooz.org>

On Oct 19, 2012, at 09:33 PM, Nick Coghlan wrote:

>At the moment, with the "3.4" used throughout the examples in both
>PEPs, it's a little confusing w.r.t the actual 3.4 release PEP. I
>could live with "Deferred" instead of "Rejected", but one or the other
>should happen.

I no longer have much interest (or frankly, time) to pursue PEP 407.  Deferred
seems about right to me.

-Barry

From barry at python.org  Fri Oct 19 16:20:25 2012
From: barry at python.org (Barry Warsaw)
Date: Fri, 19 Oct 2012 10:20:25 -0400
Subject: [Python-Dev] PEP 426 comment: field order
In-Reply-To: <CAG8k2+407WW1+7RVOFnASUuOL2nJwGAMi+nAjOXb_SGMs4rDnQ@mail.gmail.com>
References: <20121018204544.Horde.oJXcHklCcOxQgE5YhVcnAsA@webmail.df.eu>
	<CAG8k2+6Nyac+PjUkiW8GngtAVG0i8_9c3BjX5YoZE=V8P1r-Hw@mail.gmail.com>
	<CAG8k2+407WW1+7RVOFnASUuOL2nJwGAMi+nAjOXb_SGMs4rDnQ@mail.gmail.com>
Message-ID: <20121019102025.14d5f491@resist.wooz.org>

On Oct 18, 2012, at 09:23 PM, Daniel Holth wrote:

>The email module provides an ordered multidict interface to the data.
>The first tag wins (if you improperly define Name: twice for example),
>but the order of everything is preserved. We just don't need it,
>except that it might be surprising to see your classifiers randomly
>re-ordered.

Just to be clear, the email package preserves both the order and presence of
headers.  So if you do add Name: twice, both will be retained.  Plenty of
email headers (e.g. Received) can appear multiple times.

The getitem API will indeed return just the first entry, but there is an
alternative API that you can use to get all of them, in order.  Deletions and
re-insertions obviously change the order (the insertion is always an append),
although there is a .replace_header() method for preserving existing order
(kind of - only for the first instance of a header).

Cheers,
-Barry

From fijall at gmail.com  Fri Oct 19 16:21:06 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 19 Oct 2012 16:21:06 +0200
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
Message-ID: <CAK5idxSGvzKoHP98tKaRQ3rQpKvD6vMZj85DGAhAy18G97uVkQ@mail.gmail.com>

On Fri, Oct 19, 2012 at 4:13 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Fri, Oct 19, 2012 at 11:08 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
>> Is that the real intended behavior?
>
> Given the way complex numbers interact with floats generally,
> returning a complex number with no imaginary component as a floating
> point value seems legitimate and the checks in cmath overly strict.
> Otherwise you would get redundancy like:
>
>     def __complex__(self):
>         return complex(value)
>
> or
>
>     def __complex__(self):
>         return value + 0j
>
> More importantly, relaxing the checks in cmath is backwards
> compatible. while tightening up the checks in complex_new is not.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com

Cool, can we get it for 2.7?

From dholth at gmail.com  Fri Oct 19 16:29:53 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 19 Oct 2012 10:29:53 -0400
Subject: [Python-Dev] PEP 426 comment: field order
In-Reply-To: <20121019102025.14d5f491@resist.wooz.org>
References: <20121018204544.Horde.oJXcHklCcOxQgE5YhVcnAsA@webmail.df.eu>
	<CAG8k2+6Nyac+PjUkiW8GngtAVG0i8_9c3BjX5YoZE=V8P1r-Hw@mail.gmail.com>
	<CAG8k2+407WW1+7RVOFnASUuOL2nJwGAMi+nAjOXb_SGMs4rDnQ@mail.gmail.com>
	<20121019102025.14d5f491@resist.wooz.org>
Message-ID: <CAG8k2+4OiW0t4dGDh3d29MjR5vCqtEChSrsbt_hAJ5RSmq4YMQ@mail.gmail.com>

On Fri, Oct 19, 2012 at 10:20 AM, Barry Warsaw <barry at python.org> wrote:
> On Oct 18, 2012, at 09:23 PM, Daniel Holth wrote:
>
>>The email module provides an ordered multidict interface to the data.
>>The first tag wins (if you improperly define Name: twice for example),
>>but the order of everything is preserved. We just don't need it,
>>except that it might be surprising to see your classifiers randomly
>>re-ordered.
>
> Just to be clear, the email package preserves both the order and presence of
> headers.  So if you do add Name: twice, both will be retained.  Plenty of
> email headers (e.g. Received) can appear multiple times.
>
> The getitem API will indeed return just the first entry, but there is an
> alternative API that you can use to get all of them, in order.  Deletions and
> re-insertions obviously change the order (the insertion is always an append),
> although there is a .replace_header() method for preserving existing order
> (kind of - only for the first instance of a header).

It's a nice interface. I was surprised that there is no
collections.OrderedMultiDict.

From ncoghlan at gmail.com  Fri Oct 19 16:30:25 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 20 Oct 2012 00:30:25 +1000
Subject: [Python-Dev] Rejecting PEPs 407 and 413?
In-Reply-To: <20121019101436.67242aa7@resist.wooz.org>
References: <CADiSq7f_X84+sxj7k2A+iTBJ64Zu=VAKfJyMeb3T80LRmDbW3A@mail.gmail.com>
	<508132F5.5030809@hastings.org>
	<CADiSq7epDiY9W+SC0napxbuP2jsQsbyJo9mrSHEBVuJkA4qbgg@mail.gmail.com>
	<20121019101436.67242aa7@resist.wooz.org>
Message-ID: <CADiSq7egYUNZSakPYrVr3-0v=MdL+6hQ=15M7skSf0-aLseBOQ@mail.gmail.com>

On Sat, Oct 20, 2012 at 12:14 AM, Barry Warsaw <barry at python.org> wrote:
> On Oct 19, 2012, at 09:33 PM, Nick Coghlan wrote:
>
>>At the moment, with the "3.4" used throughout the examples in both
>>PEPs, it's a little confusing w.r.t the actual 3.4 release PEP. I
>>could live with "Deferred" instead of "Rejected", but one or the other
>>should happen.
>
> I no longer have much interest (or frankly, time) to pursue PEP 407.  Deferred
> seems about right to me.

Done - I marked them both as Deferred. If anyone is interested in
doing so, we can pick them up again in a year or two, as I expect this
isn't a question that will go away any time soon :)

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From anto.cuni at gmail.com  Fri Oct 19 17:15:42 2012
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 19 Oct 2012 17:15:42 +0200
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
Message-ID: <50816E9E.1070206@gmail.com>

On 10/19/2012 04:13 PM, Nick Coghlan wrote:
> On Fri, Oct 19, 2012 at 11:08 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
>> Is that the real intended behavior?
> 
> Given the way complex numbers interact with floats generally,
> returning a complex number with no imaginary component as a floating
> point value seems legitimate and the checks in cmath overly strict.
> Otherwise you would get redundancy like:
> 
>     def __complex__(self):
>         return complex(value)
> 
> or
> 
>     def __complex__(self):
>         return value + 0j
> 
> More importantly, relaxing the checks in cmath is backwards
> compatible. while tightening up the checks in complex_new is not.

indeed, you are right. So I suppose that in pypy we could just relax the check
in cmath and be happy. Is there any chance that this will be changed in 2.7
and/or 3.x?

ciao,
Anto


From benjamin at python.org  Fri Oct 19 17:26:01 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 19 Oct 2012 11:26:01 -0400
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <50816E9E.1070206@gmail.com>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
Message-ID: <CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>

2012/10/19 Antonio Cuni <anto.cuni at gmail.com>:
> indeed, you are right. So I suppose that in pypy we could just relax the check
> in cmath and be happy. Is there any chance that this will be changed in 2.7
> and/or 3.x?

Certainly 3.x, but not 2.7.



-- 
Regards,
Benjamin

From tseaver at palladion.com  Fri Oct 19 17:49:09 2012
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 19 Oct 2012 11:49:09 -0400
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
Message-ID: <k5rsq6$pt4$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/19/2012 11:26 AM, Benjamin Peterson wrote:
> 2012/10/19 Antonio Cuni <anto.cuni at gmail.com>:
>> indeed, you are right. So I suppose that in pypy we could just relax
>> the check in cmath and be happy. Is there any chance that this will
>> be changed in 2.7 and/or 3.x?
> 
> Certainly 3.x, but not 2.7.

Why not 2.7?  It is a perfectly-backward-compatible change:  no
currenly-working code could possibly break if cmath's restriction was
relaxed.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlCBdnUACgkQ+gerLs4ltQ77AQCgz90IKFRobiymE8yJmYhK+Axd
R3IAoMfZRBz40rOXk31QJmtQCnafaOnR
=dean
-----END PGP SIGNATURE-----


From benjamin at python.org  Fri Oct 19 17:56:47 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 19 Oct 2012 11:56:47 -0400
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <k5rsq6$pt4$2@ger.gmane.org>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
Message-ID: <CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>

2012/10/19 Tres Seaver <tseaver at palladion.com>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 10/19/2012 11:26 AM, Benjamin Peterson wrote:
>> 2012/10/19 Antonio Cuni <anto.cuni at gmail.com>:
>>> indeed, you are right. So I suppose that in pypy we could just relax
>>> the check in cmath and be happy. Is there any chance that this will
>>> be changed in 2.7 and/or 3.x?
>>
>> Certainly 3.x, but not 2.7.
>
> Why not 2.7?  It is a perfectly-backward-compatible change:  no
> currenly-working code could possibly break if cmath's restriction was
> relaxed.


It's a new feature. Also, it's possible that someone is relying on it
throwing for non-complex values.


-- 
Regards,
Benjamin

From status at bugs.python.org  Fri Oct 19 18:07:18 2012
From: status at bugs.python.org (Python tracker)
Date: Fri, 19 Oct 2012 18:07:18 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20121019160718.C61F41CE90@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2012-10-12 - 2012-10-19)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3815 (+26)
  closed 24249 (+48)
  total  28064 (+74)

Open issues with patches: 1701 


Issues opened (58)
==================

#2350: 'exceptions' import fixer
http://bugs.python.org/issue2350  reopened by brett.cannon

#11009: urllib.splituser is not documented
http://bugs.python.org/issue11009  reopened by techtonik

#16215: Possible double memory free in str.replace
http://bugs.python.org/issue16215  opened by serhiy.storchaka

#16216: Arithmetic operations with NULL
http://bugs.python.org/issue16216  opened by serhiy.storchaka

#16217: Tracebacks are unnecessarily verbose when using 'python -m'
http://bugs.python.org/issue16217  opened by mattheww

#16218: Python launcher does not support non ascii characters
http://bugs.python.org/issue16218  opened by turncc

#16219: segfault when using xml.etree.ElementTree.XMLTreeBuilder
http://bugs.python.org/issue16219  opened by scottmax

#16220: wsgiref does not call close() on iterable response
http://bugs.python.org/issue16220  opened by btubbs

#16221: tokenize.untokenize() "compat" mode misses the encoding when u
http://bugs.python.org/issue16221  opened by eric.snow

#16222: some terms not found by devguide's search box
http://bugs.python.org/issue16222  opened by chris.jerdonek

#16223: untokenize returns a string if no encoding token is recognized
http://bugs.python.org/issue16223  opened by eric.snow

#16224: tokenize.untokenize() misbehaves when moved to "compatiblity m
http://bugs.python.org/issue16224  opened by eric.snow

#16228: JSON crashes during encoding resized lists
http://bugs.python.org/issue16228  opened by serhiy.storchaka

#16229: Demo *redemo.py* lacking in Windows installation
http://bugs.python.org/issue16229  opened by fgracia

#16230: select.select crashes on resized lists
http://bugs.python.org/issue16230  opened by serhiy.storchaka

#16231: pickle persistent_id return value
http://bugs.python.org/issue16231  opened by pschaafsma

#16232: curses.textpad.Textbox backtrace support
http://bugs.python.org/issue16232  opened by emeaudroid.emeaudroid

#16233: IDLE: conceptual problems with *Class browser*
http://bugs.python.org/issue16233  opened by fgracia

#16234: Implement correct block_size and tests for HMAC-SHA3
http://bugs.python.org/issue16234  opened by christian.heimes

#16235: Add python-config.sh for use during cross compilation.
http://bugs.python.org/issue16235  opened by Ray.Donnelly

#16236: Doc/Makefile should have $PYTHON=python2
http://bugs.python.org/issue16236  opened by maker

#16237: bdist_rpm SPEC files created with distutils may be distro spec
http://bugs.python.org/issue16237  opened by ncoghlan

#16238: Automatically remove build directory when build options change
http://bugs.python.org/issue16238  opened by techtonik

#16239: PEP8 arithmetic operator examples
http://bugs.python.org/issue16239  opened by pwuertz

#16240: Document a way to escape metacharacters in glob/fnmatch
http://bugs.python.org/issue16240  opened by serhiy.storchaka

#16241: "-X faulthandler" is not documented in -X option docs
http://bugs.python.org/issue16241  opened by ramchandra.apte

#16242: Pickle and __getattr__
http://bugs.python.org/issue16242  opened by powderflask

#16243: Regression in inspect module
http://bugs.python.org/issue16243  opened by belopolsky

#16245: Update html.entities.html5 dictionary and parseentities.py
http://bugs.python.org/issue16245  opened by ezio.melotti

#16246: Multiprocessing infinite loop on Windows
http://bugs.python.org/issue16246  opened by stoneleaf

#16247: Report failing url in URLError?
http://bugs.python.org/issue16247  opened by ncoghlan

#16248: Security bug in tkinter allows for untrusted, arbitrary code e
http://bugs.python.org/issue16248  opened by ramchandra.apte

#16250: URLError invoked with reason as filename
http://bugs.python.org/issue16250  opened by chris.jerdonek

#16251: pickle special methods are looked up on the instance rather th
http://bugs.python.org/issue16251  opened by eric.snow

#16252: bytes and bytearray methods are undocumented
http://bugs.python.org/issue16252  opened by pitrou

#16253: Docs for PyArg_ParseTupleAndKeywords say it doesn't accept nes
http://bugs.python.org/issue16253  opened by rupole

#16254: PyUnicode_AsWideCharString() increases string size
http://bugs.python.org/issue16254  opened by dabeaz

#16255: subrocess.Popen needs /bin/sh but Android only has /system/bin
http://bugs.python.org/issue16255  opened by brousch

#16256: permissions wrong on Mac doc dir
http://bugs.python.org/issue16256  opened by Chris.Barker

#16258: test_local.TestEnUSCollection failures on Solaris 10
http://bugs.python.org/issue16258  opened by trent

#16259: Replace exec() in test.regrtest with __import__
http://bugs.python.org/issue16259  opened by ramchandra.apte

#16261: Fix bare excepts in various places in std lib
http://bugs.python.org/issue16261  opened by ramchandra.apte

#16262: srcdir != builddir builds fail, if hg is not installed
http://bugs.python.org/issue16262  opened by doko

#16263: sendmsg, recvmsg, recvmsg_into et al not being detected on Sol
http://bugs.python.org/issue16263  opened by trent

#16264: test_logging failure on Windows 7 buildbot
http://bugs.python.org/issue16264  opened by pitrou

#16267: order of decorators @abstractmethod and @classmethod is signif
http://bugs.python.org/issue16267  opened by christopherthemagnificent

#16268: dir(closure) does not find __dir__
http://bugs.python.org/issue16268  opened by christopherthemagnificent

#16271: weird dual behavior with changing __qualname__; different valu
http://bugs.python.org/issue16271  opened by christopherthemagnificent

#16272: C-API documentation clarification for tp_dictoffset
http://bugs.python.org/issue16272  opened by Chris.Colbert

#16273: f.tell() returning negative number on Windows build
http://bugs.python.org/issue16273  opened by rhettinger

#16275: test_ctypes fails on Solaris 10 SPARC 2.7 (nitrogen/s10)	(Sun 
http://bugs.python.org/issue16275  opened by trent

#16278: os.rename documentation slightly inaccurate
http://bugs.python.org/issue16278  opened by David.Benjamin

#16281: TODO in tailmatch(): it does not support backward in all cases
http://bugs.python.org/issue16281  opened by haypo

#16283: ctypes.util.find_library does not find all DLLs anymore
http://bugs.python.org/issue16283  opened by marcusva

#16284: concurrent.futures ThreadPoolExecutor keeps unnecessary refere
http://bugs.python.org/issue16284  opened by mark.dickinson

#16285: Update urllib to RFC 3986
http://bugs.python.org/issue16285  opened by Matteo.Parrucci

#16286: Optimize a==b and a!=b for bytes and str
http://bugs.python.org/issue16286  opened by haypo

#16287: Sporadic test_utime() failures on Solaris
http://bugs.python.org/issue16287  opened by trent



Most recent 15 issues with no replies (15)
==========================================

#16287: Sporadic test_utime() failures on Solaris
http://bugs.python.org/issue16287

#16284: concurrent.futures ThreadPoolExecutor keeps unnecessary refere
http://bugs.python.org/issue16284

#16283: ctypes.util.find_library does not find all DLLs anymore
http://bugs.python.org/issue16283

#16281: TODO in tailmatch(): it does not support backward in all cases
http://bugs.python.org/issue16281

#16278: os.rename documentation slightly inaccurate
http://bugs.python.org/issue16278

#16272: C-API documentation clarification for tp_dictoffset
http://bugs.python.org/issue16272

#16271: weird dual behavior with changing __qualname__; different valu
http://bugs.python.org/issue16271

#16253: Docs for PyArg_ParseTupleAndKeywords say it doesn't accept nes
http://bugs.python.org/issue16253

#16245: Update html.entities.html5 dictionary and parseentities.py
http://bugs.python.org/issue16245

#16242: Pickle and __getattr__
http://bugs.python.org/issue16242

#16241: "-X faulthandler" is not documented in -X option docs
http://bugs.python.org/issue16241

#16240: Document a way to escape metacharacters in glob/fnmatch
http://bugs.python.org/issue16240

#16238: Automatically remove build directory when build options change
http://bugs.python.org/issue16238

#16237: bdist_rpm SPEC files created with distutils may be distro spec
http://bugs.python.org/issue16237

#16235: Add python-config.sh for use during cross compilation.
http://bugs.python.org/issue16235



Most recent 15 issues waiting for review (15)
=============================================

#16286: Optimize a==b and a!=b for bytes and str
http://bugs.python.org/issue16286

#16284: concurrent.futures ThreadPoolExecutor keeps unnecessary refere
http://bugs.python.org/issue16284

#16262: srcdir != builddir builds fail, if hg is not installed
http://bugs.python.org/issue16262

#16245: Update html.entities.html5 dictionary and parseentities.py
http://bugs.python.org/issue16245

#16240: Document a way to escape metacharacters in glob/fnmatch
http://bugs.python.org/issue16240

#16236: Doc/Makefile should have $PYTHON=python2
http://bugs.python.org/issue16236

#16235: Add python-config.sh for use during cross compilation.
http://bugs.python.org/issue16235

#16230: select.select crashes on resized lists
http://bugs.python.org/issue16230

#16228: JSON crashes during encoding resized lists
http://bugs.python.org/issue16228

#16224: tokenize.untokenize() misbehaves when moved to "compatiblity m
http://bugs.python.org/issue16224

#16221: tokenize.untokenize() "compat" mode misses the encoding when u
http://bugs.python.org/issue16221

#16220: wsgiref does not call close() on iterable response
http://bugs.python.org/issue16220

#16216: Arithmetic operations with NULL
http://bugs.python.org/issue16216

#16215: Possible double memory free in str.replace
http://bugs.python.org/issue16215

#16202: sys.path[0] security issues
http://bugs.python.org/issue16202



Top 10 most discussed issues (10)
=================================

#16212: mmap() dumps core upon resizing the underlying file
http://bugs.python.org/issue16212  19 msgs

#16203: Proposal: add re.fullmatch() method
http://bugs.python.org/issue16203  14 msgs

#15745: Numerous utime ns tests fail on FreeBSD w/ ZFS (update:	and Ne
http://bugs.python.org/issue15745  13 msgs

#16202: sys.path[0] security issues
http://bugs.python.org/issue16202  13 msgs

#8402: Add a function to escape metacharacters in glob/fnmatch
http://bugs.python.org/issue8402   9 msgs

#10836: urllib.request.urlretrieve calls URLError with 3 args
http://bugs.python.org/issue10836   9 msgs

#16220: wsgiref does not call close() on iterable response
http://bugs.python.org/issue16220   9 msgs

#13538: Improve doc for str(bytesobject)
http://bugs.python.org/issue13538   8 msgs

#16213: Expose private functions in marshal used by importlib
http://bugs.python.org/issue16213   8 msgs

#16248: Security bug in tkinter allows for untrusted, arbitrary code e
http://bugs.python.org/issue16248   8 msgs



Issues closed (45)
==================

#6074: .pyc files created readonly if .py file is readonly, python wo
http://bugs.python.org/issue6074  closed by ncoghlan

#9583: Document startup option/environment interaction
http://bugs.python.org/issue9583  closed by asvetlov

#10834: Python 2.7 x86 IDLE fails to run in Windows 7
http://bugs.python.org/issue10834  closed by serwy

#12321: documentation of ElementTree.find
http://bugs.python.org/issue12321  closed by eli.bendersky

#12686: argparse - document (and improve?) use of SUPPRESS with help=
http://bugs.python.org/issue12686  closed by bethard

#13440: Explain the "status quo wins a stalemate" principle in the dev
http://bugs.python.org/issue13440  closed by python-dev

#13843: Python doesn't compile anymore on our Solaris buildbot: undefi
http://bugs.python.org/issue13843  closed by trent

#14120: ARM Ubuntu 3.x buildbot failing test_dbm
http://bugs.python.org/issue14120  closed by nadeem.vawda

#14214: test_concurrent_futures hangs
http://bugs.python.org/issue14214  closed by nadeem.vawda

#14229: On KeyboardInterrupt, the exit code should mirror the signal n
http://bugs.python.org/issue14229  closed by nadeem.vawda

#14774: _sysconfigdata.py doesn't support multiple build configuration
http://bugs.python.org/issue14774  closed by trent

#15145: Faster *_find_max_char
http://bugs.python.org/issue15145  closed by serhiy.storchaka

#15200: Faster os.walk
http://bugs.python.org/issue15200  closed by serhiy.storchaka

#15378: Underported Tools/unicode/comparecodecs.py
http://bugs.python.org/issue15378  closed by pitrou

#15586: Provide some examples for usage of ElementTree methods/attribu
http://bugs.python.org/issue15586  closed by eli.bendersky

#15744: missing tests for {RawIO,BufferedIO,TextIO}.writelines
http://bugs.python.org/issue15744  closed by pitrou

#15811: ElementTree.write() raises TypeError when xml_declaration = Tr
http://bugs.python.org/issue15811  closed by eli.bendersky

#15833: most failures to write byte-compiled file no longer suppressed
http://bugs.python.org/issue15833  closed by trent

#15936: Add link from os.urandom to random.SystemRandom
http://bugs.python.org/issue15936  closed by asvetlov

#15958: bytes.join() should allow arbitrary buffer objects
http://bugs.python.org/issue15958  closed by pitrou

#16044: xml.etree.ElementTree.Element: append method iterator param is
http://bugs.python.org/issue16044  closed by eli.bendersky

#16103: sys.stdin.readline 'unsupported' (IDLE 2.7.3, Win7, pythonw)
http://bugs.python.org/issue16103  closed by terry.reedy

#16106: antigravity tests
http://bugs.python.org/issue16106  closed by ezio.melotti

#16160: subclassing types.SimpleNamespace does not work
http://bugs.python.org/issue16160  closed by python-dev

#16166: Add PY_BYTE_ORDER macro to get endianess of platform
http://bugs.python.org/issue16166  closed by christian.heimes

#16186: shlex bug?
http://bugs.python.org/issue16186  closed by serwy

#16206: dict() docs should display multiple signatures
http://bugs.python.org/issue16206  closed by chris.jerdonek

#16214: 2to3 does not remove exceptions import
http://bugs.python.org/issue16214  closed by brett.cannon

#16225: add to "looping techniques" tutorial a note about modifying se
http://bugs.python.org/issue16225  closed by chris.jerdonek

#16226: IDLE crashes on *File / Path browser*
http://bugs.python.org/issue16226  closed by ned.deily

#16227: Add link to 3.3 version in unicode howto
http://bugs.python.org/issue16227  closed by chris.jerdonek

#16244: TimedRotatingFileHandler forces "write" mode, should use "appe
http://bugs.python.org/issue16244  closed by python-dev

#16249: unicode name accepts a punctuation glyph
http://bugs.python.org/issue16249  closed by ezio.melotti

#16257: test_socket.test_create_connection tests for wrong errno on So
http://bugs.python.org/issue16257  closed by trent

#16260: change bare except in locale.py
http://bugs.python.org/issue16260  closed by ramchandra.apte

#16265: Code sample does not collapse correctly
http://bugs.python.org/issue16265  closed by asvetlov

#16266: Rephrase sentence in tutorial about universal newlines
http://bugs.python.org/issue16266  closed by chris.jerdonek

#16269: multiprocessing.Queue uses select()
http://bugs.python.org/issue16269  closed by giampaolo.rodola

#16270: urllib hangs when closing connection
http://bugs.python.org/issue16270  closed by giampaolo.rodola

#16274: test_asyncore failing on Solaris 10 2.7 (nitrogen/s10)
http://bugs.python.org/issue16274  closed by trent

#16276: OrderedDict constructor do not keep items order
http://bugs.python.org/issue16276  closed by ezio.melotti

#16277: Objects/longobject.c not compiled without long long
http://bugs.python.org/issue16277  closed by serhiy.storchaka

#16279: namedtuple should compare equality with field names taken into
http://bugs.python.org/issue16279  closed by ncoghlan

#16280: Inconsistency with PyLong_FromVoidPtr() and PyLong_AsVoidPtr()
http://bugs.python.org/issue16280  closed by mark.dickinson

#16282: The Python Tutorial the previous link needs updating
http://bugs.python.org/issue16282  closed by chris.jerdonek

From fijall at gmail.com  Fri Oct 19 18:23:06 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 19 Oct 2012 18:23:06 +0200
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
	<CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
Message-ID: <CAK5idxRH2ri238Zon_=vY5qRA0j6ggJxWEvJ_DMaw+uJnsK-hg@mail.gmail.com>

On Fri, Oct 19, 2012 at 5:56 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2012/10/19 Tres Seaver <tseaver at palladion.com>:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 10/19/2012 11:26 AM, Benjamin Peterson wrote:
>>> 2012/10/19 Antonio Cuni <anto.cuni at gmail.com>:
>>>> indeed, you are right. So I suppose that in pypy we could just relax
>>>> the check in cmath and be happy. Is there any chance that this will
>>>> be changed in 2.7 and/or 3.x?
>>>
>>> Certainly 3.x, but not 2.7.
>>
>> Why not 2.7?  It is a perfectly-backward-compatible change:  no
>> currenly-working code could possibly break if cmath's restriction was
>> relaxed.
>
>
> It's a new feature. Also, it's possible that someone is relying on it
> throwing for non-complex values.
>

Nick just said it's a bug that cmath type checks are too strict.

From dickinsm at gmail.com  Fri Oct 19 18:31:07 2012
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 19 Oct 2012 17:31:07 +0100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
Message-ID: <CAAu3qLVLN+WpfRLC_DiNUgbRagM227VP5OeUffcoBzfdXj8rMQ@mail.gmail.com>

On Fri, Oct 19, 2012 at 4:26 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2012/10/19 Antonio Cuni <anto.cuni at gmail.com>:
>> indeed, you are right. So I suppose that in pypy we could just relax the check
>> in cmath and be happy. Is there any chance that this will be changed in 2.7
>> and/or 3.x?
>
> Certainly 3.x, but not 2.7.

+1 for relaxing the check in 3.x.

The cmath code uses "PyArg_ParseTuple(args, "D", ...) for this;
perhaps it's the "D" format for PyArg_ParseTuple that should be
relaxed.  It seems more than reasonable to allow floats wherever a
complex number is expected.

Mark

From christian at python.org  Fri Oct 19 18:31:31 2012
From: christian at python.org (Christian Heimes)
Date: Fri, 19 Oct 2012 18:31:31 +0200
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAK5idxRH2ri238Zon_=vY5qRA0j6ggJxWEvJ_DMaw+uJnsK-hg@mail.gmail.com>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
	<CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
	<CAK5idxRH2ri238Zon_=vY5qRA0j6ggJxWEvJ_DMaw+uJnsK-hg@mail.gmail.com>
Message-ID: <50818063.5060104@python.org>

Am 19.10.2012 18:23, schrieb Maciej Fijalkowski:
> Nick just said it's a bug that cmath type checks are too strict.

I'm not so sure about that. cmath just uses PyArg_ParseTuple with "D"
which itself relies upon PyComplex_AsCComplex().

D (complex) [Py_complex]
    Convert a Python complex number to a C Py_complex structure.

In order to fix the bug the code in PyComplex_AsCComplex() must be
altered to support float as return type from __complex__(). That's a
major change.

Christian

From dickinsm at gmail.com  Fri Oct 19 18:41:25 2012
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 19 Oct 2012 17:41:25 +0100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <50818063.5060104@python.org>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
	<CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
	<CAK5idxRH2ri238Zon_=vY5qRA0j6ggJxWEvJ_DMaw+uJnsK-hg@mail.gmail.com>
	<50818063.5060104@python.org>
Message-ID: <CAAu3qLX1YXPhudUc64dWhTFqinJEm7bmw8EhQs=1BB6UXwqijA@mail.gmail.com>

On Fri, Oct 19, 2012 at 5:31 PM, Christian Heimes <christian at python.org> wrote:
> In order to fix the bug the code in PyComplex_AsCComplex() must be
> altered to support float as return type from __complex__(). That's a
> major change.

Agreed that this doesn't seem appropriate for bugfix releases.

We might also want to consider having PyComplex_AsCComplex check for
__float__, again for consistency with the 'complex' constructor:

>>> class A(object):
...     def __float__(self):
...         return 42.0
...
>>> a = A()
>>> float(a)
42.0
>>> complex(a)
(42+0j)

Mark

From tseaver at palladion.com  Fri Oct 19 18:44:23 2012
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 19 Oct 2012 12:44:23 -0400
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
	<CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
Message-ID: <k5s01o$374$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/19/2012 11:56 AM, Benjamin Peterson wrote:
> 2012/10/19 Tres Seaver <tseaver at palladion.com>:
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>> 
>> On 10/19/2012 11:26 AM, Benjamin Peterson wrote:
>>> 2012/10/19 Antonio Cuni <anto.cuni at gmail.com>:
>>>> indeed, you are right. So I suppose that in pypy we could just
>>>> relax the check in cmath and be happy. Is there any chance that
>>>> this will be changed in 2.7 and/or 3.x?
>>> 
>>> Certainly 3.x, but not 2.7.
>> 
>> Why not 2.7?  It is a perfectly-backward-compatible change:  no 
>> currenly-working code could possibly break if cmath's restriction
>> was relaxed.
> 
> It's a new feature.

That is an assertion;  I can dqually assert that the current over-strict
typechecking is a bug, because it doesn't conform to the semandics of
'comples_new'.

> Also, it's possible that someone is relying on it throwing for
> non-complex values.

No already working, non-contrived code would break, becuase float is
perfectly-promotable to complex.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlCBg2cACgkQ+gerLs4ltQ5SMACfbhAMwQCwkZi1dF4WGL9uZoeR
wv4AoII2FVW8TPchCcmsh3llo7QPxroW
=gqkG
-----END PGP SIGNATURE-----


From benjamin at python.org  Fri Oct 19 18:48:35 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 19 Oct 2012 12:48:35 -0400
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <k5s01o$374$1@ger.gmane.org>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
	<CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
	<k5s01o$374$1@ger.gmane.org>
Message-ID: <CAPZV6o80YCdHDS+kCnFh-NMrGJNV=6ztRnu-PVy3nU-LH8N=MQ@mail.gmail.com>

2012/10/19 Tres Seaver <tseaver at palladion.com>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 10/19/2012 11:56 AM, Benjamin Peterson wrote:
>> 2012/10/19 Tres Seaver <tseaver at palladion.com>:
>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>>
>>> On 10/19/2012 11:26 AM, Benjamin Peterson wrote:
>>>> 2012/10/19 Antonio Cuni <anto.cuni at gmail.com>:
>>>>> indeed, you are right. So I suppose that in pypy we could just
>>>>> relax the check in cmath and be happy. Is there any chance that
>>>>> this will be changed in 2.7 and/or 3.x?
>>>>
>>>> Certainly 3.x, but not 2.7.
>>>
>>> Why not 2.7?  It is a perfectly-backward-compatible change:  no
>>> currenly-working code could possibly break if cmath's restriction
>>> was relaxed.
>>
>> It's a new feature.
>
> That is an assertion;  I can dqually assert that the current over-strict
> typechecking is a bug, because it doesn't conform to the semandics of
> 'comples_new'.

Nobody claimed it did conform to semantics of complex_new.

>
>> Also, it's possible that someone is relying on it throwing for
>> non-complex values.
>
> No already working, non-contrived code would break, becuase float is
> perfectly-promotable to complex.

I'm not saying the code which would break is good, I'm just saying it
shouldn't be broken in bugfix releases.


-- 
Regards,
Benjamin

From brett at python.org  Fri Oct 19 20:22:33 2012
From: brett at python.org (Brett Cannon)
Date: Fri, 19 Oct 2012 14:22:33 -0400
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAPZV6o80YCdHDS+kCnFh-NMrGJNV=6ztRnu-PVy3nU-LH8N=MQ@mail.gmail.com>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
	<CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
	<k5s01o$374$1@ger.gmane.org>
	<CAPZV6o80YCdHDS+kCnFh-NMrGJNV=6ztRnu-PVy3nU-LH8N=MQ@mail.gmail.com>
Message-ID: <CAP1=2W4VCATgXxdvHrVt-G7rjuaFAbeMRQEc2OUF80dq9p_bhQ@mail.gmail.com>

On Fri, Oct 19, 2012 at 12:48 PM, Benjamin Peterson <benjamin at python.org>wrote:

> 2012/10/19 Tres Seaver <tseaver at palladion.com>:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > On 10/19/2012 11:56 AM, Benjamin Peterson wrote:
> >> 2012/10/19 Tres Seaver <tseaver at palladion.com>:
> >>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
> >>>
> >>> On 10/19/2012 11:26 AM, Benjamin Peterson wrote:
> >>>> 2012/10/19 Antonio Cuni <anto.cuni at gmail.com>:
> >>>>> indeed, you are right. So I suppose that in pypy we could just
> >>>>> relax the check in cmath and be happy. Is there any chance that
> >>>>> this will be changed in 2.7 and/or 3.x?
> >>>>
> >>>> Certainly 3.x, but not 2.7.
> >>>
> >>> Why not 2.7?  It is a perfectly-backward-compatible change:  no
> >>> currenly-working code could possibly break if cmath's restriction
> >>> was relaxed.
> >>
> >> It's a new feature.
> >
> > That is an assertion;  I can dqually assert that the current over-strict
> > typechecking is a bug, because it doesn't conform to the semandics of
> > 'comples_new'.
>
> Nobody claimed it did conform to semantics of complex_new.
>

This also leads to a voluntary dovetailing of what would be accepted
between bugfix releases. Anyone remember True/False coming into existence
between bugfix releases? Wasn't that fun?

And in case someone didn't pick up on the sarcasm, it wasn't fun and it's
why we don't widen acceptability of things between bugfix releases unless
it is actually breaking code as-is (which this isn't).


>
> >
> >> Also, it's possible that someone is relying on it throwing for
> >> non-complex values.
> >
> > No already working, non-contrived code would break, becuase float is
> > perfectly-promotable to complex.
>
> I'm not saying the code which would break is good, I'm just saying it
> shouldn't be broken in bugfix releases
>

Agreed.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121019/43519187/attachment.html>

From g.brandl at gmx.net  Fri Oct 19 21:46:43 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 19 Oct 2012 21:46:43 +0200
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <k5s01o$374$1@ger.gmane.org>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
	<CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
	<k5s01o$374$1@ger.gmane.org>
Message-ID: <k5samu$28b$1@ger.gmane.org>

On 10/19/2012 06:44 PM, Tres Seaver wrote:
> On 10/19/2012 11:56 AM, Benjamin Peterson wrote:
>> 2012/10/19 Tres Seaver <tseaver at palladion.com>:
>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>>
>>> On 10/19/2012 11:26 AM, Benjamin Peterson wrote:
>>>> 2012/10/19 Antonio Cuni <anto.cuni at gmail.com>:
>>>>> indeed, you are right. So I suppose that in pypy we could just
>>>>> relax the check in cmath and be happy. Is there any chance that
>>>>> this will be changed in 2.7 and/or 3.x?
>>>>
>>>> Certainly 3.x, but not 2.7.
>>>
>>> Why not 2.7?  It is a perfectly-backward-compatible change:  no
>>> currenly-working code could possibly break if cmath's restriction
>>> was relaxed.
> 
>> It's a new feature.
> 
> That is an assertion;  I can dqually assert that the current over-strict
> typechecking is a bug, because it doesn't conform to the semandics of
> 'comples_new'.

Maybe, but an assertion made by the 2.7 release manager tends to be
the stronger one :)

Georg


From greg.ewing at canterbury.ac.nz  Sat Oct 20 01:35:13 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 20 Oct 2012 12:35:13 +1300
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <508150D8.3060108@gmail.com>
References: <508150D8.3060108@gmail.com>
Message-ID: <5081E3B1.8050707@canterbury.ac.nz>

Antonio Cuni wrote:
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: __complex__ should return a complex object
> 
> i.e., the complex constructor does not check that __complex__ returns an
> actual complex, while the cmath functions do.

Looks to me like cmath is being excessively finicky
here. Why shouldn't a float be usable in *any* context
expecting a complex?

-- 
Greg

From dholth at gmail.com  Sat Oct 20 02:07:14 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 19 Oct 2012 20:07:14 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>

Tweaked at http://hg.python.org/peps/rev/75474fb879d3

On Thu, Oct 18, 2012 at 10:55 PM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
> Executive summary:
>
> You probably should include a full ABNF grammar....

The legacy PKG-INFO does not have email parse-ability. How about an
example parser implementation instead, and drop email.parser.Parser()
support.  It will require a decent amount of additional editing.

> This isn't RFC 822 unfolding at all.  An RFC 822 "reader" will simply
> remove the CRLF and optionally "canonicalize" the spaces (the latter
> is not allowed by RFC 822, but sometimes it's observed).  This implies
> that if you use an RFC 822 reader, you need to replace instances of the
> regexp r"\s+\|" with a newline.  (If you have a conforming reader, you
> can use the regexp r"\s{7}\|" instead.)  And of course you have to
> RFC-2047-encode non-ASCII in an RFC-822 field.

There is less RFC822 in the document now.

I can see that compatibility with email.parser.Parser() must have been
a goal of the previous Metadata 1.2 spec. That is why it does the
7-spaces-plus-| trick. Otherwise two newlines would end the parsing
(or the rest of the document would be the message body).

The email parser is significantly more permissive than the RFC.

Author-email now mentions RFC 5322.

>  > License (optional)
>  > ::::::::::::::::::
>  >
>  > Text indicating the license covering the distribution where the license
>  > is not a selection from the "License" Trove classifiers. See
>  > "Classifier" below.  This field may also be used to specify a
>  > particular version of a licencse which is named via the ``Classifier``
>                                 A
> typo----------------------------+
>
>  > field, or to indicate a variation or exception to such a license.
>
> This won't do as is.  It doesn't exclude the possibility of including
> a complete license, and if that is intentional, this field needs to be
> in the same format as "Distribution".  Licenses are complex documents,
> needing at least some of the power of something like ReST.  You may as
> well give them all of it.

Typo fixed. The ability to put the full license in a separate file
exists, but would be part of the .dist-info spec. These files are
parsed at runtime and it's obnoxious to have long descriptions and
licenses in there.

>  > Project-URL (multiple-use)
>  > Provides-Extra (multiple use)
>
> Hyphen or no hyphen?  Consistency is good.

No hyphen.

Switched to using must as in RFC MUST for extension fields
ExtensionName/TagName:

From barry at python.org  Sat Oct 20 03:43:15 2012
From: barry at python.org (Barry Warsaw)
Date: Fri, 19 Oct 2012 21:43:15 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
Message-ID: <20121019214315.60ac6faf@resist.wooz.org>

On Oct 19, 2012, at 08:07 PM, Daniel Holth wrote:

>The email parser is significantly more permissive than the RFC.

Don't forget that the email package now supports policies (experimentally), so
it may be possible to tweak a policy setting to fit the bill.  Anyway, it
might at least be interesting to try.

-Barry

From ncoghlan at gmail.com  Sat Oct 20 04:04:44 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 20 Oct 2012 12:04:44 +1000
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAK5idxRH2ri238Zon_=vY5qRA0j6ggJxWEvJ_DMaw+uJnsK-hg@mail.gmail.com>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
	<CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
	<CAK5idxRH2ri238Zon_=vY5qRA0j6ggJxWEvJ_DMaw+uJnsK-hg@mail.gmail.com>
Message-ID: <CADiSq7cbDvw0Sqb7UK5H0suhD81BeS6fHuDQ-QJvP4_4wU=hNQ@mail.gmail.com>

On Sat, Oct 20, 2012 at 2:23 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Fri, Oct 19, 2012 at 5:56 PM, Benjamin Peterson <benjamin at python.org> wrote:
>> It's a new feature. Also, it's possible that someone is relying on it
>> throwing for non-complex values.
>>
>
> Nick just said it's a bug that cmath type checks are too strict.

It wouldn't be the first time PyPy has picked up on something that is
technically a bug in CPython that we nevertheless deemed too risky to
fix in CPython maintenance releases (the other main one I'm aware of
is the fact we sometimes get the + and * operand precedence wrong for
sequence types implemented in C).

I don't see the same level of risk here that Benjamin does (it's not
like the conversion process would be *returning* a float value, just
taking the "+ 0j" as implied when handed a floating point value by
__complex__, the same way the constructor does), but it's still the
RMs call as to what's acceptable for a maintenance release.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From stephen at xemacs.org  Sat Oct 20 06:36:01 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 20 Oct 2012 13:36:01 +0900
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAK5idxRH2ri238Zon_=vY5qRA0j6ggJxWEvJ_DMaw+uJnsK-hg@mail.gmail.com>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
	<CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
	<CAK5idxRH2ri238Zon_=vY5qRA0j6ggJxWEvJ_DMaw+uJnsK-hg@mail.gmail.com>
Message-ID: <87bofx69qm.fsf@uwakimon.sk.tsukuba.ac.jp>

Maciej Fijalkowski writes:

 > Nick just said it's a bug that cmath type checks are too strict.

It's a design bug, yes.  The question is, does it conform to
documented behavior?  If so, it's not an implementation bug that
should be fixed in 2.7.

From stephen at xemacs.org  Sat Oct 20 06:49:08 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 20 Oct 2012 13:49:08 +0900
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
Message-ID: <87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>

Daniel Holth writes:
 > Tweaked at http://hg.python.org/peps/rev/75474fb879d3

I'll take a look later; I have some other commitments now I should do
first.

 > On Thu, Oct 18, 2012 at 10:55 PM, Stephen J. Turnbull
 > <stephen at xemacs.org> wrote:
 > > Executive summary:
 > >
 > > You probably should include a full ABNF grammar....
 > 
 > The legacy PKG-INFO does not have email parse-ability. How about an
 > example parser implementation instead, and drop email.parser.Parser()
 > support.  It will require a decent amount of additional editing.

I don't think that the example implementation matters.  A strict RFC
822 email header parser that treats these as "unstructured fields"
will DTRT.  

The point about the ABNF grammar is based on the email community's
long experience with email standards (733, 822, 2822, 5322, ...).
It's always easier to relax definitions later when they prove too
strict rather than tighten them up.

 > There is less RFC822 in the document now.

Thank you.

 > Typo fixed. The ability to put the full license in a separate file
 > exists, but would be part of the .dist-info spec. These files are
 > parsed at runtime and it's obnoxious to have long descriptions and
 > licenses in there.

Good.

 > Switched to using must as in RFC MUST for extension fields
 > ExtensionName/TagName:

Good.

Thanks for your excellent responsiveness.


From greg.ewing at canterbury.ac.nz  Sat Oct 20 07:06:42 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 20 Oct 2012 18:06:42 +1300
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <87bofx69qm.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
	<CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
	<CAK5idxRH2ri238Zon_=vY5qRA0j6ggJxWEvJ_DMaw+uJnsK-hg@mail.gmail.com>
	<87bofx69qm.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <50823162.5020203@canterbury.ac.nz>

Stephen J. Turnbull wrote:
> It's a design bug, yes.  The question is, does it conform to
> documented behavior?

The 2.7 docs say this about __complex__:

    Called to implement the built-in function complex() ...
    Should return a value of the appropriate type.

So the question is whether float is an "appropriate type" when
you're expecting a complex.

-- 
Greg

From jeanpierreda at gmail.com  Sat Oct 20 07:09:06 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sat, 20 Oct 2012 01:09:06 -0400
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
Message-ID: <CABicbJJFDq8MrHr3v9-xNhFW33EJVWrc5+XdZuwEAUfDi6=8Dg@mail.gmail.com>

On Fri, Oct 19, 2012 at 10:13 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Fri, Oct 19, 2012 at 11:08 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
>> Is that the real intended behavior?
>
> Given the way complex numbers interact with floats generally,
> returning a complex number with no imaginary component as a floating
> point value seems legitimate and the checks in cmath overly strict.
> Otherwise you would get redundancy like:
>
>     def __complex__(self):
>         return complex(value)
>
> or
>
>     def __complex__(self):
>         return value + 0j

No you wouldn't:

    def __float__(self):
        return value

-- Devin

From agriff at tin.it  Sat Oct 20 10:24:36 2012
From: agriff at tin.it (Andrea Griffini)
Date: Sat, 20 Oct 2012 10:24:36 +0200
Subject: [Python-Dev] Improved evaluator added to ast module
In-Reply-To: <k5p801$3bf$1@ger.gmane.org>
References: <loom.20121011T175200-792@post.gmane.org>
	<CAPZV6o9H4NHT3v_n2uTLaf-aaes1LkAN161++j9MiyhmH9Z7EQ@mail.gmail.com>
	<CAG8k2+7H=hAeKpZ_BmN7hRx0qQjDQ=AQEuest2viRRteA3ua=g@mail.gmail.com>
	<loom.20121011T192849-311@post.gmane.org>
	<CAG8k2+66ND=Ucmt=9xfiYDUhQjAFSsXQC-xvE4VAb9L9h+RPgA@mail.gmail.com>
	<k5p801$3bf$1@ger.gmane.org>
Message-ID: <CALy7ODvNz+ALUEHAfWo1caB-+5xqwqcG=e4yUs+2K9PHcrw3Sw@mail.gmail.com>

On Thu, Oct 18, 2012 at 5:41 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> On 10/18/2012 03:16 PM, Daniel Holth wrote:
>> On Thu, Oct 11, 2012 at 1:36 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>>> Daniel Holth <dholth <at> gmail.com> writes:
>>>
>>>> How does this compare to the markerlib approach? In markerlib you just
>>>> make sure all the AST nodes are in a set of allowed nodes, currently
>>>> (Compare, BoolOp, Attribute, Name, Load, Str, cmpop, boolop), and then
>>>> use the normal eval(). Is one way more secure / fast / flexible than
>>>> the other?
>>>
>>> I don't think performance is an issue, and the markerlib approach seems just
>>> as reasonable as the one I've taken, except that it calls eval(), whereas my
>>> approach doesn't. It boils down to what should be allowed in expressions, and
>>> what shouldn't be.

I'm not sure if this is pertinent to the safe eval discussion, but
currently it's possible to make python crash with a segfault even by
just *parsing* an expression.

See http://bugs.python.org/issue5765

Andrea

From dickinsm at gmail.com  Sat Oct 20 11:41:24 2012
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sat, 20 Oct 2012 10:41:24 +0100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
Message-ID: <CAAu3qLUZfQSShe22KZj6Jfyk+tkAT7nPPY2L0BQtz0PCpRr5rg@mail.gmail.com>

On Fri, Oct 19, 2012 at 3:13 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Given the way complex numbers interact with floats generally,
> returning a complex number with no imaginary component as a floating
> point value seems legitimate and the checks in cmath overly strict.
> Otherwise you would get redundancy like:
>
>     def __complex__(self):
>         return complex(value)
>
> or
>
>     def __complex__(self):
>         return value + 0j

I've opened bugs.python.org/issue16290 to track this.

-- 
Mark

From dholth at gmail.com  Sat Oct 20 13:34:08 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sat, 20 Oct 2012 07:34:08 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>

The troublesome Description: parses fine -- as long as there is
anything but a \r\n - for example "\r\n " - on successive blank lines.
This tends to happen already.

Another solution comes to mind. Put the description in the payload.
The description can have any form, and the installer can stop parsing
at the first linesep pair because it only inspects Name, Version, and
requirements. A little bit of a pain because you might have to also
allow the description in the headers or rewrite it on install.


Metadata-Version: 1.3
Name: package
Version: 0.13.0
Summary: A package.
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.3


Wheel
=====

A built-package format for Python.

A wheel is a ZIP-format archive with a specially formatted filename
and the .whl extension. It is designed to contain all the files for a
PEP 376 compatible install in a way that is very close to the on-disk
format. Many packages will be properly installed with only the "Unpack"
step (simply extracting the file onto sys.path), and the unpacked archive
preserves enough information to "Spread" (copy data and scripts to their
final locations) at any later time.

From g.brandl at gmx.net  Sat Oct 20 18:00:51 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 20 Oct 2012 18:00:51 +0200
Subject: [Python-Dev] Improved evaluator added to ast module
In-Reply-To: <CALy7ODvNz+ALUEHAfWo1caB-+5xqwqcG=e4yUs+2K9PHcrw3Sw@mail.gmail.com>
References: <loom.20121011T175200-792@post.gmane.org>
	<CAPZV6o9H4NHT3v_n2uTLaf-aaes1LkAN161++j9MiyhmH9Z7EQ@mail.gmail.com>
	<CAG8k2+7H=hAeKpZ_BmN7hRx0qQjDQ=AQEuest2viRRteA3ua=g@mail.gmail.com>
	<loom.20121011T192849-311@post.gmane.org>
	<CAG8k2+66ND=Ucmt=9xfiYDUhQjAFSsXQC-xvE4VAb9L9h+RPgA@mail.gmail.com>
	<k5p801$3bf$1@ger.gmane.org>
	<CALy7ODvNz+ALUEHAfWo1caB-+5xqwqcG=e4yUs+2K9PHcrw3Sw@mail.gmail.com>
Message-ID: <k5uhre$vsu$1@ger.gmane.org>

On 10/20/2012 10:24 AM, Andrea Griffini wrote:
> On Thu, Oct 18, 2012 at 5:41 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>> On 10/18/2012 03:16 PM, Daniel Holth wrote:
>>> On Thu, Oct 11, 2012 at 1:36 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>>>> Daniel Holth <dholth <at> gmail.com> writes:
>>>>
>>>>> How does this compare to the markerlib approach? In markerlib you just
>>>>> make sure all the AST nodes are in a set of allowed nodes, currently
>>>>> (Compare, BoolOp, Attribute, Name, Load, Str, cmpop, boolop), and then
>>>>> use the normal eval(). Is one way more secure / fast / flexible than
>>>>> the other?
>>>>
>>>> I don't think performance is an issue, and the markerlib approach seems just
>>>> as reasonable as the one I've taken, except that it calls eval(), whereas my
>>>> approach doesn't. It boils down to what should be allowed in expressions, and
>>>> what shouldn't be.
> 
> I'm not sure if this is pertinent to the safe eval discussion, but
> currently it's possible to make python crash with a segfault even by
> just *parsing* an expression.
> 
> See http://bugs.python.org/issue5765

There's a big difference between being able to pass anything to an eval()
invocation, and having to exploit a segfault due to a stack overflow.

Even if your eval() argument overflows the stack, it's still much safer
than if you pass '__import__("os").unlink(...)' :)

Georg


From tseaver at palladion.com  Sat Oct 20 21:28:28 2012
From: tseaver at palladion.com (Tres Seaver)
Date: Sat, 20 Oct 2012 15:28:28 -0400
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <5081E3B1.8050707@canterbury.ac.nz>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
Message-ID: <k5uu1g$m9$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/19/2012 07:35 PM, Greg Ewing wrote:
> Antonio Cuni wrote:
>> Traceback (most recent call last): File "<stdin>", line 1, in
>> <module> TypeError: __complex__ should return a complex object
>> 
>> i.e., the complex constructor does not check that __complex__
>> returns an actual complex, while the cmath functions do.
> 
> Looks to me like cmath is being excessively finicky here. Why
> shouldn't a float be usable in *any* context expecting a complex?

Exactly:  float is perfectly Liskov-substituable for complex;  only
applications which do explicit type sniffing can tell the difference,
which makes the sniffing bogus.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlCC+1wACgkQ+gerLs4ltQ7S1QCfUjvUsmMiHuW8DDXue0HPzvXE
Qv4Anissl8zNnx8KZyJQfcDxWlddTXGA
=6KUH
-----END PGP SIGNATURE-----


From andrew.robert.moffat at gmail.com  Sat Oct 20 23:08:37 2012
From: andrew.robert.moffat at gmail.com (Andrew Moffat)
Date: Sat, 20 Oct 2012 16:08:37 -0500
Subject: [Python-Dev] Interest in seeing sh.py in the stdlib
Message-ID: <CADijH1VDUm8yXcBJKTnJbq3H8yp9ogSST9NWDGDYK4_DJjzGkQ@mail.gmail.com>

Hi,

I'm the author of sh.py, a subprocess module rewrite for Linux and OSX.  It
serves as a powerful and intuitive interface to launching subprocesses
http://amoffat.github.com/sh/.  It has been maintained on github
https://github.com/amoffat/sh for about 10 months and currently has about
25k installs, according to pythonpackages.com.

Andy Grover maintains the Fedora rpm for sh.py
http://arm.koji.fedoraproject.org/koji/buildinfo?buildID=94247  and Nick
Moffit has submitted an older version of sh.py (which was called pbs) to be
included in Debian distros
http://pkgs.org/debian-wheezy/debian-main-i386/python-pbs_0.95-1_all.deb.html

I'm interested in making sh.py more accessible to help bring Python forward
in the area of shell scripting, so I'm interested in seeing if sh would be
suitable for the standard library.  Is there any other interest in
something like this?

Thanks
Andrew Moffat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121020/f7535032/attachment.html>

From benjamin at python.org  Sat Oct 20 23:27:58 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 20 Oct 2012 17:27:58 -0400
Subject: [Python-Dev] Interest in seeing sh.py in the stdlib
In-Reply-To: <CADijH1VDUm8yXcBJKTnJbq3H8yp9ogSST9NWDGDYK4_DJjzGkQ@mail.gmail.com>
References: <CADijH1VDUm8yXcBJKTnJbq3H8yp9ogSST9NWDGDYK4_DJjzGkQ@mail.gmail.com>
Message-ID: <CAPZV6o_x6wnxht0CQjr=sY6m_wASgXzpaYnkQAGxpTPn4XEJ5g@mail.gmail.com>

2012/10/20 Andrew Moffat <andrew.robert.moffat at gmail.com>:
> Hi,
>
> I'm the author of sh.py, a subprocess module rewrite for Linux and OSX.  It
> serves as a powerful and intuitive interface to launching subprocesses
> http://amoffat.github.com/sh/.  It has been maintained on github
> https://github.com/amoffat/sh for about 10 months and currently has about
> 25k installs, according to pythonpackages.com.
>
> Andy Grover maintains the Fedora rpm for sh.py
> http://arm.koji.fedoraproject.org/koji/buildinfo?buildID=94247  and Nick
> Moffit has submitted an older version of sh.py (which was called pbs) to be
> included in Debian distros
> http://pkgs.org/debian-wheezy/debian-main-i386/python-pbs_0.95-1_all.deb.html
>
> I'm interested in making sh.py more accessible to help bring Python forward
> in the area of shell scripting, so I'm interested in seeing if sh would be
> suitable for the standard library.  Is there any other interest in something
> like this?

You should try the python-ideas list.


-- 
Regards,
Benjamin

From guido at python.org  Sun Oct 21 01:48:05 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 20 Oct 2012 16:48:05 -0700
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <k5uu1g$m9$1@ger.gmane.org>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org>
Message-ID: <CAP7+vJ+yHf1+iwZA3sZ10Vanwda+gsbvCVQAaKE3_J+dH6Ngww@mail.gmail.com>

On Sat, Oct 20, 2012 at 12:28 PM, Tres Seaver <tseaver at palladion.com> wrote:
> Exactly:  float is perfectly Liskov-substituable for complex;  only
> applications which do explicit type sniffing can tell the difference,
> which makes the sniffing bogus.

You don't have to do explicit sniffing. You could also be catching
exceptions, e.g. ordering floats works, but ordering complex raises
TypeError.

All in all I'm still in favor of treating this as a new feature. It
would otherwise encourage people writing code "for Python 2.7" that
would actually break on older versions of 2.7. Yes, I'm aware that
every bugfix release makes *some* code work that was broken before.
But this is still different. "API X now supports argument type Y"
smells like a new feature to me, no matter what you substitute for X
and Y.

-- 
--Guido van Rossum (python.org/~guido)

From steve at pearwood.info  Sun Oct 21 02:04:54 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 21 Oct 2012 11:04:54 +1100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
Message-ID: <50833C26.3050605@pearwood.info>

On 20/10/12 01:13, Nick Coghlan wrote:
> On Fri, Oct 19, 2012 at 11:08 PM, Antonio Cuni<anto.cuni at gmail.com>  wrote:
>> Is that the real intended behavior?
>
> Given the way complex numbers interact with floats generally,
> returning a complex number with no imaginary component as a floating
> point value seems legitimate


Surely the intention is for __complex__ to return a complex number? That
is, that it is desirable to have the invariant:

isinstance(x.__complex__(), complex)

?

We expect that __int__ returns an int, and raise an exception if it
doesn't, even in the case that the value returned is numerically integral:

py> class X:
...     def __int__(self):
...             return 2.0
...
py> x = X()
py> int(x)
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: __int__ returned non-int (type float)


Conceptually, I see returning a float when you expect a complex as
equally dubious as returning an integral float when you expect an int.
I think there is something dirty about a __complex__ that returns a
non-complex. Why would you deliberately do such a thing? If a class
does so, that's surely indicative of a bug, so the earlier it gets
caught, the better.



> and the checks in cmath overly strict.
>
> Otherwise you would get redundancy like:
>
>      def __complex__(self):
>          return complex(value)
> or
>
>      def __complex__(self):
>          return value + 0j


For the record, I think Nick is referring to the fact that the complex
constructor will fall back on __float__ if __complex__ does not exist,
adding 0j to x.__float__.

I don't see this as a problem. So what if people write redundant code?
It still works. And when they learn better, they can write better code.



-- 
Steven

From solipsis at pitrou.net  Sun Oct 21 02:15:54 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 21 Oct 2012 02:15:54 +0200
Subject: [Python-Dev] return type of __complex__
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
Message-ID: <20121021021554.45d6027d@pitrou.net>

On Sat, 20 Oct 2012 00:13:51 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Fri, Oct 19, 2012 at 11:08 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> > Is that the real intended behavior?
> 
> Given the way complex numbers interact with floats generally,
> returning a complex number with no imaginary component as a floating
> point value seems legitimate and the checks in cmath overly strict.
> Otherwise you would get redundancy like:
> 
>     def __complex__(self):
>         return complex(value)
> 
> or
> 
>     def __complex__(self):
>         return value + 0j

The redundancy sounds like a non-issue to me, since you can implement
__float__ instead:

>>> class C:
...     def __float__(self): return 5.0
... 
>>> complex(C())
(5+0j)
>>> cmath.cos(C())
(0.28366218546322625+0j)

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From steve at pearwood.info  Sun Oct 21 03:53:10 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 21 Oct 2012 12:53:10 +1100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <k5uu1g$m9$1@ger.gmane.org>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org>
Message-ID: <50835586.3000602@pearwood.info>

On 21/10/12 06:28, Tres Seaver wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 10/19/2012 07:35 PM, Greg Ewing wrote:
>> Antonio Cuni wrote:
>>> Traceback (most recent call last): File "<stdin>", line 1, in
>>> <module>  TypeError: __complex__ should return a complex object
>>>
>>> i.e., the complex constructor does not check that __complex__
>>> returns an actual complex, while the cmath functions do.
>>
>> Looks to me like cmath is being excessively finicky here. Why
>> shouldn't a float be usable in *any* context expecting a complex?
>
> Exactly:  float is perfectly Liskov-substituable for complex;  only
> applications which do explicit type sniffing can tell the difference,
> which makes the sniffing bogus.


But float is not *numerically* substitutable for complex, which is why
type-sniffing is not bogus at all. If you have an application which
assumes numeric quantities represent real values, then you need to
distinguish between real-valued and complex-valued arithmetic, and
treating floats as implicitly complex is the wrong thing to do.

Since most applications are based on real-values, implicit promotion to
complex is in my opinion an anti-feature.

Python 2.x legitimately distinguished between floats and complex, e.g.:

py> (-100.0)**0.5
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
ValueError: negative number cannot be raised to a fractional power

If you wanted a complex result, you can explicitly ask for one:

py> (-100.0+0j)**0.5
(6.123031769111886e-16+10j)


I see that behaviour is changed in Python 3. Was this discussed before
being changed? I see a single sample docstring buried in PEP 3141 that:

"""a**b; should promote to float or complex when necessary."""

but I can't find any discussion of the consequences of this for the
majority of users who would be surprised by the unexpected appearance
of a "j" inside their numbers.

Nor is there any hint in the docs for pow and ** that they will promote
floats to complex:

http://docs.python.org/dev/library/functions.html#pow

http://docs.python.org/dev/library/stdtypes.html#numeric-types-int-float-complex



-- 
Steven

From guido at python.org  Sun Oct 21 03:57:51 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 20 Oct 2012 18:57:51 -0700
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <50835586.3000602@pearwood.info>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
Message-ID: <CAP7+vJK-MjnoaDR49btDCk2hnSLSNSG0zsWip6iOqOHUTNwGOg@mail.gmail.com>

You can check the .image attribute, which exists on floats too (and ints).

--Guido van Rossum (sent from Android phone)
On Oct 20, 2012 6:54 PM, "Steven D'Aprano" <steve at pearwood.info> wrote:

> On 21/10/12 06:28, Tres Seaver wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 10/19/2012 07:35 PM, Greg Ewing wrote:
>>
>>> Antonio Cuni wrote:
>>>
>>>> Traceback (most recent call last): File "<stdin>", line 1, in
>>>> <module>  TypeError: __complex__ should return a complex object
>>>>
>>>> i.e., the complex constructor does not check that __complex__
>>>> returns an actual complex, while the cmath functions do.
>>>>
>>>
>>> Looks to me like cmath is being excessively finicky here. Why
>>> shouldn't a float be usable in *any* context expecting a complex?
>>>
>>
>> Exactly:  float is perfectly Liskov-substituable for complex;  only
>> applications which do explicit type sniffing can tell the difference,
>> which makes the sniffing bogus.
>>
>
>
> But float is not *numerically* substitutable for complex, which is why
> type-sniffing is not bogus at all. If you have an application which
> assumes numeric quantities represent real values, then you need to
> distinguish between real-valued and complex-valued arithmetic, and
> treating floats as implicitly complex is the wrong thing to do.
>
> Since most applications are based on real-values, implicit promotion to
> complex is in my opinion an anti-feature.
>
> Python 2.x legitimately distinguished between floats and complex, e.g.:
>
> py> (-100.0)**0.5
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ValueError: negative number cannot be raised to a fractional power
>
> If you wanted a complex result, you can explicitly ask for one:
>
> py> (-100.0+0j)**0.5
> (6.123031769111886e-16+10j)
>
>
> I see that behaviour is changed in Python 3. Was this discussed before
> being changed? I see a single sample docstring buried in PEP 3141 that:
>
> """a**b; should promote to float or complex when necessary."""
>
> but I can't find any discussion of the consequences of this for the
> majority of users who would be surprised by the unexpected appearance
> of a "j" inside their numbers.
>
> Nor is there any hint in the docs for pow and ** that they will promote
> floats to complex:
>
> http://docs.python.org/dev/**library/functions.html#pow<http://docs.python.org/dev/library/functions.html#pow>
>
> http://docs.python.org/dev/**library/stdtypes.html#numeric-**
> types-int-float-complex<http://docs.python.org/dev/library/stdtypes.html#numeric-types-int-float-complex>
>
>
>
> --
> Steven
> ______________________________**_________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> guido%40python.org<http://mail.python.org/mailman/options/python-dev/guido%40python.org>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121020/4cfa4a0f/attachment.html>

From rosuav at gmail.com  Sun Oct 21 04:38:48 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 21 Oct 2012 13:38:48 +1100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <50835586.3000602@pearwood.info>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
Message-ID: <CAPTjJmqBw0J0vXh=XB26gVgNMZ_30=vWFcKj1WuXrN-PD9w0Jg@mail.gmail.com>

On Sun, Oct 21, 2012 at 12:53 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> Python 2.x legitimately distinguished between floats and complex, e.g.:
>
> py> (-100.0)**0.5
>
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ValueError: negative number cannot be raised to a fractional power
>
> If you wanted a complex result, you can explicitly ask for one:
>
> py> (-100.0+0j)**0.5
> (6.123031769111886e-16+10j)
>
>
> I see that behaviour is changed in Python 3.

Python 2 (future directives aside) also required you to explicitly ask
for floating point. That was also changed in Python 3. That doesn't
mean that this is necessarily the right thing to do, but it does have
precedent. The square root of a negative number is by nature a complex
number.

ChrisA

From rosuav at gmail.com  Sun Oct 21 04:40:02 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 21 Oct 2012 13:40:02 +1100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAPTjJmqBw0J0vXh=XB26gVgNMZ_30=vWFcKj1WuXrN-PD9w0Jg@mail.gmail.com>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CAPTjJmqBw0J0vXh=XB26gVgNMZ_30=vWFcKj1WuXrN-PD9w0Jg@mail.gmail.com>
Message-ID: <CAPTjJmoe2B5B44sB3y-Pj7mA1HVGUzLFCv6W_MyGVzZZ-8moiA@mail.gmail.com>

On Sun, Oct 21, 2012 at 1:38 PM, Chris Angelico <rosuav at gmail.com> wrote:
> Python 2 (future directives aside) also required you to explicitly ask
> for floating point. That was also changed in Python 3.

Er, should have said that I was referring to division here...
incomplete information. Anyway. Py3 says that int/int --> float is
acceptable, so float**float --> complex is equally plausible.

ChrisA

From dholth at gmail.com  Sun Oct 21 04:56:49 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sat, 20 Oct 2012 22:56:49 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
Message-ID: <CAG8k2+4cbAK-nsNqHRKs2_rqNPzFPyq1mD5Ris-XwY7bKSSMiA@mail.gmail.com>

I'm really happy with moving the troublesome Description: field to the
payload. Email Policy() is useful too.

http://hg.python.org/peps/rev/74868fe8ba17

Metadata 1.3 is a ...

+format with no maximum line length, followed by a blank line and an
+arbitrary payload.  It is parseable by the ``email`` module with an
+appropriate ``email.policy.Policy()``.

description is deprecated...

+Since Metadata 1.3 the recommended place for the description is in the
+payload section of the document, after the last header.  The description
+needs no special formatting when included in the payload.

includes a handy...

# Metadata 1.3 demo
from email.generator import Generator
from email import header
from email.parser import Parser
from email.policy import Compat32
from email.utils import _has_surrogates

class MetadataPolicy(Compat32):
    max_line_length = 0
    continuation_whitespace = '\t'

    def _sanitize_header(self, name, value):
        if not isinstance(value, str):
            return value
        if _has_surrogates(value):
            raise NotImplementedError()
        else:
            return value

    def _fold(self, name, value, sanitize):
        body = ((self.linesep+self.continuation_whitespace)
                .join(value.splitlines()))
        return ''.join((name, ': ', body, self.linesep))

if __name__ == "__main__":
    import sys
    import textwrap

    pkg_info = """\
Metadata-Version: 1.3
Name: package
Version: 0.1.0
Summary: A package.
Description: Description
    ===========


    A description of the package.

"""

    m = Parser(policy=MetadataPolicy()).parsestr(pkg_info)

    m['License'] = 'GPL'
    description = m['Description']
    description_lines = description.splitlines()
    m.set_payload(description_lines[0]
            + '\n'
            + textwrap.dedent('\n'.join(description_lines[1:]))
            + '\n')
    del m['Description']

    # Correct if sys.stdout.encoding == 'UTF-8':
    Generator(sys.stdout, maxheaderlen=0).flatten(m)

From ncoghlan at gmail.com  Sun Oct 21 04:57:31 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 21 Oct 2012 12:57:31 +1000
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <50835586.3000602@pearwood.info>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
Message-ID: <CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>

On Sun, Oct 21, 2012 at 11:53 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On 21/10/12 06:28, Tres Seaver wrote:
>>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 10/19/2012 07:35 PM, Greg Ewing wrote:
>>>
>>> Antonio Cuni wrote:
>>>>
>>>> Traceback (most recent call last): File "<stdin>", line 1, in
>>>> <module>  TypeError: __complex__ should return a complex object
>>>>
>>>> i.e., the complex constructor does not check that __complex__
>>>> returns an actual complex, while the cmath functions do.
>>>
>>>
>>> Looks to me like cmath is being excessively finicky here. Why
>>> shouldn't a float be usable in *any* context expecting a complex?
>>
>>
>> Exactly:  float is perfectly Liskov-substituable for complex;  only
>> applications which do explicit type sniffing can tell the difference,
>> which makes the sniffing bogus.
>
>
>
> But float is not *numerically* substitutable for complex, which is why
> type-sniffing is not bogus at all. If you have an application which
> assumes numeric quantities represent real values, then you need to
> distinguish between real-valued and complex-valued arithmetic, and
> treating floats as implicitly complex is the wrong thing to do.
>
> Since most applications are based on real-values, implicit promotion to
> complex is in my opinion an anti-feature.
>
> Python 2.x legitimately distinguished between floats and complex, e.g.:
>
> py> (-100.0)**0.5
>
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ValueError: negative number cannot be raised to a fractional power
>
> If you wanted a complex result, you can explicitly ask for one:
>
> py> (-100.0+0j)**0.5
> (6.123031769111886e-16+10j)
>
>
> I see that behaviour is changed in Python 3. Was this discussed before
> being changed? I see a single sample docstring buried in PEP 3141 that:
>
> """a**b; should promote to float or complex when necessary."""
>
> but I can't find any discussion of the consequences of this for the
> majority of users who would be surprised by the unexpected appearance
> of a "j" inside their numbers.

PEP 3141 is indeed the driver for these changes, and it's based on the
Python 3.x numeric tower consisting of strict supersets: Complex >
Real > Rational > Integral

If an operation at one level of the tower produces a result in one of
the larger supersets, then *that's what it will do* rather than
throwing TypeError. int / int promoting to float is one example, as is
raising a negative number to a fractional power promoting to complex.

The general philosophy is described in
http://www.python.org/dev/peps/pep-3141/#implementing-the-arithmetic-operations

It sounds like cmath (or, more specifically, the "D" conversion code)
missed out on the associated updates).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From jeanpierreda at gmail.com  Sun Oct 21 07:06:08 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sun, 21 Oct 2012 01:06:08 -0400
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>
Message-ID: <CABicbJKLubgr0BPDsr5j2J=bvqEsJMkQrLMiSA_3FBOwEgLC6g@mail.gmail.com>

On Sat, Oct 20, 2012 at 10:57 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> PEP 3141 is indeed the driver for these changes, and it's based on the
> Python 3.x numeric tower consisting of strict supersets: Complex >
> Real > Rational > Integral

Pedant mode: That numeric tower is wrong as it applies to Python -- we
have some rational types that can represent some numbers that can't be
represented by our complex and """real""" types.

    >>> int(float(10**100)) == 10**100
    False

(Also, floats aren't reals and no computer can store any number that
is not rational and we should stop pretending they can. (Complex
numbers get a free pass because "complex numbers with rational real
and imaginary parts" is not a memorable name for a type.))

*Pedant mode deactivated*

> If an operation at one level of the tower produces a result in one of
> the larger supersets, then *that's what it will do* rather than
> throwing TypeError. int / int promoting to float is one example, as is
> raising a negative number to a fractional power promoting to complex.
>
> The general philosophy is described in
> http://www.python.org/dev/peps/pep-3141/#implementing-the-arithmetic-operations
>
> It sounds like cmath (or, more specifically, the "D" conversion code)
> missed out on the associated updates).

No, no no. The return type of conversion methods has nothing to do
with the interoperability of types in arithmetic. If it did there
would've been a systematic purging of old behaviors in these
conversion functions, and there evidently hasn't been because float(),
cmath, and math all behave "wrong", and those are collectively fairly
hard to miss.

Since float(), cmath, and math all behave the same, I'd assert that
it's complex() (and int()) that is weird and unusual.

>>> class A:
...     def __complex__(self):
...             return 1
...     def __float__(self):
...             return 1
...
>>> complex(A())
(1+0j)
>>> float(A())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __float__ returned non-float (type int)
>>> cmath.sqrt(A())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __complex__ should return a complex object
>>> math.sqrt(A())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: nb_float should return float object
>>>

-- Devin

From greg.ewing at canterbury.ac.nz  Sun Oct 21 07:26:57 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 21 Oct 2012 18:26:57 +1300
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAP7+vJ+yHf1+iwZA3sZ10Vanwda+gsbvCVQAaKE3_J+dH6Ngww@mail.gmail.com>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org>
	<CAP7+vJ+yHf1+iwZA3sZ10Vanwda+gsbvCVQAaKE3_J+dH6Ngww@mail.gmail.com>
Message-ID: <508387A1.6050909@canterbury.ac.nz>

I think I've changed my mind on this, since it was pointed
out that if you're going to return a float instead of a
complex, you should really be implementing __float__, not
__complex__.

So I think it's fine to require __complex__ to return a
complex, and the docs should perhaps be clarified on that
point.

Also PyComplex_AsComplex() should perhaps enforce that.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sun Oct 21 08:05:40 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 21 Oct 2012 19:05:40 +1300
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAPTjJmqBw0J0vXh=XB26gVgNMZ_30=vWFcKj1WuXrN-PD9w0Jg@mail.gmail.com>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CAPTjJmqBw0J0vXh=XB26gVgNMZ_30=vWFcKj1WuXrN-PD9w0Jg@mail.gmail.com>
Message-ID: <508390B4.9090306@canterbury.ac.nz>

Chris Angelico wrote:
> Python 2 (future directives aside) also required you to explicitly ask
> for floating point. That was also changed in Python 3.

The solution adopted was different, though: use different
operators for int and float division. This means you can't
accidentally end up with a float when an int is what you
intended.

The equivalent solution here would be to add a new operator
for complex exponentiation that coerces its operands to
complex, and restrict the existing one to floats only.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sun Oct 21 08:11:34 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 21 Oct 2012 19:11:34 +1300
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CABicbJKLubgr0BPDsr5j2J=bvqEsJMkQrLMiSA_3FBOwEgLC6g@mail.gmail.com>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>
	<CABicbJKLubgr0BPDsr5j2J=bvqEsJMkQrLMiSA_3FBOwEgLC6g@mail.gmail.com>
Message-ID: <50839216.1020807@canterbury.ac.nz>

Devin Jeanpierre wrote:
> (Complex
> numbers get a free pass because "complex numbers with rational real
> and imaginary parts" is not a memorable name for a type.)

Complexional?

-- 
Greg

From rosuav at gmail.com  Sun Oct 21 08:17:58 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 21 Oct 2012 17:17:58 +1100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <50839216.1020807@canterbury.ac.nz>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>
	<CABicbJKLubgr0BPDsr5j2J=bvqEsJMkQrLMiSA_3FBOwEgLC6g@mail.gmail.com>
	<50839216.1020807@canterbury.ac.nz>
Message-ID: <CAPTjJmptZZbNoCZOhbFZnW=RuG5xWZUuTRg8cW8tgmvpitDPeA@mail.gmail.com>

On Sun, Oct 21, 2012 at 5:11 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Devin Jeanpierre wrote:
>>
>> (Complex
>> numbers get a free pass because "complex numbers with rational real
>> and imaginary parts" is not a memorable name for a type.)
>
> Complexional?

"Oh is there not one maiden here
Whose homely face and bad complexion
Have caused all hope to disappear
Of ever winning man's affection?"

I'm hoping your name was a joke, because it's rather unwieldy. But
really, computers are no different from anything else; they have to
represent numbers somehow. That pretty much means it has to be
rational, as it's awkward trying to do arithmetic with surds floating
around everywhere.

ChrisA

From storchaka at gmail.com  Sun Oct 21 10:26:19 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 21 Oct 2012 11:26:19 +0300
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <20121021021554.45d6027d@pitrou.net>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<20121021021554.45d6027d@pitrou.net>
Message-ID: <k60bje$bbe$1@ger.gmane.org>

On 21.10.12 03:15, Antoine Pitrou wrote:
> The redundancy sounds like a non-issue to me, since you can implement
> __float__ instead:

 >>> class C:
...     def __init__(self, v):
...         self.v = v
...     def __complex__(self):
...         return self.v**0.5
...
 >>> import cmath
 >>> cmath.cos(C(-1))
(1.5430806348152437-7.195794243779206e-17j)
 >>> cmath.cos(C(1))
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: __complex__ should return a complex object

Returned value can be computed and result can be float or complex 
depending on values. The author of the class with defined __complex__() 
may not even be aware that under certain circumstances he would get a float.

*Always* write "return complex(...)" is too redundant.



From storchaka at gmail.com  Sun Oct 21 10:43:30 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 21 Oct 2012 11:43:30 +0300
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAPTjJmoe2B5B44sB3y-Pj7mA1HVGUzLFCv6W_MyGVzZZ-8moiA@mail.gmail.com>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CAPTjJmqBw0J0vXh=XB26gVgNMZ_30=vWFcKj1WuXrN-PD9w0Jg@mail.gmail.com>
	<CAPTjJmoe2B5B44sB3y-Pj7mA1HVGUzLFCv6W_MyGVzZZ-8moiA@mail.gmail.com>
Message-ID: <k60cjk$hon$1@ger.gmane.org>

On 21.10.12 05:40, Chris Angelico wrote:
> Anyway. Py3 says that int/int --> float is
> acceptable, so float**float --> complex is equally plausible.

The difference is that int/int --> always float, but float**float --> 
float or complex.



From storchaka at gmail.com  Sun Oct 21 10:47:26 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 21 Oct 2012 11:47:26 +0300
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <508390B4.9090306@canterbury.ac.nz>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CAPTjJmqBw0J0vXh=XB26gVgNMZ_30=vWFcKj1WuXrN-PD9w0Jg@mail.gmail.com>
	<508390B4.9090306@canterbury.ac.nz>
Message-ID: <k60cr0$jcg$1@ger.gmane.org>

On 21.10.12 09:05, Greg Ewing wrote:
> The equivalent solution here would be to add a new operator
> for complex exponentiation that coerces its operands to
> complex, and restrict the existing one to floats only.

In case of division a new operator (//) restricted to ints only, and the 
existing one coerces its operands to floats.



From steve at pearwood.info  Sun Oct 21 11:18:22 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 21 Oct 2012 20:18:22 +1100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <k60cr0$jcg$1@ger.gmane.org>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CAPTjJmqBw0J0vXh=XB26gVgNMZ_30=vWFcKj1WuXrN-PD9w0Jg@mail.gmail.com>
	<508390B4.9090306@canterbury.ac.nz> <k60cr0$jcg$1@ger.gmane.org>
Message-ID: <5083BDDE.8030004@pearwood.info>

On 21/10/12 19:47, Serhiy Storchaka wrote:
> On 21.10.12 09:05, Greg Ewing wrote:
>> The equivalent solution here would be to add a new operator
>> for complex exponentiation that coerces its operands to
>> complex, and restrict the existing one to floats only.
>
> In case of division a new operator (//) restricted to ints only,
>and the existing one coerces its operands to floats.

That is incorrect. // works fine with non-ints.


py> 42.9//3.4
12.0



-- 
Steven

From dickinsm at gmail.com  Sun Oct 21 11:45:07 2012
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sun, 21 Oct 2012 10:45:07 +0100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <508387A1.6050909@canterbury.ac.nz>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org>
	<CAP7+vJ+yHf1+iwZA3sZ10Vanwda+gsbvCVQAaKE3_J+dH6Ngww@mail.gmail.com>
	<508387A1.6050909@canterbury.ac.nz>
Message-ID: <CAAu3qLUZt6NQP5Dy8ha_pwGmpZMFX_64TuaxwgER0wr9z4qWPg@mail.gmail.com>

On Sun, Oct 21, 2012 at 6:26 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> I think I've changed my mind on this, since it was pointed
> out that if you're going to return a float instead of a
> complex, you should really be implementing __float__, not
> __complex__.

Yes, I'm wavering on this, too.  I'm reasonably convinced that the
complex constructor is wrong to accept a float return from
__complex__.  But it's not clear to me whether it's better to break
backwards compatibility by fixing that in 3.4, or to accept the
mistake and make cmath behave analogously.

> Also PyComplex_AsComplex() should perhaps enforce that.

It already does.  `complex_new` doesn't use `PyComplex_AsCComplex`.

-- 
Mark

From solipsis at pitrou.net  Sun Oct 21 12:11:01 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 21 Oct 2012 12:11:01 +0200
Subject: [Python-Dev] return type of __complex__
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CAPTjJmqBw0J0vXh=XB26gVgNMZ_30=vWFcKj1WuXrN-PD9w0Jg@mail.gmail.com>
Message-ID: <20121021121101.39512ec9@pitrou.net>

On Sun, 21 Oct 2012 13:38:48 +1100
Chris Angelico <rosuav at gmail.com> wrote:
> On Sun, Oct 21, 2012 at 12:53 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> > Python 2.x legitimately distinguished between floats and complex, e.g.:
> >
> > py> (-100.0)**0.5
> >
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in <module>
> > ValueError: negative number cannot be raised to a fractional power
> >
> > If you wanted a complex result, you can explicitly ask for one:
> >
> > py> (-100.0+0j)**0.5
> > (6.123031769111886e-16+10j)
> >
> >
> > I see that behaviour is changed in Python 3.
> 
> Python 2 (future directives aside) also required you to explicitly ask
> for floating point. That was also changed in Python 3. That doesn't
> mean that this is necessarily the right thing to do, but it does have
> precedent. The square root of a negative number is by nature a complex
> number.

In the set of complex numbers, it is. But in the set of float numbers,
a negative number has no square root. I think Steven may be worried
about the impact on people who don't know about complex numbers, which
is a reasonable concern.

I'm also not sure why we have several variants of the power operator:
**, built-in pow(), math.pow().

Regards

Antoine.



From ncoghlan at gmail.com  Sun Oct 21 12:31:31 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 21 Oct 2012 20:31:31 +1000
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CABicbJKLubgr0BPDsr5j2J=bvqEsJMkQrLMiSA_3FBOwEgLC6g@mail.gmail.com>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>
	<CABicbJKLubgr0BPDsr5j2J=bvqEsJMkQrLMiSA_3FBOwEgLC6g@mail.gmail.com>
Message-ID: <CADiSq7e3u0HOh7j4wqs8kgkRBj8vFcCv=KbPwWWMsV9h2N3fig@mail.gmail.com>

On Sun, Oct 21, 2012 at 3:06 PM, Devin Jeanpierre
<jeanpierreda at gmail.com> wrote:
> On Sat, Oct 20, 2012 at 10:57 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> PEP 3141 is indeed the driver for these changes, and it's based on the
>> Python 3.x numeric tower consisting of strict supersets: Complex >
>> Real > Rational > Integral
>
> Pedant mode: That numeric tower is wrong as it applies to Python -- we
> have some rational types that can represent some numbers that can't be
> represented by our complex and """real""" types.
>
>     >>> int(float(10**100)) == 10**100
>     False
>
> (Also, floats aren't reals and no computer can store any number that
> is not rational and we should stop pretending they can. (Complex
> numbers get a free pass because "complex numbers with rational real
> and imaginary parts" is not a memorable name for a type.))

Floats are not rational either, since they allow infinities and NaN as
values. Both float and Decimal are approximations to the Real number
set, as int and Fraction are approximations to the Integer and
Rational sets. In all cases, they cannot express the full scope of the
associated numeric concept, because they're limited by their physical
representations as bit patterns inside the computer (while that limit
is technically available memory for the unbounded representations, the
practical limit is lower than that if you want vaguely reasonable
running times for arbitrary arithmetic operations).

As far as the conversion operators go, I don't recall there being any
discussion one way or the other - the question simply never came up,
presumably because all code tested already returned complex objects
from __complex__.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sun Oct 21 12:35:24 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 21 Oct 2012 20:35:24 +1000
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <20121021121101.39512ec9@pitrou.net>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CAPTjJmqBw0J0vXh=XB26gVgNMZ_30=vWFcKj1WuXrN-PD9w0Jg@mail.gmail.com>
	<20121021121101.39512ec9@pitrou.net>
Message-ID: <CADiSq7cMo9n7gs+U3LnYM5gLqLTgrGibVp1wwkrDeq5PP0G6dQ@mail.gmail.com>

On Sun, Oct 21, 2012 at 8:11 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> I'm also not sure why we have several variants of the power operator:
> **, built-in pow(), math.pow().

Built-in pow() is provided for efficient modular arithmetic (via the
3-argument "pow(x, y, z)" form that means "x ** y % z")

I don't know the rationale math.pow() though - it may just stem from
the time when the math module was just a really thin wrapper around
the underlying C library's floating point math support.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From steve at pearwood.info  Sun Oct 21 13:00:53 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 21 Oct 2012 22:00:53 +1100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>
Message-ID: <5083D5E5.9010103@pearwood.info>

On 21/10/12 13:57, Nick Coghlan wrote:
> On Sun, Oct 21, 2012 at 11:53 AM, Steven D'Aprano<steve at pearwood.info>  wrote:

>> Python 2.x legitimately distinguished between floats and complex, e.g.:
>>
>> py>  (-100.0)**0.5
>>
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in<module>
>> ValueError: negative number cannot be raised to a fractional power
>>
>> If you wanted a complex result, you can explicitly ask for one:
>>
>> py>  (-100.0+0j)**0.5
>> (6.123031769111886e-16+10j)
>>
>>
>> I see that behaviour is changed in Python 3. Was this discussed before
>> being changed? I see a single sample docstring buried in PEP 3141 that:
>>
>> """a**b; should promote to float or complex when necessary."""
>>
>> but I can't find any discussion of the consequences of this for the
>> majority of users who would be surprised by the unexpected appearance
>> of a "j" inside their numbers.
>
> PEP 3141 is indeed the driver for these changes, and it's based on the
> Python 3.x numeric tower consisting of strict supersets: Complex>
> Real>  Rational>  Integral
>
> If an operation at one level of the tower produces a result in one of
> the larger supersets, then *that's what it will do* rather than
> throwing TypeError. int / int promoting to float is one example, as is
> raising a negative number to a fractional power promoting to complex.
>
> The general philosophy is described in
> http://www.python.org/dev/peps/pep-3141/#implementing-the-arithmetic-operations

Yes, I've read that. But it seems to me that this change appears to have
been driven by type-purity (floats are Liskov-substituable for complex)
rather than practical considerations for people who may not know what
complex numbers are, or at best may have vague recollections of being
told about them in high school, and who now have to deal with the
unexpected appearance of complex numbers in calculations which previously
gave an exception.

When you're dealing with numbers that represent real quantities, getting
a complex result is nearly always an error condition. Better to get an
exception at the point that occurs, than somewhere distant when the number
gets fed to %f formatting, or worse, no error at all, just a silently
generating garbage results.



-- 
Steven

From steve at pearwood.info  Sun Oct 21 13:02:17 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 21 Oct 2012 22:02:17 +1100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <20121021121101.39512ec9@pitrou.net>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CAPTjJmqBw0J0vXh=XB26gVgNMZ_30=vWFcKj1WuXrN-PD9w0Jg@mail.gmail.com>
	<20121021121101.39512ec9@pitrou.net>
Message-ID: <5083D639.70904@pearwood.info>

On 21/10/12 21:11, Antoine Pitrou wrote:
> On Sun, 21 Oct 2012 13:38:48 +1100
> Chris Angelico<rosuav at gmail.com>  wrote:
>> On Sun, Oct 21, 2012 at 12:53 PM, Steven D'Aprano<steve at pearwood.info>  wrote:
>>> Python 2.x legitimately distinguished between floats and complex, e.g.:
>>>
>>> py>  (-100.0)**0.5
>>>
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in<module>
>>> ValueError: negative number cannot be raised to a fractional power
>>>
>>> If you wanted a complex result, you can explicitly ask for one:
>>>
>>> py>  (-100.0+0j)**0.5
>>> (6.123031769111886e-16+10j)
>>>
>>>
>>> I see that behaviour is changed in Python 3.
>>
>> Python 2 (future directives aside) also required you to explicitly ask
>> for floating point. That was also changed in Python 3. That doesn't
>> mean that this is necessarily the right thing to do, but it does have
>> precedent. The square root of a negative number is by nature a complex
>> number.
>
> In the set of complex numbers, it is. But in the set of float numbers,
> a negative number has no square root. I think Steven may be worried
> about the impact on people who don't know about complex numbers, which
> is a reasonable concern.

Precisely.


> I'm also not sure why we have several variants of the power operator:
> **, built-in pow(), math.pow().

In Python 3.3, math.pow is the builtin pow. Presumably for backwards
compatibility reasons when they were different? If so, it was a LONG
time ago:

[steve at ando ~]$ python1.5
Python 1.5.2 (#1, Aug 27 2012, 09:09:18)  [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import math
>>> math.pow
<built-in function pow>


And yet strangely there's never been a cmath.pow.



-- 
Steven

From solipsis at pitrou.net  Sun Oct 21 13:08:02 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 21 Oct 2012 13:08:02 +0200
Subject: [Python-Dev] return type of __complex__
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CAPTjJmqBw0J0vXh=XB26gVgNMZ_30=vWFcKj1WuXrN-PD9w0Jg@mail.gmail.com>
	<20121021121101.39512ec9@pitrou.net> <5083D639.70904@pearwood.info>
Message-ID: <20121021130802.50aeae4e@pitrou.net>

On Sun, 21 Oct 2012 22:02:17 +1100
Steven D'Aprano <steve at pearwood.info> wrote:
> 
> In Python 3.3, math.pow is the builtin pow. Presumably for backwards
> compatibility reasons when they were different? If so, it was a LONG
> time ago:
> 
> [steve at ando ~]$ python1.5
> Python 1.5.2 (#1, Aug 27 2012, 09:09:18)  [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
> >>> import math
> >>> math.pow
> <built-in function pow>

You are being too optimistic:

>>> pow
<built-in function pow>
>>> math.pow
<built-in function pow>
>>> pow is math.pow
False


Regards

Antoine.



From rosuav at gmail.com  Sun Oct 21 14:06:45 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 21 Oct 2012 23:06:45 +1100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <20121021130802.50aeae4e@pitrou.net>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CAPTjJmqBw0J0vXh=XB26gVgNMZ_30=vWFcKj1WuXrN-PD9w0Jg@mail.gmail.com>
	<20121021121101.39512ec9@pitrou.net> <5083D639.70904@pearwood.info>
	<20121021130802.50aeae4e@pitrou.net>
Message-ID: <CAPTjJmomerUZUZm3giP4nCoeurs+jrD9CG2wrKgT8cUdfeE6jA@mail.gmail.com>

On Sun, Oct 21, 2012 at 10:08 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sun, 21 Oct 2012 22:02:17 +1100
> Steven D'Aprano <steve at pearwood.info> wrote:
>>
>> In Python 3.3, math.pow is the builtin pow. Presumably for backwards
>> compatibility reasons when they were different? If so, it was a LONG
>> time ago:
>>
>> [steve at ando ~]$ python1.5
>> Python 1.5.2 (#1, Aug 27 2012, 09:09:18)  [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
>> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>> >>> import math
>> >>> math.pow
>> <built-in function pow>
>
> You are being too optimistic:
>
>>>> pow
> <built-in function pow>
>>>> math.pow
> <built-in function pow>
>>>> pow is math.pow
> False

The one in math doesn't take three args.

Python 3.2 (r32:88445, Feb 20 2011, 21:29:02) [MSC v.1500 32 bit
(Intel)] on win32
>>> math.pow(1,2,3)
Traceback (most recent call last):
  File "<pyshell#3>", line 1, in <module>
    math.pow(1,2,3)
TypeError: pow expected 2 arguments, got 3

I conducted a brief survey of a non-technical nature (that is to say,
I rambled to my brother and listened to what he said in response), and
he's of the opinion that many programmers would be surprised and
confused by the sudden appearance of complex numbers in programs that
weren't using them. On the flip side, it's kinda like a NaN - you
don't get an error immediately, but you'll either see it in your
output or get an exception later on.

His suggestion was that an expression unexpectedly returning a complex
should issue a warning. Warnings aren't really useful in Python as
most people have them off, but would there be any way that this sort
of thing could be caught by a linter? It'd probably require some
fairly deep code analysis to figure out that you're taking the square
root of something that could be negative, though.

ChrisA

From stephen at xemacs.org  Sun Oct 21 14:23:29 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 21 Oct 2012 21:23:29 +0900
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <50823162.5020203@canterbury.ac.nz>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
	<CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
	<CAK5idxRH2ri238Zon_=vY5qRA0j6ggJxWEvJ_DMaw+uJnsK-hg@mail.gmail.com>
	<87bofx69qm.fsf@uwakimon.sk.tsukuba.ac.jp>
	<50823162.5020203@canterbury.ac.nz>
Message-ID: <877gqk57zy.fsf@uwakimon.sk.tsukuba.ac.jp>

Greg Ewing writes:
 > Stephen J. Turnbull wrote:
 > > It's a design bug, yes.  The question is, does it conform to
 > > documented behavior?
 > 
 > The 2.7 docs say this about __complex__:
 > 
 >     Called to implement the built-in function complex() ...
 >     Should return a value of the appropriate type.
 > 
 > So the question is whether float is an "appropriate type" when
 > you're expecting a complex.

I probably not say that, but even so my personal taste would be to fix
the docs to describe the current behavior in 2.7.  Evidently somebody
thought "float" was appropriate, or they would have just written
"Returns a complex value."  Stability is more important than catering
to my taste (even if it happens to represent a majority in some sense).


From stephen at xemacs.org  Sun Oct 21 14:27:33 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sun, 21 Oct 2012 21:27:33 +0900
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
Message-ID: <87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>

Daniel Holth writes:

 > Another solution comes to mind. Put the description in the payload.

That would work for me, but it would cause extensibility problems if
anybody later found a use for other formatted fields.  I have no
intuition for that.


From dholth at gmail.com  Sun Oct 21 15:14:03 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 21 Oct 2012 09:14:03 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAG8k2+6cHja8ecsN9aoKRq25FdcwukZXX4sWS0HGXruXdupvhw@mail.gmail.com>

On Sun, Oct 21, 2012 at 8:27 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Daniel Holth writes:
>
>  > Another solution comes to mind. Put the description in the payload.
>
> That would work for me, but it would cause extensibility problems if
> anybody later found a use for other formatted fields.  I have no
> intuition for that.

It's OK to limit the extensibility of this format. IIUC pypi usually
gets the metadata as a pre-parsed dict anyway, and the installer /
runtime does not need formatted fields at all (only parsing the name,
version, requirements to do dependency resolution).

From dickinsm at gmail.com  Sun Oct 21 15:29:34 2012
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sun, 21 Oct 2012 14:29:34 +0100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <877gqk57zy.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
	<CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
	<CAK5idxRH2ri238Zon_=vY5qRA0j6ggJxWEvJ_DMaw+uJnsK-hg@mail.gmail.com>
	<87bofx69qm.fsf@uwakimon.sk.tsukuba.ac.jp>
	<50823162.5020203@canterbury.ac.nz>
	<877gqk57zy.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAAu3qLVB+djn0ho=j1-9XJ_hv-9atLJGVpoiA0wGFHaa7jKiWQ@mail.gmail.com>

On Sun, Oct 21, 2012 at 1:23 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> I probably not say that, but even so my personal taste would be to fix
> the docs to describe the current behavior in 2.7.  Evidently somebody
> thought "float" was appropriate

The implementation of complex_new is complicated enough that it's not
at all evident that anyone thought 'float' was appropriate;  that it's
accepted may have just been a side-effect of the implementation.

-- 
Mark

From g.brandl at gmx.net  Sun Oct 21 16:28:49 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 21 Oct 2012 16:28:49 +0200
Subject: [Python-Dev] Interest in seeing sh.py in the stdlib
In-Reply-To: <CAPZV6o_x6wnxht0CQjr=sY6m_wASgXzpaYnkQAGxpTPn4XEJ5g@mail.gmail.com>
References: <CADijH1VDUm8yXcBJKTnJbq3H8yp9ogSST9NWDGDYK4_DJjzGkQ@mail.gmail.com>
	<CAPZV6o_x6wnxht0CQjr=sY6m_wASgXzpaYnkQAGxpTPn4XEJ5g@mail.gmail.com>
Message-ID: <k610qt$66p$1@ger.gmane.org>

On 10/20/2012 11:27 PM, Benjamin Peterson wrote:
> 2012/10/20 Andrew Moffat <andrew.robert.moffat at gmail.com>:
>> Hi,
>>
>> I'm the author of sh.py, a subprocess module rewrite for Linux and OSX.  It
>> serves as a powerful and intuitive interface to launching subprocesses
>> http://amoffat.github.com/sh/.  It has been maintained on github
>> https://github.com/amoffat/sh for about 10 months and currently has about
>> 25k installs, according to pythonpackages.com.
>>
>> Andy Grover maintains the Fedora rpm for sh.py
>> http://arm.koji.fedoraproject.org/koji/buildinfo?buildID=94247  and Nick
>> Moffit has submitted an older version of sh.py (which was called pbs) to be
>> included in Debian distros
>> http://pkgs.org/debian-wheezy/debian-main-i386/python-pbs_0.95-1_all.deb.html
>>
>> I'm interested in making sh.py more accessible to help bring Python forward
>> in the area of shell scripting, so I'm interested in seeing if sh would be
>> suitable for the standard library.  Is there any other interest in something
>> like this?
> 
> You should try the python-ideas list.

In addition, let me say upfront that while the library looks very nice at
a glance, its behavior is much too magical to be included as a standard
library IMO.

cheers,
Georg


From guido at python.org  Sun Oct 21 16:35:32 2012
From: guido at python.org (Guido van Rossum)
Date: Sun, 21 Oct 2012 07:35:32 -0700
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <5083D639.70904@pearwood.info>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CAPTjJmqBw0J0vXh=XB26gVgNMZ_30=vWFcKj1WuXrN-PD9w0Jg@mail.gmail.com>
	<20121021121101.39512ec9@pitrou.net> <5083D639.70904@pearwood.info>
Message-ID: <CAP7+vJJk+_jKnH=_cNF+f2PH4cjMA7+RMzZqKAvhAtJrsEXotA@mail.gmail.com>

Math.pow() predates ** by many releases.

On Sunday, October 21, 2012, Steven D'Aprano wrote:

> On 21/10/12 21:11, Antoine Pitrou wrote:
>
>> On Sun, 21 Oct 2012 13:38:48 +1100
>> Chris Angelico<rosuav at gmail.com>  wrote:
>>
>>> On Sun, Oct 21, 2012 at 12:53 PM, Steven D'Aprano<steve at pearwood.info>
>>>  wrote:
>>>
>>>> Python 2.x legitimately distinguished between floats and complex, e.g.:
>>>>
>>>> py>  (-100.0)**0.5
>>>>
>>>> Traceback (most recent call last):
>>>>    File "<stdin>", line 1, in<module>
>>>> ValueError: negative number cannot be raised to a fractional power
>>>>
>>>> If you wanted a complex result, you can explicitly ask for one:
>>>>
>>>> py>  (-100.0+0j)**0.5
>>>> (6.123031769111886e-16+10j)
>>>>
>>>>
>>>> I see that behaviour is changed in Python 3.
>>>>
>>>
>>> Python 2 (future directives aside) also required you to explicitly ask
>>> for floating point. That was also changed in Python 3. That doesn't
>>> mean that this is necessarily the right thing to do, but it does have
>>> precedent. The square root of a negative number is by nature a complex
>>> number.
>>>
>>
>> In the set of complex numbers, it is. But in the set of float numbers,
>> a negative number has no square root. I think Steven may be worried
>> about the impact on people who don't know about complex numbers, which
>> is a reasonable concern.
>>
>
> Precisely.
>
>
>  I'm also not sure why we have several variants of the power operator:
>> **, built-in pow(), math.pow().
>>
>
> In Python 3.3, math.pow is the builtin pow. Presumably for backwards
> compatibility reasons when they were different? If so, it was a LONG
> time ago:
>
> [steve at ando ~]$ python1.5
> Python 1.5.2 (#1, Aug 27 2012, 09:09:18)  [GCC 4.1.2 20080704 (Red Hat
> 4.1.2-52)] on linux2
> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>
>> import math
>>>> math.pow
>>>>
>>> <built-in function pow>
>
>
> And yet strangely there's never been a cmath.pow.
>
>
>
> --
> Steven
> ______________________________**_________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> guido%40python.org<http://mail.python.org/mailman/options/python-dev/guido%40python.org>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121021/79473e9c/attachment.html>

From christian at python.org  Sun Oct 21 17:32:02 2012
From: christian at python.org (Christian Heimes)
Date: Sun, 21 Oct 2012 17:32:02 +0200
Subject: [Python-Dev] cpython: Build the _sha3 module with VS 2008.
In-Reply-To: <3Xl4PT03YszQt2@mail.python.org>
References: <3Xl4PT03YszQt2@mail.python.org>
Message-ID: <50841572.6040709@python.org>

Am 21.10.2012 17:23, schrieb antoine.pitrou:
> http://hg.python.org/cpython/rev/ce9c9cbd1b11
> changeset:   79875:ce9c9cbd1b11
> user:        Antoine Pitrou <solipsis at pitrou.net>
> date:        Sun Oct 21 17:21:04 2012 +0200
> summary:
>   Build the _sha3 module with VS 2008.

For VS 2010 I decided against this approach and built the extension as a
separate .pyd extension because the _sha3 module is rather large (> 200
kB). I think your change is going to break VS 2010.

What do the other think? Should the code by built-in by default?

From tjreedy at udel.edu  Sun Oct 21 17:34:05 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 21 Oct 2012 11:34:05 -0400
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAAu3qLUZt6NQP5Dy8ha_pwGmpZMFX_64TuaxwgER0wr9z4qWPg@mail.gmail.com>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org>
	<CAP7+vJ+yHf1+iwZA3sZ10Vanwda+gsbvCVQAaKE3_J+dH6Ngww@mail.gmail.com>
	<508387A1.6050909@canterbury.ac.nz>
	<CAAu3qLUZt6NQP5Dy8ha_pwGmpZMFX_64TuaxwgER0wr9z4qWPg@mail.gmail.com>
Message-ID: <k614lc$4gh$1@ger.gmane.org>

On 10/21/2012 5:45 AM, Mark Dickinson wrote:
> On Sun, Oct 21, 2012 at 6:26 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> I think I've changed my mind on this, since it was pointed
>> out that if you're going to return a float instead of a
>> complex, you should really be implementing __float__, not
>> __complex__.
>
> Yes, I'm wavering on this, too.  I'm reasonably convinced that the
> complex constructor is wrong to accept a float return from
> __complex__.  But it's not clear to me whether it's better to break
> backwards compatibility by fixing that in 3.4, or to accept the
> mistake and make cmath behave analogously.

I think we should fix it. A float return appears very rare, as well as 
wrong.


-- 
Terry Jan Reedy


From tjreedy at udel.edu  Sun Oct 21 17:36:45 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 21 Oct 2012 11:36:45 -0400
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <877gqk57zy.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
	<CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
	<CAK5idxRH2ri238Zon_=vY5qRA0j6ggJxWEvJ_DMaw+uJnsK-hg@mail.gmail.com>
	<87bofx69qm.fsf@uwakimon.sk.tsukuba.ac.jp>
	<50823162.5020203@canterbury.ac.nz>
	<877gqk57zy.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <k614qc$4gh$2@ger.gmane.org>

On 10/21/2012 8:23 AM, Stephen J. Turnbull wrote:
> Greg Ewing writes:
>   > Stephen J. Turnbull wrote:
>   > > It's a design bug, yes.  The question is, does it conform to
>   > > documented behavior?
>   >
>   > The 2.7 docs say this about __complex__:
>   >
>   >     Called to implement the built-in function complex() ...
>   >     Should return a value of the appropriate type.

I would take that as meaning complex or subclass thereof or whatever is 
consistent with float() and int().

>   >
>   > So the question is whether float is an "appropriate type" when
>   > you're expecting a complex.
>
> I probably not say that, but even so my personal taste would be to fix
> the docs to describe the current behavior in 2.7.  Evidently somebody
> thought "float" was appropriate, or they would have just written
> "Returns a complex value."  Stability is more important than catering
> to my taste (even if it happens to represent a majority in some sense).
>


-- 
Terry Jan Reedy


From rdmurray at bitdance.com  Sun Oct 21 17:46:12 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 21 Oct 2012 11:46:12 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+6cHja8ecsN9aoKRq25FdcwukZXX4sWS0HGXruXdupvhw@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6cHja8ecsN9aoKRq25FdcwukZXX4sWS0HGXruXdupvhw@mail.gmail.com>
Message-ID: <20121021154613.8FF27250002@webabinitio.net>

On Sun, 21 Oct 2012 09:14:03 -0400, Daniel Holth <dholth at gmail.com> wrote:
> On Sun, Oct 21, 2012 at 8:27 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> > Daniel Holth writes:
> >
> >  > Another solution comes to mind. Put the description in the payload.
> >
> > That would work for me, but it would cause extensibility problems if
> > anybody later found a use for other formatted fields.  I have no
> > intuition for that.
> 
> It's OK to limit the extensibility of this format. IIUC pypi usually
> gets the metadata as a pre-parsed dict anyway, and the installer /
> runtime does not need formatted fields at all (only parsing the name,
> version, requirements to do dependency resolution).

Well, if you did want to maintain extensibility, you could go whole
hog and allow MIME types and Content-Type: mulitpart/mixed :)

--David

From jeanpierreda at gmail.com  Sun Oct 21 19:15:16 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sun, 21 Oct 2012 13:15:16 -0400
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CADiSq7e3u0HOh7j4wqs8kgkRBj8vFcCv=KbPwWWMsV9h2N3fig@mail.gmail.com>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>
	<CABicbJKLubgr0BPDsr5j2J=bvqEsJMkQrLMiSA_3FBOwEgLC6g@mail.gmail.com>
	<CADiSq7e3u0HOh7j4wqs8kgkRBj8vFcCv=KbPwWWMsV9h2N3fig@mail.gmail.com>
Message-ID: <CABicbJKeKqNDURvTOaymE+HuQEadS4pLxHGP__C4BmuMPrNojQ@mail.gmail.com>

I guess I was asking for this. (Sorry for OT conversation.)

On Sun, Oct 21, 2012 at 6:31 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> (Also, floats aren't reals and no computer can store any number that
>> is not rational and we should stop pretending they can. (Complex
>> numbers get a free pass because "complex numbers with rational real
>> and imaginary parts" is not a memorable name for a type.))
>
> Floats are not rational either, since they allow infinities and NaN as
> values. Both float and Decimal are approximations to the Real number
> set, as int and Fraction are approximations to the Integer and
> Rational sets.

You're missing a step here. Are you claiming that the real numbers
have infinities or NaN?

Why do floats approximate the reals, while Fraction approximates the
rationals? After all, for any real number, a Fraction instance can
become arbitrarily close to it (given sufficient memory). The same is
not true of a float, which can only get to a relative distance of
machine epsilon, and then only when the real number is close to zero.
It would seem as if Fractions approximate the reals and rationals, and
floats do not approximate anything at all (at least, not arbitrarily
closely*).

* I think "can be made arbitrarily close to the value in question" is
the usual meaning of "approximates" in mathematics, but I'm not sure
about numerical computing (maybe it just means "with sufficiently
small difference for values we care about"?)

> In all cases, they cannot express the full scope of the
> associated numeric concept, because they're limited by their physical
> representations as bit patterns inside the computer (while that limit
> is technically available memory for the unbounded representations, the
> practical limit is lower than that if you want vaguely reasonable
> running times for arbitrary arithmetic operations).

There is a deeper difference in how badly we can represent the
rationals versus the reals. For an arbitrary rational number, we can
represent it precisely on a computer with sufficiently many bytes of
memory. That is not true for real numbers: the vast majority of real
numbers cannot be represented precisely in any form on a computer, no
matter how large it is. And while there are irrational numbers we can
represent precisely on a computer, floats don't represent any of them;
they only represent a selection of rational numbers, and nan/inf/-inf.

> As far as the conversion operators go, I don't recall there being any
> discussion one way or the other - the question simply never came up,
> presumably because all code tested already returned complex objects
> from __complex__.

That's a fair objection to what I said. I guess the discussion is here/now.

-- Devin

From solipsis at pitrou.net  Sun Oct 21 20:53:45 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 21 Oct 2012 20:53:45 +0200
Subject: [Python-Dev] cpython: Build the _sha3 module with VS 2008.
References: <3Xl4PT03YszQt2@mail.python.org>
	<50841572.6040709@python.org>
Message-ID: <20121021205345.43152f13@pitrou.net>

On Sun, 21 Oct 2012 17:32:02 +0200
Christian Heimes <christian at python.org> wrote:
> Am 21.10.2012 17:23, schrieb antoine.pitrou:
> > http://hg.python.org/cpython/rev/ce9c9cbd1b11
> > changeset:   79875:ce9c9cbd1b11
> > user:        Antoine Pitrou <solipsis at pitrou.net>
> > date:        Sun Oct 21 17:21:04 2012 +0200
> > summary:
> >   Build the _sha3 module with VS 2008.
> 
> For VS 2010 I decided against this approach and built the extension as a
> separate .pyd extension because the _sha3 module is rather large (> 200
> kB). I think your change is going to break VS 2010.

I don't know. If it breaks VS 2010 then it should be reverted. On the
other hand I'd like to continue building Python under Windows without
needing to install yet another MSVC version.

Regards

Antoine.



From martin at v.loewis.de  Sun Oct 21 21:27:11 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 21 Oct 2012 21:27:11 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Build the _sha3 module
	with VS 2008.
In-Reply-To: <50841572.6040709@python.org>
References: <3Xl4PT03YszQt2@mail.python.org> <50841572.6040709@python.org>
Message-ID: <50844C8F.8060706@v.loewis.de>

Am 21.10.12 17:32, schrieb Christian Heimes:
> Am 21.10.2012 17:23, schrieb antoine.pitrou:
>> http://hg.python.org/cpython/rev/ce9c9cbd1b11
>> changeset:   79875:ce9c9cbd1b11
>> user:        Antoine Pitrou <solipsis at pitrou.net>
>> date:        Sun Oct 21 17:21:04 2012 +0200
>> summary:
>>    Build the _sha3 module with VS 2008.
>
> For VS 2010 I decided against this approach and built the extension as a
> separate .pyd extension because the _sha3 module is rather large (> 200
> kB). I think your change is going to break VS 2010.
>
> What do the other think? Should the code by built-in by default?

It's a packaging and maintenance issue. If it is in pythonxy.dll, the
project file becomes easier to maintain. That's why I was always opposed
to splitting up the existing pythonxy.dll into many project files.

OTOH, when adding a module, a decision can be made based on size (e.g.);
if the module depends on extra libraries, it shouldn't be in
pythonxy.dll.

For VS 2008, I don't care - this set of project files isn't really
intended for production use, but rather for ongoing development.

In the long term, I wish we could use the platform implementations
of SHA-3 rather than shipping our own.

Regards,
Martin

From christian at python.org  Sun Oct 21 21:31:02 2012
From: christian at python.org (Christian Heimes)
Date: Sun, 21 Oct 2012 21:31:02 +0200
Subject: [Python-Dev] cpython: Build the _sha3 module with VS 2008.
In-Reply-To: <20121021205345.43152f13@pitrou.net>
References: <3Xl4PT03YszQt2@mail.python.org> <50841572.6040709@python.org>
	<20121021205345.43152f13@pitrou.net>
Message-ID: <50844D76.1070506@python.org>

Am 21.10.2012 20:53, schrieb Antoine Pitrou:
> I don't know. If it breaks VS 2010 then it should be reverted. On the
> other hand I'd like to continue building Python under Windows without
> needing to install yet another MSVC version.

I backed out your commit and committed a _sha3.vcproj file instead:

http://hg.python.org/cpython/rev/c8217046e2cd

That fixes the issue with VS 2008. I don't have any older versions of
MSVC on my box so I can't fix the other projects.

From solipsis at pitrou.net  Sun Oct 21 21:40:45 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 21 Oct 2012 21:40:45 +0200
Subject: [Python-Dev] cpython: Build the _sha3 module with VS 2008.
In-Reply-To: <50844D76.1070506@python.org>
References: <3Xl4PT03YszQt2@mail.python.org> <50841572.6040709@python.org>
	<20121021205345.43152f13@pitrou.net> <50844D76.1070506@python.org>
Message-ID: <20121021214045.697c64ee@pitrou.net>

On Sun, 21 Oct 2012 21:31:02 +0200
Christian Heimes <christian at python.org> wrote:
> Am 21.10.2012 20:53, schrieb Antoine Pitrou:
> > I don't know. If it breaks VS 2010 then it should be reverted. On the
> > other hand I'd like to continue building Python under Windows without
> > needing to install yet another MSVC version.
> 
> I backed out your commit and committed a _sha3.vcproj file instead:
> 
> http://hg.python.org/cpython/rev/c8217046e2cd
> 
> That fixes the issue with VS 2008. I don't have any older versions of
> MSVC on my box so I can't fix the other projects.

Thank you :)

Antoine.

From christian at python.org  Sun Oct 21 21:52:01 2012
From: christian at python.org (Christian Heimes)
Date: Sun, 21 Oct 2012 21:52:01 +0200
Subject: [Python-Dev] cpython: Build the _sha3 module with VS 2008.
In-Reply-To: <50844C8F.8060706@v.loewis.de>
References: <3Xl4PT03YszQt2@mail.python.org> <50841572.6040709@python.org>
	<50844C8F.8060706@v.loewis.de>
Message-ID: <50845261.1080707@python.org>

Am 21.10.2012 21:27, schrieb "Martin v. L?wis":
> It's a packaging and maintenance issue. If it is in pythonxy.dll, the
> project file becomes easier to maintain. That's why I was always opposed
> to splitting up the existing pythonxy.dll into many project files.
> 
> OTOH, when adding a module, a decision can be made based on size (e.g.);
> if the module depends on extra libraries, it shouldn't be in
> pythonxy.dll.
> 
> For VS 2008, I don't care - this set of project files isn't really
> intended for production use, but rather for ongoing development.
> 
> In the long term, I wish we could use the platform implementations
> of SHA-3 rather than shipping our own.

Hello Martin,

Ultimately it's your decision. You are the expert and build master for
Windows releases. I've no hard feelings if you prefer to include the
code in the main dll. Just say so and I will change the module ASAP.

The sha3 module depends on the Keccak reference implementation. I guess
the loop unrolling and massive usage of macros in the optimized
implementation increases the size of the code considerable. I didn't
want to increase the main python.dll from 2.7 MB to almost 3 MB.

We may have to ship our own SHA-3 code for quite a while. OpenSSL hasn't
even began to include SHA-3. The bugs.python.org entry about Keccak is
still the number one hit of a Google search about "openssl keccak".

Christian

From ncoghlan at gmail.com  Mon Oct 22 01:11:26 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 22 Oct 2012 09:11:26 +1000
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>

On Sun, Oct 21, 2012 at 10:27 PM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
> Daniel Holth writes:
>
>  > Another solution comes to mind. Put the description in the payload.
>
> That would work for me, but it would cause extensibility problems if
> anybody later found a use for other formatted fields.  I have no
> intuition for that.

That level of extensibility mainly shows up with the extra metadata
files in the .dist-info directory (so it's part of the wheel and
installation database formats, but not a concern for the main metadata
file).

An in-memory metadata interchange format would need to deal with it,
but such a format can also use properly structured datatypes.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From greg.ewing at canterbury.ac.nz  Mon Oct 22 02:10:13 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 22 Oct 2012 13:10:13 +1300
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <5083D5E5.9010103@pearwood.info>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>
	<5083D5E5.9010103@pearwood.info>
Message-ID: <50848EE5.5070105@canterbury.ac.nz>

Steven D'Aprano wrote:

> When you're dealing with numbers that represent real quantities, getting
> a complex result is nearly always an error condition. Better to get an
> exception at the point that occurs, than somewhere distant when the number
> gets fed to %f formatting, or worse, no error at all, just a silently
> generating garbage results.

Yeah. I don't think Inland Revenue would be very impressed if
I tried to tell them I had imaginary tax-to-pay, advantageous
though it might be for me.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Mon Oct 22 02:15:27 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 22 Oct 2012 13:15:27 +1300
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <877gqk57zy.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <508150D8.3060108@gmail.com>
	<CADiSq7ewL5-+EgELiFw0+qOpUWVL-p5aJwEJjA7OGHZUZVG9OQ@mail.gmail.com>
	<50816E9E.1070206@gmail.com>
	<CAPZV6o-TVTTZEoVbCVw0uCS1Xd2+tpTafm__pj4QeQKWOnvbxQ@mail.gmail.com>
	<k5rsq6$pt4$2@ger.gmane.org>
	<CAPZV6o-yS1+BefSYM2wk4871b8JdfZsTedp29QXY1tofwJwQ9A@mail.gmail.com>
	<CAK5idxRH2ri238Zon_=vY5qRA0j6ggJxWEvJ_DMaw+uJnsK-hg@mail.gmail.com>
	<87bofx69qm.fsf@uwakimon.sk.tsukuba.ac.jp>
	<50823162.5020203@canterbury.ac.nz>
	<877gqk57zy.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <5084901F.9040408@canterbury.ac.nz>

Stephen J. Turnbull wrote:
> Evidently somebody
> thought "float" was appropriate, or they would have just written
> "Returns a complex value."

It's not quite as simple as that, because that paragraph in the
docs is actually shared between the descriptions of __int__,
__float__ and __complex__. So it's ambiguous whether "appropriate"
means of the exact type corresponding to the name of the function,
or something substitutable for that type.

-- 
Greg

From rosuav at gmail.com  Mon Oct 22 05:25:45 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 22 Oct 2012 14:25:45 +1100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <50848EE5.5070105@canterbury.ac.nz>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>
	<5083D5E5.9010103@pearwood.info>
	<50848EE5.5070105@canterbury.ac.nz>
Message-ID: <CAPTjJmqSgZ-53UAPcTjhDUdVZMv6JBgSv0K3nyVcatVAw9qq7g@mail.gmail.com>

On Mon, Oct 22, 2012 at 11:10 AM, Greg Ewing
<greg.ewing at canterbury.ac.nz> wrote:
> Steven D'Aprano wrote:
>
>> When you're dealing with numbers that represent real quantities, getting
>> a complex result is nearly always an error condition. Better to get an
>> exception at the point that occurs, than somewhere distant when the number
>> gets fed to %f formatting, or worse, no error at all, just a silently
>> generating garbage results.
>
>
> Yeah. I don't think Inland Revenue would be very impressed if
> I tried to tell them I had imaginary tax-to-pay, advantageous
> though it might be for me.

Unless the IRS instructs you to submit the square root of your income,
I doubt that this will ever come up. Amusing though the notion be.

There really aren't that many situations where a program will be
completely oblivious of complex/imaginary numbers and be able to
encounter them... are there?

ChrisA

From steve at pearwood.info  Mon Oct 22 06:35:08 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Mon, 22 Oct 2012 15:35:08 +1100
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAPTjJmqSgZ-53UAPcTjhDUdVZMv6JBgSv0K3nyVcatVAw9qq7g@mail.gmail.com>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>
	<5083D5E5.9010103@pearwood.info>
	<50848EE5.5070105@canterbury.ac.nz>
	<CAPTjJmqSgZ-53UAPcTjhDUdVZMv6JBgSv0K3nyVcatVAw9qq7g@mail.gmail.com>
Message-ID: <20121022043508.GC10202@ando>

On Mon, Oct 22, 2012 at 02:25:45PM +1100, Chris Angelico wrote:

> There really aren't that many situations where a program will be
> completely oblivious of complex/imaginary numbers and be able to
> encounter them... are there?

Fortunately the math module does not promote float to complex:

py> math.asin(2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: math domain error

(although the error message could be a bit nicer...)

so as far as I can tell, the only way you could accidentally get a 
complex number without expecting one is by exponentiation, either by ** 
or the builtin pow. This includes square roots. Exponentiation isn't as 
common as the basic four arithmetic operators, but it is hardly exotic.

Any time you have a non-integer power and a negative base, you will get 
a complex number. So there are two error conditions which used to give 
an exception but now promote to complex:

- taking the root (square root, cube root, etc.) of a number which 
  should be positive, but accidentally becomes negative (perhaps due to
  round-off error, you get a small -ve number instead of zero);

- the base may legitimately be negative, and the exponent is supposed
  to be a positive integer but accidentally because non-integral.

I suppose as someone who is rather fond of complex numbers, I should be 
pleased about this change, which raises the profile of complex and makes 
it closer to a "first class" numeric type. But my concern is that people 
who never dreamed that complex numbers even existed suddenly need to 
beware of them, and write LBYL code to avoid them, rather than relying 
on Python raising an exception.

In a nutshell, the semantics of ** have changed between 2.x and 3.x, and 
that's going to have consequences when porting code from 2 to 3. Perhaps 
2to3 needs to warn about the use of ** and pow?

[wishful thinking]
If we had floating point contexts, we could just tell people to stick 

import float
float.context.promote_to_complex = False

at the top of their module and they'd be fine.


-- 
Steven

From stephen at xemacs.org  Mon Oct 22 08:13:05 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 22 Oct 2012 15:13:05 +0900
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <CAPTjJmqSgZ-53UAPcTjhDUdVZMv6JBgSv0K3nyVcatVAw9qq7g@mail.gmail.com>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>
	<5083D5E5.9010103@pearwood.info>
	<50848EE5.5070105@canterbury.ac.nz>
	<CAPTjJmqSgZ-53UAPcTjhDUdVZMv6JBgSv0K3nyVcatVAw9qq7g@mail.gmail.com>
Message-ID: <873917591q.fsf@uwakimon.sk.tsukuba.ac.jp>

Chris Angelico writes:

 > There really aren't that many situations where a program will be
 > completely oblivious of complex/imaginary numbers and be able to
 > encounter them... are there?

Stats.  Happens in naive student programs with naive student models.

Specifically, in computing variance or SSE, it's often convenient to
subtract the square of the mean from the sum of the squares.  With
ill-conditioned data (specifically, perfectly multicollinear series
that arise when you do a linear regression on an accounting identity)
I occasionally[1] see a negative variance in submitted output.  That
implies a complex standard deviation.

Unlike Inland Revenue or even the IRS, third-rate MBA students are
completely able to take complex results in stride.  That's the value
of a higher education! ;-)

Of course regressing an accounting identity is a (big) mistake anyway,
but the students are more likely to argue with a professor than with a
computer.

You might also see this in various applications in computational
geometry.

Footnotes: 
[1]  Every couple of years.


From techtonik at gmail.com  Mon Oct 22 11:02:31 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 22 Oct 2012 12:02:31 +0300
Subject: [Python-Dev] Web accessible core code search and other devguide
	questions
Message-ID: <CAPkN8x+pkeNNG3csy5Pyc1HBriNzzKSEq61wBK3shWoxw0Oe9Q@mail.gmail.com>

Hi again,

http://docs.python.org/devguide/faq.html?highlight=search

What can I use to browse, search and troubleshoot core Python sources online?

Why the question "Where do I find Python core code?" is not the first
in the dev. guide? =)
There is clearly a lot of stuff on http://hg.python.org/ that requires
some guidance.

I'd also include some more technical information at the top.

And FAQ is very strange. "What standards of behaviour are expected in
these communication channels?" - is that really the question people
frequently ask? IMHO the question should be "Where can I get help with
development?" and the answer include a link to dev.guide chapter on
this matter.
--
anatoly t.

From larry at hastings.org  Mon Oct 22 13:34:26 2012
From: larry at hastings.org (Larry Hastings)
Date: Mon, 22 Oct 2012 04:34:26 -0700
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <20121022043508.GC10202@ando>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>
	<5083D5E5.9010103@pearwood.info>
	<50848EE5.5070105@canterbury.ac.nz>
	<CAPTjJmqSgZ-53UAPcTjhDUdVZMv6JBgSv0K3nyVcatVAw9qq7g@mail.gmail.com>
	<20121022043508.GC10202@ando>
Message-ID: <50852F42.8030101@hastings.org>

On 10/21/2012 09:35 PM, Steven D'Aprano wrote:
> so as far as I can tell, the only way you could accidentally get a
> complex number without expecting one is by exponentiation, either by **
> or the builtin pow. This includes square roots. Exponentiation isn't as
> common as the basic four arithmetic operators, but it is hardly exotic.
>
> Any time you have a non-integer power and a negative base, you will get
> a complex number.

No, only via the builtin pow.

    % python3
    Python 3.3.0 (default, Sep 29 2012, 22:42:55)
    [GCC 4.6.3] on linux
    Type "help", "copyright", "credits" or "license" for more information.
     >>> import math
     >>> pow(-1, 0.5)
    (6.123031769111886e-17+1j)
     >>> -1 ** 0.5
    -1.0
     >>> math.pow(-1, 0.5)
    Traceback (most recent call last):
       File "<stdin>", line 1, in <module>
    ValueError: math domain error
     >>>



//arry/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121022/e220509c/attachment.html>

From jeanpierreda at gmail.com  Mon Oct 22 13:37:57 2012
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Mon, 22 Oct 2012 07:37:57 -0400
Subject: [Python-Dev] return type of __complex__
In-Reply-To: <50852F42.8030101@hastings.org>
References: <508150D8.3060108@gmail.com> <5081E3B1.8050707@canterbury.ac.nz>
	<k5uu1g$m9$1@ger.gmane.org> <50835586.3000602@pearwood.info>
	<CADiSq7ddvT50BqCm7cwbXTTKgYWb5F_E5dSQU22FzD12kuhBOw@mail.gmail.com>
	<5083D5E5.9010103@pearwood.info> <50848EE5.5070105@canterbury.ac.nz>
	<CAPTjJmqSgZ-53UAPcTjhDUdVZMv6JBgSv0K3nyVcatVAw9qq7g@mail.gmail.com>
	<20121022043508.GC10202@ando> <50852F42.8030101@hastings.org>
Message-ID: <CABicbJKe+nLfAaVjUu1fzX=1gGdpdxvFm0obxsRRiOGOe_O83A@mail.gmail.com>

On Mon, Oct 22, 2012 at 7:34 AM, Larry Hastings <larry at hastings.org> wrote:
>>>> -1 ** 0.5
> -1.0

Be careful about order of operations!

    >>> (-1)**.5
    (6.123233995736766e-17+1j)

-- Devin

From techtonik at gmail.com  Mon Oct 22 15:31:34 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 22 Oct 2012 16:31:34 +0300
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH on
	Windows
Message-ID: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>

Could anybody reopen http://bugs.python.org/issue8766 ? I can't.
Reproducible 100% with Python 3.2 and 3.3 (3.1 didn't test).

> set PYTHONHOME=C:\
> python

BTW, what is the role of PYTHONPATH on Windows?
Is it a path for %INSTALLDIR%\Lib\site-packages?
--
anatoly t.

From rdmurray at bitdance.com  Mon Oct 22 15:36:56 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 22 Oct 2012 09:36:56 -0400
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH on
	Windows
In-Reply-To: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
References: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
Message-ID: <20121022133656.DBDCB25016F@webabinitio.net>

You are more likely to get action on bugs by posting to the bug tracker.

That said, since 8766 was fixed, your issue is most likely a new one.
Please open a new bug report.

--David

On Mon, 22 Oct 2012 16:31:34 +0300, anatoly techtonik <techtonik at gmail.com> wrote:
> Could anybody reopen http://bugs.python.org/issue8766 ? I can't.
> Reproducible 100% with Python 3.2 and 3.3 (3.1 didn't test).
> 
> > set PYTHONHOME=C:\
> > python
> 
> BTW, what is the role of PYTHONPATH on Windows?
> Is it a path for %INSTALLDIR%\Lib\site-packages?
> --
> anatoly t.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/rdmurray%40bitdance.com

From christian at python.org  Mon Oct 22 15:37:29 2012
From: christian at python.org (Christian Heimes)
Date: Mon, 22 Oct 2012 15:37:29 +0200
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH on
	Windows
In-Reply-To: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
References: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
Message-ID: <50854C19.6040406@python.org>

Am 22.10.2012 15:31, schrieb anatoly techtonik:
> Could anybody reopen http://bugs.python.org/issue8766 ? I can't.
> Reproducible 100% with Python 3.2 and 3.3 (3.1 didn't test).
> 
>> set PYTHONHOME=C:\
>> python

The segfault isn't caused by faulty code but by a call to abort() in
Py_FatalError(). The interpreter fails with an hard error because it is
unable to locate the encodings package.


$ PYTHONHOME=/ python3.2
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named encodings
Abgebrochen (Speicherabzug geschrieben)

$ gdb /usr/bin/python3.2 core
[...]
(gdb) bt
#0  0x00007fb05cd37445 in __GI_raise (sig=<optimized out>) at
../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x00007fb05cd3abab in __GI_abort () at abort.c:91
#2  0x0000000000482d0e in Py_FatalError ()
#3  0x0000000000482f14 in initfsencoding ()
#4  0x0000000000483e56 in Py_InitializeEx ()
#5  0x0000000000493e1a in Py_Main ()
#6  0x000000000041d307 in main ()


From victor.stinner at gmail.com  Mon Oct 22 15:40:20 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Mon, 22 Oct 2012 15:40:20 +0200
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH on
	Windows
In-Reply-To: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
References: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
Message-ID: <CAMpsgwY42sq1YJdQiDewM7S=wdX4LNF92j8szubL_Y0-xTZBsw@mail.gmail.com>

2012/10/22 anatoly techtonik <techtonik at gmail.com>:
> Could anybody reopen http://bugs.python.org/issue8766 ? I can't.
> Reproducible 100% with Python 3.2 and 3.3 (3.1 didn't test).
>
>> set PYTHONHOME=C:\
>> python

The issue #8766 is about PYTHONPATH environment variable, not
PYTHONHOME. Test on Linux with Python 3.4:

$ PYTHONHOME=/x ./python
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'
Abandon (core dumped)

It's not a segfault, but an "abort". It's different: abort() is called
by the application (Python) when something goes wrong. In this case,
Python is unable to locate the standard library. It's not a bug: I
asked Python to look in /x directory, whereas this directory doesn't
exist.

> BTW, what is the role of PYTHONPATH on Windows?
> Is it a path for %INSTALLDIR%\Lib\site-packages?

Same role than PYTHONPATH on Linux: specify where third party modules
are installed.

Victor

From phd at phdru.name  Mon Oct 22 15:42:57 2012
From: phd at phdru.name (Oleg Broytman)
Date: Mon, 22 Oct 2012 17:42:57 +0400
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH on
 Windows
In-Reply-To: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
References: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
Message-ID: <20121022134257.GA14907@iskra.aviel.ru>

On Mon, Oct 22, 2012 at 04:31:34PM +0300, anatoly techtonik <techtonik at gmail.com> wrote:
> Could anybody reopen http://bugs.python.org/issue8766 ? I can't.

   Even if you add a comment?

> BTW, what is the role of PYTHONPATH on Windows?

   The same as on Unix -- it's inserted into (but not replaced)
sys.path. And nothing else.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From martin at v.loewis.de  Mon Oct 22 15:59:30 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 22 Oct 2012 15:59:30 +0200
Subject: [Python-Dev] cpython: Build the _sha3 module with VS 2008.
In-Reply-To: <50845261.1080707@python.org>
References: <3Xl4PT03YszQt2@mail.python.org> <50841572.6040709@python.org>
	<50844C8F.8060706@v.loewis.de> <50845261.1080707@python.org>
Message-ID: <50855142.8090800@v.loewis.de>

Am 21.10.12 21:52, schrieb Christian Heimes:
> Ultimately it's your decision. You are the expert and build master for
> Windows releases. I've no hard feelings if you prefer to include the
> code in the main dll. Just say so and I will change the module ASAP.

It doesn't really matter much either way to me; the status quo is fine
(although I think a patch to Tools/msi/msi.py is still needed).

> We may have to ship our own SHA-3 code for quite a while. OpenSSL hasn't
> even began to include SHA-3.

Well... I find it cute to have SHA-3 implemented in Python, however, it 
probably wouldn't have hurt to only add support five years from now.

Regards,
Martin



From barry at python.org  Mon Oct 22 16:18:19 2012
From: barry at python.org (Barry Warsaw)
Date: Mon, 22 Oct 2012 10:18:19 -0400
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH on
 Windows
In-Reply-To: <CAMpsgwY42sq1YJdQiDewM7S=wdX4LNF92j8szubL_Y0-xTZBsw@mail.gmail.com>
References: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
	<CAMpsgwY42sq1YJdQiDewM7S=wdX4LNF92j8szubL_Y0-xTZBsw@mail.gmail.com>
Message-ID: <20121022101819.4c11acbc@limelight.wooz.org>

On Oct 22, 2012, at 03:40 PM, Victor Stinner wrote:

>The issue #8766 is about PYTHONPATH environment variable, not
>PYTHONHOME. Test on Linux with Python 3.4:
>
>$ PYTHONHOME=/x ./python
>Fatal Python error: Py_Initialize: Unable to get the locale encoding
>ImportError: No module named 'encodings'
>Abandon (core dumped)
>
>It's not a segfault, but an "abort". It's different: abort() is called
>by the application (Python) when something goes wrong. In this case,
>Python is unable to locate the standard library. It's not a bug: I
>asked Python to look in /x directory, whereas this directory doesn't
>exist.

We have this bug in Ubuntu:

https://bugs.launchpad.net/ubuntu/+source/python3.2/+bug/938869

This is caused by the VMware player installer playing games with $PYTHONHOME
to try to point it at its own *Python 2* runtime, and trying to call
lsb_release (a Python 3 script on Ubuntu 12.10) while $PYTHONHOME is pointing
at the wrong runtime.

I personally think this is a bug in the VMware player installer script.

-Barry

From christian at python.org  Mon Oct 22 16:30:23 2012
From: christian at python.org (Christian Heimes)
Date: Mon, 22 Oct 2012 16:30:23 +0200
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH on
	Windows
In-Reply-To: <20121022101819.4c11acbc@limelight.wooz.org>
References: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
	<CAMpsgwY42sq1YJdQiDewM7S=wdX4LNF92j8szubL_Y0-xTZBsw@mail.gmail.com>
	<20121022101819.4c11acbc@limelight.wooz.org>
Message-ID: <5085587F.6050001@python.org>

Am 22.10.2012 16:18, schrieb Barry Warsaw:
> This is caused by the VMware player installer playing games with $PYTHONHOME
> to try to point it at its own *Python 2* runtime, and trying to call
> lsb_release (a Python 3 script on Ubuntu 12.10) while $PYTHONHOME is pointing
> at the wrong runtime.
> 
> I personally think this is a bug in the VMware player installer script.

I think it's a bug in lsb_release, too. It should use the -E option in
its shebang:

with "#!/usr/bin/python"

$ PYTHONHOME=/invalid /usr/bin/lsb_release
ImportError: No module named site

with "#!/usr/bin/python -E"

$ PYTHONHOME=/invalid /usr/bin/lsb_release
LSB Version:
core-2.0-amd64:core-2.0-noarch:core-3.0-amd64:core-3.0-noarch:core-3.1-amd64:core-3.1-noarch:core-3.2-amd64:core-3.2-noarch:core-4.0-amd64:core-4.0-noarch

From barry at python.org  Mon Oct 22 16:43:51 2012
From: barry at python.org (Barry Warsaw)
Date: Mon, 22 Oct 2012 10:43:51 -0400
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH on
	Windows
In-Reply-To: <5085587F.6050001@python.org>
References: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
	<CAMpsgwY42sq1YJdQiDewM7S=wdX4LNF92j8szubL_Y0-xTZBsw@mail.gmail.com>
	<20121022101819.4c11acbc@limelight.wooz.org>
	<5085587F.6050001@python.org>
Message-ID: <20121022104351.058f2650@limelight.wooz.org>

On Oct 22, 2012, at 04:30 PM, Christian Heimes wrote:

>I think it's a bug in lsb_release, too. It should use the -E option in
>its shebang:

Indeed, thanks!  I'd forgotten about -E.

(bug updated btw)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121022/b95779c0/attachment.pgp>

From christian at python.org  Mon Oct 22 16:47:33 2012
From: christian at python.org (Christian Heimes)
Date: Mon, 22 Oct 2012 16:47:33 +0200
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH on
	Windows
In-Reply-To: <20121022104351.058f2650@limelight.wooz.org>
References: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
	<CAMpsgwY42sq1YJdQiDewM7S=wdX4LNF92j8szubL_Y0-xTZBsw@mail.gmail.com>
	<20121022101819.4c11acbc@limelight.wooz.org>
	<5085587F.6050001@python.org>
	<20121022104351.058f2650@limelight.wooz.org>
Message-ID: <50855C85.9040102@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Am 22.10.2012 16:43, schrieb Barry Warsaw:
> On Oct 22, 2012, at 04:30 PM, Christian Heimes wrote:
> 
>> I think it's a bug in lsb_release, too. It should use the -E
>> option in its shebang:
> 
> Indeed, thanks!  I'd forgotten about -E.
> 
> (bug updated btw)

You might want -s, too.

- -E     : ignore PYTHON* environment variables (such as PYTHONPATH)
- -s     : don't add user site directory to sys.path

- -E -s are useful for system commands and mandatory for suid commands.

Christian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iQIcBAEBCAAGBQJQhVx9AAoJEMeIxMHUVQ1F3YEP/RUP6jp0I73zhXKX218PTW3C
1TFqe3V+1hoItaG3wAxTtGFfhlNfiQ2P0cJxXz3XecqKR5N23tC1ESn+sjHBfCtS
rzB4d8h3F3BO1r15NzTnEKiZD6SxMAtiPksd6HQtwcAiPmiSbKded1v8WS2nQfKV
xaz05NSEkbsG09Cj4DIGx+Vb7dJb3y+j9oVxI/8EQijURfSXXqx10x3mTy3upH6c
3LK+qtPbnrC3OFdHVKnZcAcHTNXzALs3vagcye04I3Jes927y6SdboEx77kptRbm
rDmMQyRbvQgsYP+tXhJvJg8zf68t7F20pP4ato+FkYlaJJjkDXeSloGD8x8KSGvP
P5QHmeMjFCazrKIjYNH00mLLUlW7cMZzapPvJP/sUaMFSVn+eEsU7L0Wj2fxxIsu
s7kBAvso7t8Ins3OP2vn+RbFfo1wj8FV7DtBQoPo0MAjXKLZToaE7hCrqx0pBEmc
nodQxgwaJXsRQAx4n4ZpfRCzqeUTIlgbvNpf7UPTEIGaWz6J98FZe5I3Gpa9VUDR
h40lHvHoCy7UypmBYAvBh6q/EamSrpanUPIECHaxKqiehnUeChUz67A5hsIQ482z
7frVmgTcLHn8Q8+XlyXbCy6nBNU1Jpgy3yWbybPPsrDZWSJVtpdfgSURkimDZ2wG
tRSwF3gWtTTU4te2NkYP
=Suu/
-----END PGP SIGNATURE-----

From techtonik at gmail.com  Mon Oct 22 18:26:15 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 22 Oct 2012 19:26:15 +0300
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH on
	Windows
In-Reply-To: <CAMpsgwY42sq1YJdQiDewM7S=wdX4LNF92j8szubL_Y0-xTZBsw@mail.gmail.com>
References: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
	<CAMpsgwY42sq1YJdQiDewM7S=wdX4LNF92j8szubL_Y0-xTZBsw@mail.gmail.com>
Message-ID: <CAPkN8xJgEAVLM1D8Sgoq59Z7b2W-pRngWzD4bTqa7S09PNPpsw@mail.gmail.com>

On Mon, Oct 22, 2012 at 4:40 PM, Victor Stinner
<victor.stinner at gmail.com> wrote:
>>
>>> set PYTHONHOME=C:\
>>> python
>
> The issue #8766 is about PYTHONPATH environment variable, not
> PYTHONHOME. Test on Linux with Python 3.4:
>
> $ PYTHONHOME=/x ./python
> Fatal Python error: Py_Initialize: Unable to get the locale encoding
> ImportError: No module named 'encodings'
> Abandon (core dumped)
>
> It's not a segfault, but an "abort". It's different: abort() is called
> by the application (Python) when something goes wrong. In this case,
> Python is unable to locate the standard library. It's not a bug: I
> asked Python to look in /x directory, whereas this directory doesn't
> exist.

I don't know what is abort() on Linux, but I believe coredumps is not
something you want to get while setting some environment variable. On
Windows it outputs a standard crash dialog box, which immediately
raises questions about Python stability and potential exploitability
in this direction.

>> BTW, what is the role of PYTHONPATH on Windows?
>> Is it a path for %INSTALLDIR%\Lib\site-packages?
>
> Same role than PYTHONPATH on Linux: specify where third party modules
> are installed.

I meant PYTHONHOME, sorry.

From christian at python.org  Mon Oct 22 18:37:57 2012
From: christian at python.org (Christian Heimes)
Date: Mon, 22 Oct 2012 18:37:57 +0200
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH on
	Windows
In-Reply-To: <CAPkN8xJgEAVLM1D8Sgoq59Z7b2W-pRngWzD4bTqa7S09PNPpsw@mail.gmail.com>
References: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
	<CAMpsgwY42sq1YJdQiDewM7S=wdX4LNF92j8szubL_Y0-xTZBsw@mail.gmail.com>
	<CAPkN8xJgEAVLM1D8Sgoq59Z7b2W-pRngWzD4bTqa7S09PNPpsw@mail.gmail.com>
Message-ID: <50857665.6010909@python.org>

Am 22.10.2012 18:26, schrieb anatoly techtonik:
> I don't know what is abort() on Linux, but I believe coredumps is not
> something you want to get while setting some environment variable. On
> Windows it outputs a standard crash dialog box, which immediately
> raises questions about Python stability and potential exploitability
> in this direction.

abort() is a C stdlib function that kills the current process with
SGIABRT or similar. It's designed to draw attention to a fatal error.

Are you proposing that Python should rather use _exit() than abort()
here? Both forcedly shut down the process immediately.

Christian


From dholth at gmail.com  Mon Oct 22 18:51:19 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 22 Oct 2012 12:51:19 -0400
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
Message-ID: <CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>

On Thu, Oct 18, 2012 at 3:37 PM, Daniel Holth <dholth at gmail.com> wrote:
> On Thu, Oct 18, 2012 at 2:21 PM,  <martin at v.loewis.de> wrote:
>> I'm -1 on the usage of ed25519 in PEP 427. While the PEP proposes to use
>> JSON
>> Web signatures, this algorithm is not supported by the current JWS draft
>> [1].
>>
>> Instead, I suggest to use the ES256 algorithm from JWS, i.e. ECDSA with the
>> NIST P-256 curve and SHA-256. This has the advantage of using standard
>> algorithms [2].
>>
>> I don't know what the rationale for suggesting ed25519 is; I suppose that
>> existence of a pure-Python implementation played a role. However:
>> - ECDSA also has a pure-Python implementation
>> - ECDSA is well-supported by OpenSSL, i.e. a signature generator may also
>>   invoke the OpenSSL command line for efficient implementation. I believe
>>   M2Crypto also exposes enough of OpenSSL tp perform ECDSA signing and
>>   verification.

Added a FAQ entry:

Why are you using Ed25519 and JWS instead of PGP, S/MIME, or ECDSA?
    Wheel's signing scheme is designed to protect against cryptography
    that is not used.  Wheel tries to encourage signing by making it very
    fast and easy.  Signature verification is encouraged by including
    the signature in the archive itself rather than making it a separate
    download, and by including a Python implementation of the entire
    signing system in the reference implementation.

    JWS and Ed25519 yield small, pure-Python implementations.  Ed25519
    is fast enough that public-key cryptography can be considered for
    applications where it was traditionally too slow to be used, so
    wheels can be signed without worrying about performance.  In Ed25519,
    unlike ECDSA, only key generation, but not signing, depends on
    a continuing high-quality source of entropy.  The combination of
    increased performance, convenience, and availability compared to
    using a separate program means digital signatures can always be
    enabled in wheel.

    Wheel uses simplified keys and a signature system where key generation
    is about as fast as signing, making it possible to consider signing
    keys an abundant resource.  Keys could represent a build server or
    a package rather than the publisher's entire digital identity.

From dholth at gmail.com  Mon Oct 22 18:53:03 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 22 Oct 2012 12:53:03 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
Message-ID: <CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>

http://hg.python.org/peps/rev/50e8ea1a17a0

Based on the other "required" field's absence in the wild, only
"Metadata-Version", "Name", "Version", and "Summary" are required.
Hopefully a clearer explanation of 0, 1, or many occurrences of each
field.

From solipsis at pitrou.net  Mon Oct 22 21:07:46 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 22 Oct 2012 21:07:46 +0200
Subject: [Python-Dev] PEP 427 comment: code signing
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
Message-ID: <20121022210746.15d18874@pitrou.net>

On Mon, 22 Oct 2012 12:51:19 -0400
Daniel Holth <dholth at gmail.com> wrote:
> On Thu, Oct 18, 2012 at 3:37 PM, Daniel Holth <dholth at gmail.com> wrote:
> > On Thu, Oct 18, 2012 at 2:21 PM,  <martin at v.loewis.de> wrote:
> >> I'm -1 on the usage of ed25519 in PEP 427. While the PEP proposes to use
> >> JSON
> >> Web signatures, this algorithm is not supported by the current JWS draft
> >> [1].
> >>
> >> Instead, I suggest to use the ES256 algorithm from JWS, i.e. ECDSA with the
> >> NIST P-256 curve and SHA-256. This has the advantage of using standard
> >> algorithms [2].
> >>
> >> I don't know what the rationale for suggesting ed25519 is; I suppose that
> >> existence of a pure-Python implementation played a role. However:
> >> - ECDSA also has a pure-Python implementation
> >> - ECDSA is well-supported by OpenSSL, i.e. a signature generator may also
> >>   invoke the OpenSSL command line for efficient implementation. I believe
> >>   M2Crypto also exposes enough of OpenSSL tp perform ECDSA signing and
> >>   verification.
> 
> Added a FAQ entry:
> 
> Why are you using Ed25519 and JWS instead of PGP, S/MIME, or ECDSA?
>     Wheel's signing scheme is designed to protect against cryptography
>     that is not used.  Wheel tries to encourage signing by making it very
>     fast and easy.  Signature verification is encouraged by including
>     the signature in the archive itself rather than making it a separate
>     download, and by including a Python implementation of the entire
>     signing system in the reference implementation.

Can you explain how the RECORD.jws file is obtained (and, if possible,
post an example of how it looks like)?
Specifically, which data is signed exactly? The binary contents of the
RECORD file? Something else?

Thanks

Antoine.



From dholth at gmail.com  Mon Oct 22 21:20:01 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 22 Oct 2012 15:20:01 -0400
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <20121022210746.15d18874@pitrou.net>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
	<20121022210746.15d18874@pitrou.net>
Message-ID: <CAG8k2+6T8WLKYK11OE4NMDAb=2F4CYCoqUyAh4uj9fyhwvVm4Q@mail.gmail.com>

On Mon, Oct 22, 2012 at 3:07 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 22 Oct 2012 12:51:19 -0400
> Daniel Holth <dholth at gmail.com> wrote:
>> On Thu, Oct 18, 2012 at 3:37 PM, Daniel Holth <dholth at gmail.com> wrote:
>> > On Thu, Oct 18, 2012 at 2:21 PM,  <martin at v.loewis.de> wrote:
>> >> I'm -1 on the usage of ed25519 in PEP 427. While the PEP proposes to use
>> >> JSON
>> >> Web signatures, this algorithm is not supported by the current JWS draft
>> >> [1].
>> >>
>> >> Instead, I suggest to use the ES256 algorithm from JWS, i.e. ECDSA with the
>> >> NIST P-256 curve and SHA-256. This has the advantage of using standard
>> >> algorithms [2].
>> >>
>> >> I don't know what the rationale for suggesting ed25519 is; I suppose that
>> >> existence of a pure-Python implementation played a role. However:
>> >> - ECDSA also has a pure-Python implementation
>> >> - ECDSA is well-supported by OpenSSL, i.e. a signature generator may also
>> >>   invoke the OpenSSL command line for efficient implementation. I believe
>> >>   M2Crypto also exposes enough of OpenSSL tp perform ECDSA signing and
>> >>   verification.
>>
>> Added a FAQ entry:
>>
>> Why are you using Ed25519 and JWS instead of PGP, S/MIME, or ECDSA?
>>     Wheel's signing scheme is designed to protect against cryptography
>>     that is not used.  Wheel tries to encourage signing by making it very
>>     fast and easy.  Signature verification is encouraged by including
>>     the signature in the archive itself rather than making it a separate
>>     download, and by including a Python implementation of the entire
>>     signing system in the reference implementation.
>
> Can you explain how the RECORD.jws file is obtained (and, if possible,
> post an example of how it looks like)?
> Specifically, which data is signed exactly? The binary contents of the
> RECORD file? Something else?

The decoded contents are like the JSON documents at
http://www.python.org/dev/peps/pep-0427/#json-web-signatures-extensions

Signing is implemented at:
https://bitbucket.org/dholth/wheel/src/tip/wheel/signatures/__init__.py?at=default#cl-25

The SHA-256 hash of RECORD is what is signed together with JWS
signature header. The JWS spec elaborates on the general format.

From techtonik at gmail.com  Mon Oct 22 21:28:26 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 22 Oct 2012 22:28:26 +0300
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH on
	Windows
In-Reply-To: <50857665.6010909@python.org>
References: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
	<CAMpsgwY42sq1YJdQiDewM7S=wdX4LNF92j8szubL_Y0-xTZBsw@mail.gmail.com>
	<CAPkN8xJgEAVLM1D8Sgoq59Z7b2W-pRngWzD4bTqa7S09PNPpsw@mail.gmail.com>
	<50857665.6010909@python.org>
Message-ID: <CAPkN8xLxTAabDtk-TCF4WQBw_6KtMqoQteJui_Kg9fEnHzZSyA@mail.gmail.com>

On Mon, Oct 22, 2012 at 7:37 PM, Christian Heimes <christian at python.org> wrote:
> Am 22.10.2012 18:26, schrieb anatoly techtonik:
>> I don't know what is abort() on Linux, but I believe coredumps is not
>> something you want to get while setting some environment variable. On
>> Windows it outputs a standard crash dialog box, which immediately
>> raises questions about Python stability and potential exploitability
>> in this direction.
>
> abort() is a C stdlib function that kills the current process with
> SGIABRT or similar. It's designed to draw attention to a fatal error.
>
> Are you proposing that Python should rather use _exit() than abort()
> here? Both forcedly shut down the process immediately.

I am not a C coder and don't have any core Unix programming
background. If Python is unable to start because it can not find its
libraries, I prefer an explanative error message with standard system
error code. Even if it is Fatal Python error - this case is still in
user land and should be fixed normally. The error message could be
improved though. Right now I get:

E:\>python
Fatal Python error: Py_Initialize: unable to load the file system codec
ImportError: No module named 'encodings'

This could be improved to:

Fatal Python error: Py_Initialize: unable to find module named 'encodings'
in 'C:\'

--
anatoly t.

From solipsis at pitrou.net  Mon Oct 22 21:27:12 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 22 Oct 2012 21:27:12 +0200
Subject: [Python-Dev] PEP 427: pure / plat distinction
Message-ID: <20121022212712.3312cec2@pitrou.net>


Hello,

How does the pure / plat distinction as outlined in PEP 427 cope with
Debian's system of separating installed files into pyshared (for *.py
and *.egg-info files) and pythonX.Y/dist-packages (for *.pyc and *.so
files)?

Thanks,

Antoine.



From solipsis at pitrou.net  Mon Oct 22 21:37:00 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 22 Oct 2012 21:37:00 +0200
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <CAG8k2+6T8WLKYK11OE4NMDAb=2F4CYCoqUyAh4uj9fyhwvVm4Q@mail.gmail.com>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
	<20121022210746.15d18874@pitrou.net>
	<CAG8k2+6T8WLKYK11OE4NMDAb=2F4CYCoqUyAh4uj9fyhwvVm4Q@mail.gmail.com>
Message-ID: <20121022213700.1985758c@pitrou.net>

On Mon, 22 Oct 2012 15:20:01 -0400
Daniel Holth <dholth at gmail.com> wrote:
> 
> The decoded contents are like the JSON documents at
> http://www.python.org/dev/peps/pep-0427/#json-web-signatures-extensions
> 
> Signing is implemented at:
> https://bitbucket.org/dholth/wheel/src/tip/wheel/signatures/__init__.py?at=default#cl-25
> 
> The SHA-256 hash of RECORD is what is signed together with JWS
> signature header. The JWS spec elaborates on the general format.

Thank you. Could you fix the terminology in the PEP? You are using the
term "payload" in a different sense from the JWS draft. Specifically,
the PEP should mention that the "JWS Payload" is the binary
contents of the RECORD file.

What you are calling payload is actually the "JWS Signature".

Regards

Antoine.

From dholth at gmail.com  Mon Oct 22 21:39:07 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 22 Oct 2012 15:39:07 -0400
Subject: [Python-Dev] PEP 427: pure / plat distinction
In-Reply-To: <20121022212712.3312cec2@pitrou.net>
References: <20121022212712.3312cec2@pitrou.net>
Message-ID: <CAG8k2+7bo3hcS6MCwc1e9+Nm=a22r4AcLSn_WULUE0yhmiHBpA@mail.gmail.com>

On Mon, Oct 22, 2012 at 3:27 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Hello,
>
> How does the pure / plat distinction as outlined in PEP 427 cope with
> Debian's system of separating installed files into pyshared (for *.py
> and *.egg-info files) and pythonX.Y/dist-packages (for *.pyc and *.so
> files)?
>
> Thanks,
>
> Antoine.

It is only there because distutils has a purelib/platlib distinction,
so the spec represents it. See
http://hg.python.org/cpython/file/49de26395d1a/Lib/distutils/command/install.py#l85

From dholth at gmail.com  Mon Oct 22 21:49:34 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 22 Oct 2012 15:49:34 -0400
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <20121022213700.1985758c@pitrou.net>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
	<20121022210746.15d18874@pitrou.net>
	<CAG8k2+6T8WLKYK11OE4NMDAb=2F4CYCoqUyAh4uj9fyhwvVm4Q@mail.gmail.com>
	<20121022213700.1985758c@pitrou.net>
Message-ID: <CAG8k2+44LK4A_shudEvWGk0u+5XxhWQ+msm48q+rsd3WwUVHYw@mail.gmail.com>

On Mon, Oct 22, 2012 at 3:37 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 22 Oct 2012 15:20:01 -0400
> Daniel Holth <dholth at gmail.com> wrote:
>>
>> The decoded contents are like the JSON documents at
>> http://www.python.org/dev/peps/pep-0427/#json-web-signatures-extensions
>>
>> Signing is implemented at:
>> https://bitbucket.org/dholth/wheel/src/tip/wheel/signatures/__init__.py?at=default#cl-25
>>
>> The SHA-256 hash of RECORD is what is signed together with JWS
>> signature header. The JWS spec elaborates on the general format.
>
> Thank you. Could you fix the terminology in the PEP? You are using the
> term "payload" in a different sense from the JWS draft. Specifically,
> the PEP should mention that the "JWS Payload" is the binary
> contents of the RECORD file.
>
> What you are calling payload is actually the "JWS Signature".
>
> Regards

Which line is confusing? The payload is the hash of the contents of
RECORD as a small JSON document: { "hash":
"sha256=ADD-r2urObZHcxBW3Cr-vDCu5RJwT4CaRTHiFmbcIYY" } instead of
including a base64-encoded copy of RECORD in the signature.

From solipsis at pitrou.net  Mon Oct 22 22:12:25 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 22 Oct 2012 22:12:25 +0200
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <CAG8k2+44LK4A_shudEvWGk0u+5XxhWQ+msm48q+rsd3WwUVHYw@mail.gmail.com>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
	<20121022210746.15d18874@pitrou.net>
	<CAG8k2+6T8WLKYK11OE4NMDAb=2F4CYCoqUyAh4uj9fyhwvVm4Q@mail.gmail.com>
	<20121022213700.1985758c@pitrou.net>
	<CAG8k2+44LK4A_shudEvWGk0u+5XxhWQ+msm48q+rsd3WwUVHYw@mail.gmail.com>
Message-ID: <20121022221225.67866ca3@pitrou.net>

On Mon, 22 Oct 2012 15:49:34 -0400
Daniel Holth <dholth at gmail.com> wrote:
> On Mon, Oct 22, 2012 at 3:37 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > On Mon, 22 Oct 2012 15:20:01 -0400
> > Daniel Holth <dholth at gmail.com> wrote:
> >>
> >> The decoded contents are like the JSON documents at
> >> http://www.python.org/dev/peps/pep-0427/#json-web-signatures-extensions
> >>
> >> Signing is implemented at:
> >> https://bitbucket.org/dholth/wheel/src/tip/wheel/signatures/__init__.py?at=default#cl-25
> >>
> >> The SHA-256 hash of RECORD is what is signed together with JWS
> >> signature header. The JWS spec elaborates on the general format.
> >
> > Thank you. Could you fix the terminology in the PEP? You are using the
> > term "payload" in a different sense from the JWS draft. Specifically,
> > the PEP should mention that the "JWS Payload" is the binary
> > contents of the RECORD file.
> >
> > What you are calling payload is actually the "JWS Signature".
> >
> > Regards
> 
> Which line is confusing? The payload is the hash of the contents of
> RECORD as a small JSON document: { "hash":
> "sha256=ADD-r2urObZHcxBW3Cr-vDCu5RJwT4CaRTHiFmbcIYY" } instead of
> including a base64-encoded copy of RECORD in the signature.

Thanks for the explanation. Can you add it to the PEP?

In your JWS header example:

{
 "alg": "Ed25519",
 "typ": "JWT",
 "key": {
 "alg": "Ed25519",
 "vk": "tmAYCrSfj8gtJ10v3VkvW7jOndKmQIYE12hgnFu3cvk"
 }
}

Why are you using "key" instead of "jwk" for the JSON Web Key?

Regards

Antoine.

From solipsis at pitrou.net  Mon Oct 22 22:13:39 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 22 Oct 2012 22:13:39 +0200
Subject: [Python-Dev] PEP 427: pure / plat distinction
In-Reply-To: <CAG8k2+7bo3hcS6MCwc1e9+Nm=a22r4AcLSn_WULUE0yhmiHBpA@mail.gmail.com>
References: <20121022212712.3312cec2@pitrou.net>
	<CAG8k2+7bo3hcS6MCwc1e9+Nm=a22r4AcLSn_WULUE0yhmiHBpA@mail.gmail.com>
Message-ID: <20121022221339.2c522d17@pitrou.net>

On Mon, 22 Oct 2012 15:39:07 -0400
Daniel Holth <dholth at gmail.com> wrote:
> On Mon, Oct 22, 2012 at 3:27 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> >
> > Hello,
> >
> > How does the pure / plat distinction as outlined in PEP 427 cope with
> > Debian's system of separating installed files into pyshared (for *.py
> > and *.egg-info files) and pythonX.Y/dist-packages (for *.pyc and *.so
> > files)?
> >
> > Thanks,
> >
> > Antoine.
> 
> It is only there because distutils has a purelib/platlib distinction,
> so the spec represents it. See
> http://hg.python.org/cpython/file/49de26395d1a/Lib/distutils/command/install.py#l85

Ok, so does this mean Debian would have to build their own installer if
they want to spread the wheel's contents as appropriate for their
directory layout?

Regards

Antoine.

From dholth at gmail.com  Mon Oct 22 22:18:50 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 22 Oct 2012 16:18:50 -0400
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <20121022221225.67866ca3@pitrou.net>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
	<20121022210746.15d18874@pitrou.net>
	<CAG8k2+6T8WLKYK11OE4NMDAb=2F4CYCoqUyAh4uj9fyhwvVm4Q@mail.gmail.com>
	<20121022213700.1985758c@pitrou.net>
	<CAG8k2+44LK4A_shudEvWGk0u+5XxhWQ+msm48q+rsd3WwUVHYw@mail.gmail.com>
	<20121022221225.67866ca3@pitrou.net>
Message-ID: <CAG8k2+5bUr4KXAMsLag04rsVL9NUtqWHurM050Eb390xNKc6Dw@mail.gmail.com>

On Mon, Oct 22, 2012 at 4:12 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 22 Oct 2012 15:49:34 -0400
> Daniel Holth <dholth at gmail.com> wrote:
>> On Mon, Oct 22, 2012 at 3:37 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> > On Mon, 22 Oct 2012 15:20:01 -0400
>> > Daniel Holth <dholth at gmail.com> wrote:
>> >>
>> >> The decoded contents are like the JSON documents at
>> >> http://www.python.org/dev/peps/pep-0427/#json-web-signatures-extensions
>> >>
>> >> Signing is implemented at:
>> >> https://bitbucket.org/dholth/wheel/src/tip/wheel/signatures/__init__.py?at=default#cl-25
>> >>
>> >> The SHA-256 hash of RECORD is what is signed together with JWS
>> >> signature header. The JWS spec elaborates on the general format.
>> >
>> > Thank you. Could you fix the terminology in the PEP? You are using the
>> > term "payload" in a different sense from the JWS draft. Specifically,
>> > the PEP should mention that the "JWS Payload" is the binary
>> > contents of the RECORD file.
>> >
>> > What you are calling payload is actually the "JWS Signature".
>> >
>> > Regards
>>
>> Which line is confusing? The payload is the hash of the contents of
>> RECORD as a small JSON document: { "hash":
>> "sha256=ADD-r2urObZHcxBW3Cr-vDCu5RJwT4CaRTHiFmbcIYY" } instead of
>> including a base64-encoded copy of RECORD in the signature.
>
> Thanks for the explanation. Can you add it to the PEP?
>
> In your JWS header example:
>
> {
>  "alg": "Ed25519",
>  "typ": "JWT",
>  "key": {
>  "alg": "Ed25519",
>  "vk": "tmAYCrSfj8gtJ10v3VkvW7jOndKmQIYE12hgnFu3cvk"
>  }
> }
>
> Why are you using "key" instead of "jwk" for the JSON Web Key?

bug.

From solipsis at pitrou.net  Mon Oct 22 22:20:22 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 22 Oct 2012 22:20:22 +0200
Subject: [Python-Dev] PEP 427: data directory
Message-ID: <20121022222022.6bd940c1@pitrou.net>


Hello,

The FAQ has this weird statement:

?This specification does not have an opinion on how you should organize
your code. The .data directory is just a place for any files that are
not normally installed inside site-packages or on the PYTHONPATH.?

But, say, if I want to install some init script to /etc/init.d by using
distutils' data_files argument:

setup.py(...,
    data_files=[('/etc/init.d', ['my-init-script'])]
    )

How is it stored and represented by the wheel format?

Regards

Antoine.



From dholth at gmail.com  Mon Oct 22 22:25:36 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 22 Oct 2012 16:25:36 -0400
Subject: [Python-Dev] PEP 427: pure / plat distinction
In-Reply-To: <20121022221339.2c522d17@pitrou.net>
References: <20121022212712.3312cec2@pitrou.net>
	<CAG8k2+7bo3hcS6MCwc1e9+Nm=a22r4AcLSn_WULUE0yhmiHBpA@mail.gmail.com>
	<20121022221339.2c522d17@pitrou.net>
Message-ID: <CAG8k2+7uycyy-6NSqmD0XYt2oAu849eG4obubd7_9_9Tem-v_Q@mail.gmail.com>

On Mon, Oct 22, 2012 at 4:13 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 22 Oct 2012 15:39:07 -0400
> Daniel Holth <dholth at gmail.com> wrote:
>> On Mon, Oct 22, 2012 at 3:27 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> >
>> > Hello,
>> >
>> > How does the pure / plat distinction as outlined in PEP 427 cope with
>> > Debian's system of separating installed files into pyshared (for *.py
>> > and *.egg-info files) and pythonX.Y/dist-packages (for *.pyc and *.so
>> > files)?
>> >
>> > Thanks,
>> >
>> > Antoine.
>>
>> It is only there because distutils has a purelib/platlib distinction,
>> so the spec represents it. See
>> http://hg.python.org/cpython/file/49de26395d1a/Lib/distutils/command/install.py#l85
>
> Ok, so does this mean Debian would have to build their own installer if
> they want to spread the wheel's contents as appropriate for their
> directory layout?

Sorry if this is a lame observation, but Debian has already built
their own installer called dpkg.

The Python-specific binary format will mainly be used to populate
virtualenvs. It may also turn out to be a convenient intermediate step
to build an .rpm or .deb but that is not the primary design goal.

From dholth at gmail.com  Mon Oct 22 22:26:57 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 22 Oct 2012 16:26:57 -0400
Subject: [Python-Dev] PEP 427: data directory
In-Reply-To: <20121022222022.6bd940c1@pitrou.net>
References: <20121022222022.6bd940c1@pitrou.net>
Message-ID: <CAG8k2+7m6yhL5SO1WU12LyppfBMYYmivetVW7m-DuTz0Fc9SEA@mail.gmail.com>

On Mon, Oct 22, 2012 at 4:20 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Hello,
>
> The FAQ has this weird statement:
>
> ?This specification does not have an opinion on how you should organize
> your code. The .data directory is just a place for any files that are
> not normally installed inside site-packages or on the PYTHONPATH.?
>
> But, say, if I want to install some init script to /etc/init.d by using
> distutils' data_files argument:
>
> setup.py(...,
>     data_files=[('/etc/init.d', ['my-init-script'])]
>     )
>
> How is it stored and represented by the wheel format?
>
> Regards
>
> Antoine.

Distutils' "data" category in a wheel package-1.0.data/data/ directory
usually maps to / (or the root of the virtualenv). Your init script
would probably wind up in there.

From solipsis at pitrou.net  Mon Oct 22 22:27:46 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 22 Oct 2012 22:27:46 +0200
Subject: [Python-Dev] PEP 427: pure / plat distinction
In-Reply-To: <CAG8k2+7uycyy-6NSqmD0XYt2oAu849eG4obubd7_9_9Tem-v_Q@mail.gmail.com>
References: <20121022212712.3312cec2@pitrou.net>
	<CAG8k2+7bo3hcS6MCwc1e9+Nm=a22r4AcLSn_WULUE0yhmiHBpA@mail.gmail.com>
	<20121022221339.2c522d17@pitrou.net>
	<CAG8k2+7uycyy-6NSqmD0XYt2oAu849eG4obubd7_9_9Tem-v_Q@mail.gmail.com>
Message-ID: <1350937666.3366.15.camel@localhost.localdomain>

Le lundi 22 octobre 2012 ? 16:25 -0400, Daniel Holth a ?crit :
> On Mon, Oct 22, 2012 at 4:13 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > On Mon, 22 Oct 2012 15:39:07 -0400
> > Daniel Holth <dholth at gmail.com> wrote:
> >> On Mon, Oct 22, 2012 at 3:27 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> >> >
> >> > Hello,
> >> >
> >> > How does the pure / plat distinction as outlined in PEP 427 cope with
> >> > Debian's system of separating installed files into pyshared (for *.py
> >> > and *.egg-info files) and pythonX.Y/dist-packages (for *.pyc and *.so
> >> > files)?
> >> >
> >> > Thanks,
> >> >
> >> > Antoine.
> >>
> >> It is only there because distutils has a purelib/platlib distinction,
> >> so the spec represents it. See
> >> http://hg.python.org/cpython/file/49de26395d1a/Lib/distutils/command/install.py#l85
> >
> > Ok, so does this mean Debian would have to build their own installer if
> > they want to spread the wheel's contents as appropriate for their
> > directory layout?
> 
> Sorry if this is a lame observation, but Debian has already built
> their own installer called dpkg.

Right :)

> The Python-specific binary format will mainly be used to populate
> virtualenvs. It may also turn out to be a convenient intermediate step
> to build an .rpm or .deb but that is not the primary design goal.

Well, I would expect people to also use it for regular installations,
especially under Windows (thanks to the ABI tag for extension modules).

Regards

Antoine.



From solipsis at pitrou.net  Mon Oct 22 22:28:39 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 22 Oct 2012 22:28:39 +0200
Subject: [Python-Dev] PEP 427: data directory
In-Reply-To: <CAG8k2+7m6yhL5SO1WU12LyppfBMYYmivetVW7m-DuTz0Fc9SEA@mail.gmail.com>
References: <20121022222022.6bd940c1@pitrou.net>
	<CAG8k2+7m6yhL5SO1WU12LyppfBMYYmivetVW7m-DuTz0Fc9SEA@mail.gmail.com>
Message-ID: <20121022222839.20f08147@pitrou.net>

On Mon, 22 Oct 2012 16:26:57 -0400
Daniel Holth <dholth at gmail.com> wrote:
> On Mon, Oct 22, 2012 at 4:20 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> >
> > Hello,
> >
> > The FAQ has this weird statement:
> >
> > ?This specification does not have an opinion on how you should organize
> > your code. The .data directory is just a place for any files that are
> > not normally installed inside site-packages or on the PYTHONPATH.?
> >
> > But, say, if I want to install some init script to /etc/init.d by using
> > distutils' data_files argument:
> >
> > setup.py(...,
> >     data_files=[('/etc/init.d', ['my-init-script'])]
> >     )
> >
> > How is it stored and represented by the wheel format?
> >
> > Regards
> >
> > Antoine.
> 
> Distutils' "data" category in a wheel package-1.0.data/data/ directory
> usually maps to / (or the root of the virtualenv). Your init script
> would probably wind up in there.

Ok, thanks. Might be nice to add as a FAQ entry :-)

Regards

Antoine.

From dholth at gmail.com  Mon Oct 22 22:35:25 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 22 Oct 2012 16:35:25 -0400
Subject: [Python-Dev] PEP 427: data directory
In-Reply-To: <CAG8k2+7m6yhL5SO1WU12LyppfBMYYmivetVW7m-DuTz0Fc9SEA@mail.gmail.com>
References: <20121022222022.6bd940c1@pitrou.net>
	<CAG8k2+7m6yhL5SO1WU12LyppfBMYYmivetVW7m-DuTz0Fc9SEA@mail.gmail.com>
Message-ID: <CAG8k2+7194=dxa=A1m6c+EiaFs93HpKq_+rEuUBK-xEX=aO2Bg@mail.gmail.com>

On Mon, Oct 22, 2012 at 4:26 PM, Daniel Holth <dholth at gmail.com> wrote:
> On Mon, Oct 22, 2012 at 4:20 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>
>> Hello,
>>
>> The FAQ has this weird statement:
>>
>> ?This specification does not have an opinion on how you should organize
>> your code. The .data directory is just a place for any files that are
>> not normally installed inside site-packages or on the PYTHONPATH.?

I mean to say that just because there is a .data/ directory it doesn't
mean you have to stop using pkgutil.get_data(package, resource). Some
people will take those 4 letters "data" to mean that any file that
doesn't start with .py has to go there instead of in the root of the
archive.

From martin at v.loewis.de  Tue Oct 23 00:20:27 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Tue, 23 Oct 2012 00:20:27 +0200
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
Message-ID: <20121023002027.Horde.-FSSUlNNcXdQhcarnobBE-A@webmail.df.eu>


Zitat von Daniel Holth <dholth at gmail.com>:

> Why are you using Ed25519 and JWS instead of PGP, S/MIME, or ECDSA?
>     Wheel's signing scheme is designed to protect against cryptography
>     that is not used.  Wheel tries to encourage signing by making it very
>     fast and easy.  Signature verification is encouraged by including
>     the signature in the archive itself rather than making it a separate
>     download, and by including a Python implementation of the entire
>     signing system in the reference implementation.
>
>     JWS and Ed25519 yield small, pure-Python implementations.  Ed25519
>     is fast enough that public-key cryptography can be considered for
>     applications where it was traditionally too slow to be used, so
>     wheels can be signed without worrying about performance.

I believe this analysis of reasons for not using cryptography is incorrect.
Speed never is an issue in deciding whether or not to use cryptographic
algorithms, today (except for cases with very limited CPU capabilities).
Instead, the primary reason for not choosing cryptography is ease-of-use.

For that reason, I still think that using an established algorithm would
be the better choice. I remain -1 on this choice.

Regards,
Martin



From dholth at gmail.com  Tue Oct 23 00:34:07 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 22 Oct 2012 18:34:07 -0400
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <20121023002027.Horde.-FSSUlNNcXdQhcarnobBE-A@webmail.df.eu>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
	<20121023002027.Horde.-FSSUlNNcXdQhcarnobBE-A@webmail.df.eu>
Message-ID: <CAG8k2+4DhSkvYtk807ZX78ear__pZCeSwNVO5FXzPtLcudSYQQ@mail.gmail.com>

On Mon, Oct 22, 2012 at 6:20 PM,  <martin at v.loewis.de> wrote:
>
> Zitat von Daniel Holth <dholth at gmail.com>:
>
>
>> Why are you using Ed25519 and JWS instead of PGP, S/MIME, or ECDSA?
>>     Wheel's signing scheme is designed to protect against cryptography
>>     that is not used.  Wheel tries to encourage signing by making it very
>>     fast and easy.  Signature verification is encouraged by including
>>     the signature in the archive itself rather than making it a separate
>>     download, and by including a Python implementation of the entire
>>     signing system in the reference implementation.
>>
>>     JWS and Ed25519 yield small, pure-Python implementations.  Ed25519
>>     is fast enough that public-key cryptography can be considered for
>>     applications where it was traditionally too slow to be used, so
>>     wheels can be signed without worrying about performance.
>
>
> I believe this analysis of reasons for not using cryptography is incorrect.
> Speed never is an issue in deciding whether or not to use cryptographic
> algorithms, today (except for cases with very limited CPU capabilities).
> Instead, the primary reason for not choosing cryptography is ease-of-use.
>
> For that reason, I still think that using an established algorithm would
> be the better choice. I remain -1 on this choice.

You are right that in this application, it probably doesn't matter. In
other applications like public key authentication for individual
packets the Ed25519 performance is necessary. A relevant advantage is
the deterministic signatures property; the Playstation lost their key
because they forgot to use randomness when generating ECDSA
signatures.

From turnbull at sk.tsukuba.ac.jp  Tue Oct 23 01:39:03 2012
From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Tue, 23 Oct 2012 08:39:03 +0900
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH
	on	Windows
In-Reply-To: <CAPkN8xLxTAabDtk-TCF4WQBw_6KtMqoQteJui_Kg9fEnHzZSyA@mail.gmail.com>
References: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
	<CAMpsgwY42sq1YJdQiDewM7S=wdX4LNF92j8szubL_Y0-xTZBsw@mail.gmail.com>
	<CAPkN8xJgEAVLM1D8Sgoq59Z7b2W-pRngWzD4bTqa7S09PNPpsw@mail.gmail.com>
	<50857665.6010909@python.org>
	<CAPkN8xLxTAabDtk-TCF4WQBw_6KtMqoQteJui_Kg9fEnHzZSyA@mail.gmail.com>
Message-ID: <87r4oq3wmg.fsf@uwakimon.sk.tsukuba.ac.jp>

anatoly techtonik writes:

 > I am not a C coder and don't have any core Unix programming
 > background. If Python is unable to start because it can not find its
 > libraries, I prefer an explanative error message with standard system
 > error code. Even if it is Fatal Python error - this case is still in
 > user land and should be fixed normally.

Python can't know that the problem is in "user land" as you call it.
For all Python knows, the directory containing its libraries is
corrupt, or that the file system code itself experienced a mysterious
glitch.

While in the technical sense the problem here is in userland, not in
the kernel, the distinction you're making is incoherent.  For most
users, it doesn't matter whether an environment variable is set
incorrectly without their knowledge, or if the kernel is buggy, or if
the disk is corrupt.  And from Python's point of view, the world as a
whole no longer makes.

So it shuts down abnormally.  That's what an abort means, in
programming as in rocket launches.  Users should be scared if this
happens; somebody really screwed up.  (Unless it's themselves, and
then they only have themselves to blame.)

 > The error message could be
 > improved though. Right now I get:
 > 
 > E:\>python
 > Fatal Python error: Py_Initialize: unable to load the file system codec
 > ImportError: No module named 'encodings'
 > 
 > This could be improved to:
 > 
 > Fatal Python error: Py_Initialize: unable to find module named 'encodings'
 > in 'C:\'

That may be an improvement.  But in that case it might be worth
explaining where "C:\" came from (in this case PYTHONHOME, I guess?)


From breamoreboy at yahoo.co.uk  Tue Oct 23 02:04:47 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 23 Oct 2012 01:04:47 +0100
Subject: [Python-Dev] Fwd: Python 3.3 can't sort memoryviews as they're
	unorderable
In-Reply-To: <k60lsp$jok$1@ger.gmane.org>
References: <k60lsp$jok$1@ger.gmane.org>
Message-ID: <k64mpo$uao$1@ger.gmane.org>


-------- Original Message --------
Subject: Python 3.3 can't sort memoryviews as they're unorderable
Date: Sun, 21 Oct 2012 12:24:32 +0100
From: Mark Lawrence <breamoreboy at yahoo.co.uk>
To: python-list at python.org
Newsgroups: gmane.comp.python.general

http://docs.python.org/dev/whatsnew/3.3.html states "memoryview
comparisons now use the logical structure of the operands and compare
all array elements by value".  So I'd have thought that you should be
able to compare them and hence sort them, but this is the state of play.

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600 32
bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> memoryview(bytearray(range(5))) == memoryview(bytearray(range(5)))
True
>>> memoryview(bytearray(range(5))) != memoryview(bytearray(range(5)))
False
>>> memoryview(bytearray(range(5))) < memoryview(bytearray(range(5)))
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: unorderable types: memoryview() < memoryview()

Okay then, let's subclass memoryview to provide the functionality.

>>> class Test(memoryview):
...     pass
...
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: type 'memoryview' is not an acceptable base type

Oh dear.
http://docs.python.org/py3k/library/stdtypes.html#typememoryview only
gives examples of equality comparisons and there was nothing that I
could see in PEP3118 to explain the rationale behind the lack of other
comparisons.  What have I missed?

Nobody on the main Python ml could answer this so can someone please 
explain the background to how memoryviews work in this instance as I'm 
confused, thanks.

Mark Lawrence.




From greg.ewing at canterbury.ac.nz  Tue Oct 23 02:04:56 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 23 Oct 2012 13:04:56 +1300
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH	on
 Windows
In-Reply-To: <87r4oq3wmg.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
	<CAMpsgwY42sq1YJdQiDewM7S=wdX4LNF92j8szubL_Y0-xTZBsw@mail.gmail.com>
	<CAPkN8xJgEAVLM1D8Sgoq59Z7b2W-pRngWzD4bTqa7S09PNPpsw@mail.gmail.com>
	<50857665.6010909@python.org>
	<CAPkN8xLxTAabDtk-TCF4WQBw_6KtMqoQteJui_Kg9fEnHzZSyA@mail.gmail.com>
	<87r4oq3wmg.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <5085DF28.5050304@canterbury.ac.nz>

Stephen J. Turnbull wrote:
> For most
> users, it doesn't matter whether an environment variable is set
> incorrectly without their knowledge, or if the kernel is buggy, or if
> the disk is corrupt.  And from Python's point of view, the world as a
> whole no longer makes.
> 
> So it shuts down abnormally.  That's what an abort means, in
> programming as in rocket launches.

Seems to me the only reason to use SIGABRT rather than exit()
with an error code is to provoke a core dump, and that's only
useful if we suspect a bug in Python itself. That's not the
case here -- the cause is clearly something external.

-- 
Greg

From stephen at xemacs.org  Tue Oct 23 02:24:11 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 23 Oct 2012 09:24:11 +0900
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <20121023002027.Horde.-FSSUlNNcXdQhcarnobBE-A@webmail.df.eu>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
	<20121023002027.Horde.-FSSUlNNcXdQhcarnobBE-A@webmail.df.eu>
Message-ID: <87pq4a3uj8.fsf@uwakimon.sk.tsukuba.ac.jp>

martin at v.loewis.de writes:

 > Instead, the primary reason for not choosing cryptography is ease-of-use.
 > 
 > For that reason, I still think that using an established algorithm would
 > be the better choice.

I don't understand this.  An established algorithm might be
cryptographically safer, but what could be easier to use than the
default algorithm that is built-in?


From dholth at gmail.com  Tue Oct 23 02:42:35 2012
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 22 Oct 2012 20:42:35 -0400
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <87pq4a3uj8.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
	<20121023002027.Horde.-FSSUlNNcXdQhcarnobBE-A@webmail.df.eu>
	<87pq4a3uj8.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAG8k2+7kFhg6rxgh=dPgepcGMrW+mM_jvo91vapJkBq-uhwyNg@mail.gmail.com>

You could just include a different algorithm. He meant that the speed of
ed25519 is not an advantage in this use case. Did I mention that its c
implementation has defense against timing attacks?
On Oct 22, 2012 8:24 PM, "Stephen J. Turnbull" <stephen at xemacs.org> wrote:

> martin at v.loewis.de writes:
>
>  > Instead, the primary reason for not choosing cryptography is
> ease-of-use.
>  >
>  > For that reason, I still think that using an established algorithm would
>  > be the better choice.
>
> I don't understand this.  An established algorithm might be
> cryptographically safer, but what could be easier to use than the
> default algorithm that is built-in?
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121022/ef6b54b3/attachment-0001.html>

From victor.stinner at gmail.com  Tue Oct 23 02:50:32 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 23 Oct 2012 02:50:32 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles
Message-ID: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>

Hi,

I forked CPython repository to work on my "split unicodeobject.c" project:
http://hg.python.org/sandbox/split-unicodeobject.c

The result is 10 files (included the existing unicodeobject.c):

  1176 Objects/unicodecharmap.c
  1678 Objects/unicodecodecs.c
  1362 Objects/unicodeformat.c
   253 Objects/unicodeimpl.h
   733 Objects/unicodelegacy.c
  1836 Objects/unicodenew.c
  2777 Objects/unicodeobject.c
  2421 Objects/unicodeoperators.c
  1235 Objects/unicodeoscodecs.c
  1288 Objects/unicodeutfcodecs.c
 14759 total

This is just a proposition (and work in progress). Everything can be changed :-)

"unicodenew.c" is not a good name. Content of this file may be moved
somewhere else.

Some files may be merged again if the separation is not justified.

I don't like the "unicode" prefix for filenames, I would prefer a new directory.

--

Shorter files are easier to review and maintain. The compilation is
faster if only one file is modified.

The MBCS codec requires windows.h. The whole unicodeobject.c includes
it just for this codec. With the split, only unicodeoscodecs.c
includes this file.

The MBCS codec needs also a "winver" variable. This variable is
defined between the BLOOM filter and the unicode_result_unchanged()
function. How can you explain how these things are sorted? Where
should I add a new function or variable? With the split, the variable
is now defined very close to where is it used. You don't have to
scroll 7000 lines to see where it is used.

If you would like to work on a specific function, you don't have to
use the search function of your editor to skip thousands to lines. For
example, the 18 functions and 2 types related to the charmap codec are
now grouped into one unique and short C file.

It was already possible to extend and maintain unicodeobject.c (some
people proved it!), but it should now be much simpler with shorter
files.

Note: unicodeobject.c is also composed by the huge stringlib library
(4000 lines), which is shared with the bytes type.

--

* Objects/unicodeimpl.h

Private macros and prototype of private functions.

Many unicode_xxx() functions has been renamed to _PyUnicode_xxx() to
be able to reuse them in different files.

* Objects/unicodenew.c

Functions to create a new Unicode string (PyUnicode_New), convert
from/to UCS4 and wchar_t*, resize a string. The ugly part of the PEP
393.

* Objects/unicodeoperators.c

find, replace, compare, split, fill, etc.

* Objects/unicodeobject.c

"str" type with all methods, _string module and unicodeiter type.

* Objects/unicodeformat.c

PyUnicode_FromFormat() and PyUnicode_Format()

* Objects/unicodecodecs.c

Text codecs for Python Unicode strings:
   - PyUnicode_Decode()
   - PyUnicode_AsEncodedObject()
   - PyUnicode_DecodeUnicodeEscape()
   - PyUnicode_DecodeRawUnicodeEscape(), PyUnicode_AsRawUnicodeEscapeString()
   - _PyUnicode_DecodeUnicodeInternal()
   - PyUnicode_DecodeLatin1(), PyUnicode_AsLatin1String()
   - PyUnicode_AsASCIIString()
   - PyUnicode_EncodeDecimal()
   - many helpers for other codecs
   - ...

* Objects/unicodecharmap.c

Character Mapping Codec:
   - PyUnicode_BuildEncodingMap()
   - PyUnicode_DecodeCharmap()
   - PyUnicode_AsCharmapString()
   - PyUnicode_Translate()

* Objects/unicodeoscodecs.c

Operating system codecs: MBCS codec, locale (FS) codec => FS encode/decode.

* Objects/unicodeutfcodecs.c

UTF-7/8/16/32 codecs and ASCII decoder.

* Objects/unicodelegacy.c

Legacy and deprecated Unicode API: Py_UNICODE type.

Victor

From dholth at gmail.com  Tue Oct 23 06:01:35 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 23 Oct 2012 00:01:35 -0400
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <CAG8k2+7kFhg6rxgh=dPgepcGMrW+mM_jvo91vapJkBq-uhwyNg@mail.gmail.com>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
	<20121023002027.Horde.-FSSUlNNcXdQhcarnobBE-A@webmail.df.eu>
	<87pq4a3uj8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+7kFhg6rxgh=dPgepcGMrW+mM_jvo91vapJkBq-uhwyNg@mail.gmail.com>
Message-ID: <CAG8k2+6k8UDcuY5GGZpwuzjBgJti2w+eY=HLPG69Rsu-dkF4Hg@mail.gmail.com>

Well, let's try again. I know Ed25519 is not on anyone's list, but I
would like to show at minimum that it is interesting.

%timeit wheel.tool.unpack('lxml-3.0.1-cp27-none-linux_x86_64.whl')
10 loops, best of 3: 123 ms per loop

.unpack() verifies the signature and extracts the file. Installation
is not much more costly than unpack. Removing the C extension to use
the Python Ed25519 instead,

%timeit wheel.tool.unpack('lxml-3.0.1-cp27-none-linux_x86_64.whl')
1 loops, best of 3: 153 ms per loop

With python-ecdsa, verifying a single signature,

import ecdsa
sk = ecdsa.SigningKey.generate(ecdsa.NIST256p)
sig = sk.sign("message")
vk = sk.get_verifying_key()
%timeit vk.verify(sig, "message")
1 loops, best of 3: 144 ms per loop

takes about as long as the whole unpack() including verification.

You will notice the difference when you are rebuilding a virtualenv
with 30+ dependencies every time you commit, keeping the old
virtualenv around in case something broke. It wouldn't matter if we
were building from source because we would avoid creating new
virtualenvs just to save time.

Also python-ecdsa does not work on Python 3.

ECDSA's verifying (public) keys at the same security level are 64
bytes instead of the 32-byte Ed25519 keys, which would make the
proposed "inline public keys" syntax half as convenient.

package[ed25519=ouBJlTJJ4SJXoy8Bi1KRlewWLU6JW7HUXTgvU1YRuiA]

I hope that the speed of wheel, and the properties of its signature
scheme, will inspire people to invent new applications that require
those features.

If necessary we could also just remove algorithms from the spec and
make the signing implementation pluggable. For key management reasons,
the signing interface used by the build tool is already "run the wheel
command line tool in a subprocess" which is also what you would do if
you needed to take advantage of the allowed S/MIME RECORD.p7s.

From martin at v.loewis.de  Tue Oct 23 07:42:28 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Tue, 23 Oct 2012 07:42:28 +0200
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <CAG8k2+6k8UDcuY5GGZpwuzjBgJti2w+eY=HLPG69Rsu-dkF4Hg@mail.gmail.com>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
	<20121023002027.Horde.-FSSUlNNcXdQhcarnobBE-A@webmail.df.eu>
	<87pq4a3uj8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+7kFhg6rxgh=dPgepcGMrW+mM_jvo91vapJkBq-uhwyNg@mail.gmail.com>
	<CAG8k2+6k8UDcuY5GGZpwuzjBgJti2w+eY=HLPG69Rsu-dkF4Hg@mail.gmail.com>
Message-ID: <20121023074228.Horde.YauzZqGZi1VQhi5Ev0N2_6A@webmail.df.eu>


Zitat von Daniel Holth <dholth at gmail.com>:

> Well, let's try again. I know Ed25519 is not on anyone's list, but I
> would like to show at minimum that it is interesting.

I have no doubt about that, and I believe that this ("it is interesting")
might be your primary motivation for using it, i.e. "coolness", which
is really a form of purity.

> %timeit wheel.tool.unpack('lxml-3.0.1-cp27-none-linux_x86_64.whl')
> 1 loops, best of 3: 153 ms per loop
>
> With python-ecdsa, verifying a single signature,
>
> import ecdsa
> sk = ecdsa.SigningKey.generate(ecdsa.NIST256p)
> sig = sk.sign("message")
> vk = sk.get_verifying_key()
> %timeit vk.verify(sig, "message")
> 1 loops, best of 3: 144 ms per loop
>
> takes about as long as the whole unpack() including verification.

This isn't really convincing, as you are comparing different operations.
It may well be that in pure-python ed25519, it's also the validation
that takes most of the time.

But it's really irrelevant. We are talking about a runtime of 0.1s.
If this really becomes an issue, an openssl version can be used instead.

> Also python-ecdsa does not work on Python 3.

Which certainly can be fixed.

> If necessary we could also just remove algorithms from the spec and
> make the signing implementation pluggable. For key management reasons,
> the signing interface used by the build tool is already "run the wheel
> command line tool in a subprocess" which is also what you would do if
> you needed to take advantage of the allowed S/MIME RECORD.p7s.

I'm also -1 on the notion that the entire key distribution matter is out
of scope. With that approach, I feel that the package signing is essentially
pointless.

As a general note on this, this entire issue lacks a threat model:
what kind of attack do you want to protect against? I can't think of
any realistic threat that is effectively protected against with your
signature scheme.

Regards,
Martin



From martin at v.loewis.de  Tue Oct 23 07:46:18 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Tue, 23 Oct 2012 07:46:18 +0200
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <87pq4a3uj8.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
	<20121023002027.Horde.-FSSUlNNcXdQhcarnobBE-A@webmail.df.eu>
	<87pq4a3uj8.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20121023074618.Horde.wMgIAKGZi1VQhi8q7-aHAPA@webmail.df.eu>


Zitat von "Stephen J. Turnbull" <stephen at xemacs.org>:

> martin at v.loewis.de writes:
>
>  > Instead, the primary reason for not choosing cryptography is ease-of-use.
>  >
>  > For that reason, I still think that using an established algorithm would
>  > be the better choice.
>
> I don't understand this.  An established algorithm might be
> cryptographically safer, but what could be easier to use than the
> default algorithm that is built-in?

That's exactly what I want: it (PEP 427) should use one of the algorithms
that is built-in (into web signatures). Web signatures give a choice of
three algorithms; yet Daniel proposes to deviate and use a non-builtin
algorithm.

None of the algorithms in question are built in in Python; the two
standard algorithms with public keys (i.e. RSA and ECDSA) are both
built into OpenSSL.

Regards,
Martin



From p.f.moore at gmail.com  Tue Oct 23 09:04:38 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 23 Oct 2012 08:04:38 +0100
Subject: [Python-Dev] PEP 427: data directory
In-Reply-To: <CAG8k2+7194=dxa=A1m6c+EiaFs93HpKq_+rEuUBK-xEX=aO2Bg@mail.gmail.com>
References: <20121022222022.6bd940c1@pitrou.net>
	<CAG8k2+7m6yhL5SO1WU12LyppfBMYYmivetVW7m-DuTz0Fc9SEA@mail.gmail.com>
	<CAG8k2+7194=dxa=A1m6c+EiaFs93HpKq_+rEuUBK-xEX=aO2Bg@mail.gmail.com>
Message-ID: <CACac1F-GomgYYTe11ESbDsHq9UN_+P0r62y8Oy8iBvdCdpWSNA@mail.gmail.com>

On 22 October 2012 21:35, Daniel Holth <dholth at gmail.com> wrote:
> On Mon, Oct 22, 2012 at 4:26 PM, Daniel Holth <dholth at gmail.com> wrote:
>> On Mon, Oct 22, 2012 at 4:20 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>>
>>> Hello,
>>>
>>> The FAQ has this weird statement:
>>>
>>> ?This specification does not have an opinion on how you should organize
>>> your code. The .data directory is just a place for any files that are
>>> not normally installed inside site-packages or on the PYTHONPATH.?
>
> I mean to say that just because there is a .data/ directory it doesn't
> mean you have to stop using pkgutil.get_data(package, resource). Some
> people will take those 4 letters "data" to mean that any file that
> doesn't start with .py has to go there instead of in the root of the
> archive.

It might be worth being very explicit in the PEP that the wheel format
is based specifically on the distutils/sysconfig categories for file
locations (purelib, platlib, data, scripts, ...). If a setup.py is
written with custom code to install scripts in locations not
controlled by the various --install-XXX flags to distutils, it is
undefined how to package the non-standard files in the wheel format.
This is the same behaviour as for the wininst format (certainly, I've
checked the code :-)) and the egg format (I think, but I didn't check
the code for this).

Antoine's example of /etc/init.d is *not* custom code, it is standard
distutils (although not very well documented :-() What setup.py
install does is to make the given path relative to the data directory
which is sys.prefix on Windows, and appears to be similar on Posix
(the value of sysconfig.get_path('data') should give it, but I don't
have a Linux box handy to try it on). I don't see any obvious
documentation confirming that it does what I assume Antoine actually
wants, which is to put the file in /etc/init.d (which would of course
make the package non-relocatable, and hence mean that it would break
isolation in a virtualenv).

Apologies if this is obvious to everyone, but I suspect it should be
stated explicitly in the PEP, if only to avoid confusion. Something
like the following:

"""
The contents of a wheel file consist of:

1. The root of the wheel, which is all files to be installed in
"purelib" (or "platlib" if there is no purelib)
2. The dist-info directory, containing metadata
3. The data directory, which contains one subdirectory for each
sysconfig scheme not already covered (i.e., data, scripts, include,
and possibly platlib)

How files which would be installed to locations not defined by a
sysconfig path are stored in a wheel is not defined by this version of
the spec.

This layout offers all of the functionality of the existing wininst
and egg binary formats.
"""

Paul

From dirkjan at ochtman.nl  Tue Oct 23 09:14:19 2012
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Tue, 23 Oct 2012 09:14:19 +0200
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <20121023074618.Horde.wMgIAKGZi1VQhi8q7-aHAPA@webmail.df.eu>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
	<20121023002027.Horde.-FSSUlNNcXdQhcarnobBE-A@webmail.df.eu>
	<87pq4a3uj8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<20121023074618.Horde.wMgIAKGZi1VQhi8q7-aHAPA@webmail.df.eu>
Message-ID: <CAKmKYaCd=XWF-7fj4_v9YejJ81aSuPm7Lb3M+DsRwr7S3d4=Lw@mail.gmail.com>

On Tue, Oct 23, 2012 at 7:46 AM,  <martin at v.loewis.de> wrote:
> That's exactly what I want: it (PEP 427) should use one of the algorithms
> that is built-in (into web signatures). Web signatures give a choice of
> three algorithms; yet Daniel proposes to deviate and use a non-builtin
> algorithm.
>
> None of the algorithms in question are built in in Python; the two
> standard algorithms with public keys (i.e. RSA and ECDSA) are both
> built into OpenSSL.

What leads you to say that? ISTM Python has perfectly good support for
JWS/JWA's HS256 algorithm. In fact, here's an implementation that I
think would conform to the current JWS draft:

def sign(payload, key):
        h = json.dumps({'alg': 'HS256'})
        input = b64uencode(h) + '.' + b64uencode(json.dumps(payload))
        sig = hmac.new(key, input, hashlib.sha256).digest()
        return input + '.' + b64uencode(sig)

(b64u implementations elided; see
https://bitbucket.org/djc/persona-totp for the rest of the code.)

Cheers,

Dirkjan

From benjamin at python.org  Tue Oct 23 10:22:10 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 23 Oct 2012 04:22:10 -0400
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
Message-ID: <CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>

2012/10/22 Victor Stinner <victor.stinner at gmail.com>:
> Hi,
>
> I forked CPython repository to work on my "split unicodeobject.c" project:
> http://hg.python.org/sandbox/split-unicodeobject.c
>
> The result is 10 files (included the existing unicodeobject.c):
>
>   1176 Objects/unicodecharmap.c
>   1678 Objects/unicodecodecs.c
>   1362 Objects/unicodeformat.c
>    253 Objects/unicodeimpl.h
>    733 Objects/unicodelegacy.c
>   1836 Objects/unicodenew.c
>   2777 Objects/unicodeobject.c
>   2421 Objects/unicodeoperators.c
>   1235 Objects/unicodeoscodecs.c
>   1288 Objects/unicodeutfcodecs.c
>  14759 total
>
> This is just a proposition (and work in progress). Everything can be changed :-)
>
> "unicodenew.c" is not a good name. Content of this file may be moved
> somewhere else.
>
> Some files may be merged again if the separation is not justified.
>
> I don't like the "unicode" prefix for filenames, I would prefer a new directory.
>
> --
>
> Shorter files are easier to review and maintain. The compilation is
> faster if only one file is modified.
>
> The MBCS codec requires windows.h. The whole unicodeobject.c includes
> it just for this codec. With the split, only unicodeoscodecs.c
> includes this file.
>
> The MBCS codec needs also a "winver" variable. This variable is
> defined between the BLOOM filter and the unicode_result_unchanged()
> function. How can you explain how these things are sorted? Where
> should I add a new function or variable? With the split, the variable
> is now defined very close to where is it used. You don't have to
> scroll 7000 lines to see where it is used.
>
> If you would like to work on a specific function, you don't have to
> use the search function of your editor to skip thousands to lines. For
> example, the 18 functions and 2 types related to the charmap codec are
> now grouped into one unique and short C file.
>
> It was already possible to extend and maintain unicodeobject.c (some
> people proved it!), but it should now be much simpler with shorter
> files.

I would like to repeat my opposition to splitting unicodeobject.c. I
don't think the benefits of such a split have been well justified,
certainly not to the point that the claim about "much simpler"
maintenance is true.


-- 
Regards,
Benjamin

From mal at egenix.com  Tue Oct 23 11:28:39 2012
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 23 Oct 2012 11:28:39 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
Message-ID: <50866347.60507@egenix.com>

On 23.10.2012 10:22, Benjamin Peterson wrote:
> 2012/10/22 Victor Stinner <victor.stinner at gmail.com>:
>> Hi,
>>
>> I forked CPython repository to work on my "split unicodeobject.c" project:
>> http://hg.python.org/sandbox/split-unicodeobject.c
>>
>> The result is 10 files (included the existing unicodeobject.c):
>>
>>   1176 Objects/unicodecharmap.c
>>   1678 Objects/unicodecodecs.c
>>   1362 Objects/unicodeformat.c
>>    253 Objects/unicodeimpl.h
>>    733 Objects/unicodelegacy.c
>>   1836 Objects/unicodenew.c
>>   2777 Objects/unicodeobject.c
>>   2421 Objects/unicodeoperators.c
>>   1235 Objects/unicodeoscodecs.c
>>   1288 Objects/unicodeutfcodecs.c
>>  14759 total
>>
>> This is just a proposition (and work in progress). Everything can be changed :-)
>>
>> "unicodenew.c" is not a good name. Content of this file may be moved
>> somewhere else.
>>
>> Some files may be merged again if the separation is not justified.
>>
>> I don't like the "unicode" prefix for filenames, I would prefer a new directory.
>>
>> --
>>
>> Shorter files are easier to review and maintain. The compilation is
>> faster if only one file is modified.
>>
>> The MBCS codec requires windows.h. The whole unicodeobject.c includes
>> it just for this codec. With the split, only unicodeoscodecs.c
>> includes this file.
>>
>> The MBCS codec needs also a "winver" variable. This variable is
>> defined between the BLOOM filter and the unicode_result_unchanged()
>> function. How can you explain how these things are sorted? Where
>> should I add a new function or variable? With the split, the variable
>> is now defined very close to where is it used. You don't have to
>> scroll 7000 lines to see where it is used.
>>
>> If you would like to work on a specific function, you don't have to
>> use the search function of your editor to skip thousands to lines. For
>> example, the 18 functions and 2 types related to the charmap codec are
>> now grouped into one unique and short C file.
>>
>> It was already possible to extend and maintain unicodeobject.c (some
>> people proved it!), but it should now be much simpler with shorter
>> files.
> 
> I would like to repeat my opposition to splitting unicodeobject.c. I
> don't think the benefits of such a split have been well justified,
> certainly not to the point that the claim about "much simpler"
> maintenance is true.

Same feelings here.

If you do go ahead with such a split, please only split the source
files and keep the unicodeobject.c file which then includes all
the other files. Such a restructuring should not result in compilers
no longer being able to optimize code by inlining functions
in one of the most important basic types we have in Python 3.

Also note that splitting the file in multiple smaller ones will
actually create more maintenance overhead, since patches will
likely no longer be easy to merge from 3.3 to 3.4.

BTW: The positive effect of having everything in one file is
that you no longer have to figure which files to look when
trying to find a piece of logic... it's just a ctrl-f or
ctrl-s away :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 23 2012)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2012-09-27: Released eGenix PyRun 1.1.0 ...       http://egenix.com/go35
2012-09-26: Released mxODBC.Connect 2.0.1 ...     http://egenix.com/go34
2012-09-25: Released mxODBC 3.2.1 ...             http://egenix.com/go33
2012-10-23: Python Meeting Duesseldorf ...                         today

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From victor.stinner at gmail.com  Tue Oct 23 12:05:21 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 23 Oct 2012 12:05:21 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <50866347.60507@egenix.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<50866347.60507@egenix.com>
Message-ID: <CAMpsgwbBe2yD2Dy+tYCvBracUhaRonx7-M7da3oZ6FGUqbWsjQ@mail.gmail.com>

> Such a restructuring should not result in compilers
> no longer being able to optimize code by inlining functions
> in one of the most important basic types we have in Python 3.

I agree that performances are important. But I'm not convinced than
moving functions has a real impact on performances, not that such
issues cannot be fixed.

I tried to limit changes impacting performances. Inlining is (only?)
interesting for short functions. PEP 393 introduces many macros for
this. I also added some "Fast" functiions
(_PyUnicode_FastCopyCharacters() and _PyUnicode_FastFill()) which
don't check parameters and do the real work. I don't think that it's
really useful to inline _PyUnicode_FastFill() in the caller for
example.

I will check performances of all str methods. For example, str.count()
is now calling PyUnicode_Count() instead of the static count().
PyUnicode_Count() adds some extra checks, some of them are not
necessary, and it's not a static function, so it cannot(?) be inlined.
But I bet that the overhead is really low.

Note: Since GCC 4.5, Link Time Optimization are possible. I don't know
if GCC is able to inline functions defined in different files, but C
compilers are better at each release.

--

I will check the impact of performances on _PyUnicode_Widen() and
_PyUnicode_Putchar(), which are no more static. _PyUnicode_Widen() and
_PyUnicode_Putchar() are used in Unicode codecs when it's more
expensive to compute the exact length and maximum character of the
output string. These functions are optimistic (hope that the output
will not grow too much and the string is not "widen" too much times,
so it should be faster for ASCII).

I implemented a similar approach in my PyUnicodeWriter API, and I plan
to reuse this API to simplify the API. PyUnicodeWriter uses some macro
to limit the overhead of having to check before each write if we need
to enlarge or widen the internal buffer, and allow to write directly
into the buffer using low level functions like PyUnicode_WRITE.

I also hope a performance improvement because the PyUnicodeWriter API
can also overallocate the internal buffer to reduce the number of
calls to realloc() (which is usually slow).

> Also note that splitting the file in multiple smaller ones will
> actually create more maintenance overhead, since patches will
> likely no longer be easy to merge from 3.3 to 3.4.

I'm a candidate to maintain unicodeobject.c. In your check
unicodeobject.c (recent) history, I'm one of the most active developer
on this file since two years (especially in 2012). I'm not sure that
merges on this file are so hard.

Victor

From solipsis at pitrou.net  Tue Oct 23 12:11:11 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 23 Oct 2012 12:11:11 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <CAMpsgwbBe2yD2Dy+tYCvBracUhaRonx7-M7da3oZ6FGUqbWsjQ@mail.gmail.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<50866347.60507@egenix.com>
	<CAMpsgwbBe2yD2Dy+tYCvBracUhaRonx7-M7da3oZ6FGUqbWsjQ@mail.gmail.com>
Message-ID: <k65qfu$8pg$1@ger.gmane.org>

Le 23/10/2012 12:05, Victor Stinner a ?crit :
>> Such a restructuring should not result in compilers
>> no longer being able to optimize code by inlining functions
>> in one of the most important basic types we have in Python 3.
>
> I agree that performances are important. But I'm not convinced than
> moving functions has a real impact on performances, not that such
> issues cannot be fixed.

I agree with Marc-Andr?, there's no point in compiling those files 
separately. #include'ing them in the master unicodeobject.c file is fine.

Regards

Antoine.



From dholth at gmail.com  Tue Oct 23 13:16:57 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 23 Oct 2012 07:16:57 -0400
Subject: [Python-Dev] PEP 427 comment: code signing
In-Reply-To: <20121023074228.Horde.YauzZqGZi1VQhi5Ev0N2_6A@webmail.df.eu>
References: <20121018202114.Horde.QpY1DElCcOxQgEiaNVC2wjA@webmail.df.eu>
	<CAG8k2+54PPLJWdhPMayGmQn+X4sfdfzr8n_BqdSH7JVfbcyhCw@mail.gmail.com>
	<CAG8k2+7TSsy=TF4wqg8ADJ2hBZq-h6gFh_no2JeTaO=cQRAD9A@mail.gmail.com>
	<20121023002027.Horde.-FSSUlNNcXdQhcarnobBE-A@webmail.df.eu>
	<87pq4a3uj8.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+7kFhg6rxgh=dPgepcGMrW+mM_jvo91vapJkBq-uhwyNg@mail.gmail.com>
	<CAG8k2+6k8UDcuY5GGZpwuzjBgJti2w+eY=HLPG69Rsu-dkF4Hg@mail.gmail.com>
	<20121023074228.Horde.YauzZqGZi1VQhi5Ev0N2_6A@webmail.df.eu>
Message-ID: <CAG8k2+7mpf9b0TC2CBn0Rfcuv7mgbSgHBNTb4qbOQcPjbdQ9ZA@mail.gmail.com>

On Tue, Oct 23, 2012 at 1:42 AM,  <martin at v.loewis.de> wrote:

> I'm also -1 on the notion that the entire key distribution matter is out
> of scope. With that approach, I feel that the package signing is essentially
> pointless.
>
> As a general note on this, this entire issue lacks a threat model:
> what kind of attack do you want to protect against? I can't think of
> any realistic threat that is effectively protected against with your
> signature scheme.

It is designed to protect against a man-in-the-middle attack. What if
I'm at pycon using an open access point? An attacker has proxied the
connection to provide malware instead of the correct packages.
Thankfully, the tahoe-lafs developers have sent me a PGP-signed
requirements file with the keys of all the dependencies they trust:

allmydata-tahoe[algorithmkey=YDWz8J6HAQc1V4_EoO-1cEGHSHjRd-5HYjj4hPCmSVZUZDm67-NngM2_XcMJOddXBv6xNLjK91DEn20KZCSFeBdMzeohE1YNq__4CT91StT0cQ_zhrQ1vwwwILZuOfgPmVep3lw2Jn3KVnl1PBw7P1WjuCctENxwuFz3NuWhER_uldA-0ted0SYKvvD5zI85epp8mRucxw0d7NUTdtTci7Hyx-ujTBDlTIB-tEIQ_9eJf9BznFuqvvfYf4qlfgjF4nvDgU1pQfbu6RSBOVdZEFgNqoPWV-Qo_4HjyKA7WG0Xk9OI92Jl3JkZRV2bP-KdRwbnUj7dyVSvhr2ilWx0s380epSSPLBByrmW8dkj_b8NJwSVk5J6rWMqKplINZlpWW5j3O1pn2U5e-XG6AWBNZd5r89MhXHhz2PA9CyGb7sINRljC716tdz-RYaFD2lScdszygNsMBvLHcyz9GQmjPtK4GhQQGrRyu40Q3BWJWq4l70pBBmG]

Twisted[algorithmkey=YDWz8J6HAQc1V4_EoO-1cEGHSHjRd-5HYjj4hPCmSVZUZDm67-NngM2_XcMJOddXBv6xNLjK91DEn20KZCSFeBdMzeohE1YNq__4CT91StT0cQ_zhrQ1vwwwILZuOfgPmVep3lw2Jn3KVnl1PBw7P1WjuCctENxwuFz3NuWhER_uldA-0ted0SYKvvD5zI85epp8mRucxw0d7NUTdtTci7Hyx-ujTBDlTIB-tEIQ_9eJf9BznFuqvvfYf4qlfgjF4nvDgU1pQfbu6RSBOVdZEFgNqoPWV-Qo_4HjyKA7WG0Xk9OI92Jl3JkZRV2bP-KdRwbnUj7dyVSvhr2ilWx0s380epSSPLBByrmW8dkj_b8NJwSVk5J6rWMqKplINZlpWW5j3O1pn2U5e-XG6AWBNZd5r89MhXHhz2PA9CyGb7sINRljC716tdz-RYaFD2lScdszygNsMBvLHcyz9GQmjPtK4GhQQGrRyu40Q3BWJWq4l70pBBmG]

...

and so on. I pip install --signed-only -r tahoe-requirements.txt (not
implemented yet) to install the application, knowing the packages come
from the publishers the app developer expected.

From amauryfa at gmail.com  Tue Oct 23 14:03:44 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 23 Oct 2012 14:03:44 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <k65qfu$8pg$1@ger.gmane.org>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<50866347.60507@egenix.com>
	<CAMpsgwbBe2yD2Dy+tYCvBracUhaRonx7-M7da3oZ6FGUqbWsjQ@mail.gmail.com>
	<k65qfu$8pg$1@ger.gmane.org>
Message-ID: <CAGmFidYSJK9mmWxpHSWC9=6iO31=b6YOU_nzBOEAS+TTWF3ZfA@mail.gmail.com>

2012/10/23 Antoine Pitrou <solipsis at pitrou.net>:
> I agree with Marc-Andr?, there's no point in compiling those files
> separately. #include'ing them in the master unicodeobject.c file is fine.

I also find the unicodeobject.c difficult to navigate.
Even if we don't split the file, I'd advocate a better presentation of
its content.

Could we have at least clear sections, with titles and descriptions?
And use the ^L page separator for Emacs users?

Code in posixmodule.c could also benefit of a better layout.

-- 
Amaury Forgeot d'Arc

From dholth at gmail.com  Tue Oct 23 15:09:41 2012
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 23 Oct 2012 09:09:41 -0400
Subject: [Python-Dev] PEP 427: data directory
In-Reply-To: <CACac1F-GomgYYTe11ESbDsHq9UN_+P0r62y8Oy8iBvdCdpWSNA@mail.gmail.com>
References: <20121022222022.6bd940c1@pitrou.net>
	<CAG8k2+7m6yhL5SO1WU12LyppfBMYYmivetVW7m-DuTz0Fc9SEA@mail.gmail.com>
	<CAG8k2+7194=dxa=A1m6c+EiaFs93HpKq_+rEuUBK-xEX=aO2Bg@mail.gmail.com>
	<CACac1F-GomgYYTe11ESbDsHq9UN_+P0r62y8Oy8iBvdCdpWSNA@mail.gmail.com>
Message-ID: <CAG8k2+4W36-kJBRHd2mXeujpT_Lb+W9naq=udb9YMfXB3uzuYg@mail.gmail.com>

On Tue, Oct 23, 2012 at 3:04 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 22 October 2012 21:35, Daniel Holth <dholth at gmail.com> wrote:
>> On Mon, Oct 22, 2012 at 4:26 PM, Daniel Holth <dholth at gmail.com> wrote:
>>> On Mon, Oct 22, 2012 at 4:20 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>>>
>>>> Hello,
>>>>
>>>> The FAQ has this weird statement:
>>>>
>>>> ?This specification does not have an opinion on how you should organize
>>>> your code. The .data directory is just a place for any files that are
>>>> not normally installed inside site-packages or on the PYTHONPATH.?
>>
>> I mean to say that just because there is a .data/ directory it doesn't
>> mean you have to stop using pkgutil.get_data(package, resource). Some
>> people will take those 4 letters "data" to mean that any file that
>> doesn't start with .py has to go there instead of in the root of the
>> archive.
>
> It might be worth being very explicit in the PEP that the wheel format
> is based specifically on the distutils/sysconfig categories for file
> locations (purelib, platlib, data, scripts, ...). If a setup.py is
> written with custom code to install scripts in locations not
> controlled by the various --install-XXX flags to distutils, it is
> undefined how to package the non-standard files in the wheel format.
> This is the same behaviour as for the wininst format (certainly, I've
> checked the code :-)) and the egg format (I think, but I didn't check
> the code for this).
>
> Antoine's example of /etc/init.d is *not* custom code, it is standard
> distutils (although not very well documented :-() What setup.py
> install does is to make the given path relative to the data directory
> which is sys.prefix on Windows, and appears to be similar on Posix
> (the value of sysconfig.get_path('data') should give it, but I don't
> have a Linux box handy to try it on). I don't see any obvious
> documentation confirming that it does what I assume Antoine actually
> wants, which is to put the file in /etc/init.d (which would of course
> make the package non-relocatable, and hence mean that it would break
> isolation in a virtualenv).
>
> Apologies if this is obvious to everyone, but I suspect it should be
> stated explicitly in the PEP, if only to avoid confusion. Something
> like the following:
>
> """
> The contents of a wheel file consist of:
>
> 1. The root of the wheel, which is all files to be installed in
> "purelib" (or "platlib" if there is no purelib)
> 2. The dist-info directory, containing metadata
> 3. The data directory, which contains one subdirectory for each
> sysconfig scheme not already covered (i.e., data, scripts, include,
> and possibly platlib)
>
> How files which would be installed to locations not defined by a
> sysconfig path are stored in a wheel is not defined by this version of
> the spec.
>
> This layout offers all of the functionality of the existing wininst
> and egg binary formats.
> """

Thanks, this part of the PEP needs clarification. Note the PEP is
currently using the distutils.command.install schemes instead of
sysconfig in order to get the 'headers' key.

Is anyone reading this actually using the purelib != platlib feature?

From g.brandl at gmx.net  Tue Oct 23 18:29:53 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 23 Oct 2012 18:29:53 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
Message-ID: <k66glu$r11$1@ger.gmane.org>

On 10/23/2012 10:22 AM, Benjamin Peterson wrote:
> 2012/10/22 Victor Stinner <victor.stinner at gmail.com>:
>> Hi,
>>
>> I forked CPython repository to work on my "split unicodeobject.c" project:
>> http://hg.python.org/sandbox/split-unicodeobject.c
>>
>> The result is 10 files (included the existing unicodeobject.c):
>>
>>   1176 Objects/unicodecharmap.c
>>   1678 Objects/unicodecodecs.c
>>   1362 Objects/unicodeformat.c
>>    253 Objects/unicodeimpl.h
>>    733 Objects/unicodelegacy.c
>>   1836 Objects/unicodenew.c
>>   2777 Objects/unicodeobject.c
>>   2421 Objects/unicodeoperators.c
>>   1235 Objects/unicodeoscodecs.c
>>   1288 Objects/unicodeutfcodecs.c
>>  14759 total
>>
>> This is just a proposition (and work in progress). Everything can be changed :-)
>>
>> "unicodenew.c" is not a good name. Content of this file may be moved
>> somewhere else.
>>
>> Some files may be merged again if the separation is not justified.
>>
>> I don't like the "unicode" prefix for filenames, I would prefer a new directory.
>>
>> --
>>
>> Shorter files are easier to review and maintain. The compilation is
>> faster if only one file is modified.
>>
>> The MBCS codec requires windows.h. The whole unicodeobject.c includes
>> it just for this codec. With the split, only unicodeoscodecs.c
>> includes this file.
>>
>> The MBCS codec needs also a "winver" variable. This variable is
>> defined between the BLOOM filter and the unicode_result_unchanged()
>> function. How can you explain how these things are sorted? Where
>> should I add a new function or variable? With the split, the variable
>> is now defined very close to where is it used. You don't have to
>> scroll 7000 lines to see where it is used.
>>
>> If you would like to work on a specific function, you don't have to
>> use the search function of your editor to skip thousands to lines. For
>> example, the 18 functions and 2 types related to the charmap codec are
>> now grouped into one unique and short C file.
>>
>> It was already possible to extend and maintain unicodeobject.c (some
>> people proved it!), but it should now be much simpler with shorter
>> files.
> 
> I would like to repeat my opposition to splitting unicodeobject.c. I
> don't think the benefits of such a split have been well justified,
> certainly not to the point that the claim about "much simpler"
> maintenance is true.

I agree.  I haven't edited much in unicodeobject.c lately, so this is
just an expression of my preference in general to keep things together.

We tell new Python programmers to stop worrying about using indentation
for grouping because editors are meant to make this easy.  A similar
argument applies to navigating large files: with a decent editor there is
no real problem with large files.

I agree completely with suggestions to improve sectioning and/or comments
within the file.

But once you make any split, people will look for things in the wrong file.
It happens for me every time I look for something in either object.c or
abstract.c -- that's an instance where the function name prefix doesn't imply
the implementation file name, which is otherwise very clear and easy in the
Python sources.

Especially since you're suggesting a huge number of new files, I question the
argument of better navigability.

Georg

BTW:

> If you would like to work on a specific function, you don't have to
> use the search function of your editor to skip thousands to lines. For
> example, the 18 functions and 2 types related to the charmap codec are
> now grouped into one unique and short C file.

After opening the right file, I *still* use the search function to get to
the function I want to edit.  Don't tell me using a scroll bar to scan
for the right place is faster...




From merwok at netwok.org  Wed Oct 24 03:19:29 2012
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 23 Oct 2012 21:19:29 -0400
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <20121012175039.GA1284@chang>
References: <5064D399.5050208@netwok.org> <506B36DB.1090001@netwok.org>
	<CAPTjJmqGWfoCKKBHuJwJGpg-roNoQQ0NMt_HVDCAaau4U+4MQA@mail.gmail.com>
	<506B685C.3050608@netwok.org> <20121012175039.GA1284@chang>
Message-ID: <50874221.2090905@netwok.org>

Hello,

Le 12/10/2012 13:50, Petri Lehtinen a ?crit :
> It's two and a half weeks left, but I've not seen any announcements
> yet!
Indeed, work and other commitments took over, so we (Montr?al-Python)
decided to move the bug day instead of announcing it late.  The date
that would work for us is November 3rd.

Brian, is it okay for Boston?
Maciej, what about your group?
Comitters, who could join on IRC?

Sorry for the false start.

Best regards

From rdmurray at bitdance.com  Wed Oct 24 03:52:25 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 23 Oct 2012 21:52:25 -0400
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <50874221.2090905@netwok.org>
References: <5064D399.5050208@netwok.org> <506B36DB.1090001@netwok.org>
	<CAPTjJmqGWfoCKKBHuJwJGpg-roNoQQ0NMt_HVDCAaau4U+4MQA@mail.gmail.com>
	<506B685C.3050608@netwok.org> <20121012175039.GA1284@chang>
	<50874221.2090905@netwok.org>
Message-ID: <20121024015226.4B695250054@webabinitio.net>

On Tue, 23 Oct 2012 21:19:29 -0400, <merwok at netwok.org> wrote:
> Le 12/10/2012 13:50, Petri Lehtinen a ??crit :
> > It's two and a half weeks left, but I've not seen any announcements
> > yet!
> Indeed, work and other commitments took over, so we (Montr??al-Python)
> decided to move the bug day instead of announcing it late.  The date
> that would work for us is November 3rd.
> 
> Brian, is it okay for Boston?
> Maciej, what about your group?
> Comitters, who could join on IRC?
> 
> Sorry for the false start.

This is very disappointing.  You had previously said that it was a go.
People (who may or may not have spoken up here) may have already
arranged space and be planning on it, despite the lack of announcement.
I certainly was planning on it.

I'm not available on the 3rd.

I'll continue with my plans to be active in the IRC channel and focused
on Python bug fixing this Saturday from some time in the AM until about
18:00, GMT -4.  If no one else shows up I'll just have a personal bug
day, but I'm guessing at least a few people might show up despite the
lack of a wider formal announcement.

--David

From merwok at netwok.org  Wed Oct 24 04:10:54 2012
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Tue, 23 Oct 2012 22:10:54 -0400
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <20121024015226.4B695250054@webabinitio.net>
References: <5064D399.5050208@netwok.org> <506B36DB.1090001@netwok.org>
	<CAPTjJmqGWfoCKKBHuJwJGpg-roNoQQ0NMt_HVDCAaau4U+4MQA@mail.gmail.com>
	<506B685C.3050608@netwok.org> <20121012175039.GA1284@chang>
	<50874221.2090905@netwok.org>
	<20121024015226.4B695250054@webabinitio.net>
Message-ID: <50874E2E.2040401@netwok.org>

Le 23/10/2012 21:52, R. David Murray a ?crit :
> This is very disappointing.  You had previously said that it was a go.
> People (who may or may not have spoken up here) may have already
> arranged space and be planning on it, despite the lack of announcement.
> I certainly was planning on it.

Okay, I was assuming silence meant the user groups had made no definite
arrangements.  The decision of not having a physical sprint in Montreal
doesn't have to affect the whole day, so I will be connected from home,
and the other people who wanted to participate can join in.

To make it clear: Bug day on IRC this Saturday is still on.

I'm updating the old wiki page to point to the new devguide and Petri's
bug list.
- http://docs.python.org/devguide/
- http://piratepad.net/pyconfi-sprint-issues

Sorry for the confusion.

From brian at python.org  Wed Oct 24 04:12:19 2012
From: brian at python.org (Brian Curtin)
Date: Tue, 23 Oct 2012 21:12:19 -0500
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <50874221.2090905@netwok.org>
References: <5064D399.5050208@netwok.org> <506B36DB.1090001@netwok.org>
	<CAPTjJmqGWfoCKKBHuJwJGpg-roNoQQ0NMt_HVDCAaau4U+4MQA@mail.gmail.com>
	<506B685C.3050608@netwok.org> <20121012175039.GA1284@chang>
	<50874221.2090905@netwok.org>
Message-ID: <CAD+XWwpoK1ys8B+_qKfgi1+C5E_6ov3p5iuBF0CCnLHS0vWuPQ@mail.gmail.com>

On Tue, Oct 23, 2012 at 8:19 PM, ?ric Araujo <merwok at netwok.org> wrote:
> Hello,
>
> Le 12/10/2012 13:50, Petri Lehtinen a ?crit :
>> It's two and a half weeks left, but I've not seen any announcements
>> yet!
> Indeed, work and other commitments took over, so we (Montr?al-Python)
> decided to move the bug day instead of announcing it late.  The date
> that would work for us is November 3rd.
>
> Brian, is it okay for Boston?
> Maciej, what about your group?
> Comitters, who could join on IRC?
>
> Sorry for the false start.

Nothing was formally planned to happen in Chicago due to the usual
lack of interest, and I doubt that'll change for November 3rd. I would
suggest just going on with whatever sprint date(s) your group is able
to make it for. This is already very loosely organized as it is, so
trying to coordinate multiple groups in multiple cities is not worth
it.

Anyway, I'll probably be around this Saturday and November 3rd to put
in some work.

From ronaldoussoren at mac.com  Wed Oct 24 13:04:36 2012
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 24 Oct 2012 13:04:36 +0200
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
Message-ID: <8DC03FC0-5182-4979-B219-F1FA5E9EB56E@mac.com>


On 18 Oct, 2012, at 19:29, Daniel Holth <dholth at gmail.com> wrote:

> I'd like to submit the Wheel PEPs 425 (filename metadata), 426
> (Metadata 1.3), and 427 (wheel itself) for acceptance. The format has
> been stable since May and we are preparing a patch to support it in
> pip, but we need to earn consensus before including it in the most
> widely used installer.

PEP 425: 

* "The version is py_version_nodot. CPython gets away with no dot, but if one is needed the underscore _ is used instead"

   I don't particularly like replacing dots by underscores. That needed because you use the dot character in compressed tag sets, but why not use a comma to separate items in the compressed tag set?

* "The platform tag is simply distutils.util.get_platform() with all hyphens - and periods . replaced with underscore _."

   Why the replacement?  The need for replacement could be avoided by using a different separator between elements of a tag (for example "~" or "+"), and furthermore the platform tag is at a know
   location, and hence the use of hyphens in the platform tag is harmless (use "python_tag, abi_tag, platform_tag = tag.split('-', 2)" to split the tag into its elements.

* "compressed tag sets"

   Using '"," instead of "." to separate elements of the tag set takes away the need to replace dots in tag elements, and seems more natural to me (you'd also use comma to separate the elements
   when you write them down in prose or python code.

Ronald

From ncoghlan at gmail.com  Wed Oct 24 14:19:40 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 24 Oct 2012 22:19:40 +1000
Subject: [Python-Dev] Fwd: Python 3.3 can't sort memoryviews as they're
	unorderable
In-Reply-To: <k64mpo$uao$1@ger.gmane.org>
References: <k60lsp$jok$1@ger.gmane.org>
	<k64mpo$uao$1@ger.gmane.org>
Message-ID: <CADiSq7cSWaceY72CqjXAsUq_YQ54VRi8ACaqT092C+UsE4WVww@mail.gmail.com>

(Oops, originally replied only to Mark)

Is a 3x3 array greater or less than a 2x4 array or another 3x3 array?

The contents of a 1D memory view may be sortable, but the "logical
structure" part isn't, and neither is any multi-dimensional view.

I'm surprised by the lack of inheritance support though - is that a
regression from 3.2? If yes, that's definitely a bug to be fixed in a 3.3
maintenance release, otherwise it's probably a feature request for 3.4.

Cheers,
Nick.

--
Sent from my phone, thus the relative brevity :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121024/24e7db95/attachment.html>

From ronaldoussoren at mac.com  Wed Oct 24 13:28:48 2012
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 24 Oct 2012 13:28:48 +0200
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
Message-ID: <B182D06E-27C5-46AB-9A3B-0AC182724FA9@mac.com>


On 18 Oct, 2012, at 19:29, Daniel Holth <dholth at gmail.com> wrote:

> I'd like to submit the Wheel PEPs 425 (filename metadata), 426
> (Metadata 1.3), and 427 (wheel itself) for acceptance. The format has
> been stable since May and we are preparing a patch to support it in
> pip, but we need to earn consensus before including it in the most
> widely used installer.

PEP 427:

* The installation section mentions that .py files should be compiled to .pyc/.pyo files, and that "Uninstallers should be smart enough to remove .pyc even if it is not mentioned in RECORD.". 

   Wouldn't it be better to add the compiled files to the RECORD file? That would break the digital signature, but I'm not sure if verifying the signature post-installation is useful (or if it's even
   intended to work). 

* Why is urlsafe_b64encode_nopad used to encode the hash in the record file, instead of the normal hex encoding that's directly supported by the hash module and system tools?

* The way to specify the required public key in package requirements in ugly (it looks like an abuse of setuptools' extras mechanism). Is there really no nicer way to specify this?

* As was noted before there is no threat model for the signature feature, which makes it hard to evaluate if the feature.  In particular, what is the advantage of this over PGP signatures of wheels? (PyPI already supports detached signatures, and such signatures are used more widely in the OSS world)

* RECORD.p7s is not described at all. I'm assuming this is intented to be a X.509 signature of RECORD in pkcs7 format. Why PKCS7 and not PEM? The latter seems to be easier to work with.

Ronald

From dholth at gmail.com  Wed Oct 24 14:48:11 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 24 Oct 2012 08:48:11 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <B182D06E-27C5-46AB-9A3B-0AC182724FA9@mac.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<B182D06E-27C5-46AB-9A3B-0AC182724FA9@mac.com>
Message-ID: <CAG8k2+7kAKh2R7-BSwv-EwHggHc-87dLf35jCCbb5e1cr=jMVQ@mail.gmail.com>

On Wed, Oct 24, 2012 at 7:28 AM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
> On 18 Oct, 2012, at 19:29, Daniel Holth <dholth at gmail.com> wrote:
>
>> I'd like to submit the Wheel PEPs 425 (filename metadata), 426
>> (Metadata 1.3), and 427 (wheel itself) for acceptance. The format has
>> been stable since May and we are preparing a patch to support it in
>> pip, but we need to earn consensus before including it in the most
>> widely used installer.
>
> PEP 427:
>
> * The installation section mentions that .py files should be compiled to .pyc/.pyo files, and that "Uninstallers should be smart enough to remove .pyc even if it is not mentioned in RECORD.".
>
>    Wouldn't it be better to add the compiled files to the RECORD file? That would break the digital signature, but I'm not sure if verifying the signature post-installation is useful (or if it's even
>    intended to work).

The trouble with mentioning .pyc files in RECORD is that someone can
install Python 3.4, and suddenly you have additional .pyc files,
approximately __pycache__/pyfile.cp34.pyc. So you should remove more
than what you installed anyway.

You can't verify the signature post-installation. #!python and RECORD
have been rewritten at this point.

> * Why is urlsafe_b64encode_nopad used to encode the hash in the record file, instead of the normal hex encoding that's directly supported by the hash module and system tools?

It's nice and small. The encoder is just
base64.urlsafe_b64encode(digest).rstrip('=')

> * The way to specify the required public key in package requirements in ugly (it looks like an abuse of setuptools' extras mechanism). Is there really no nicer way to specify this?
>
> * As was noted before there is no threat model for the signature feature, which makes it hard to evaluate if the feature.  In particular, what is the advantage of this over PGP signatures of wheels? (PyPI already supports detached signatures, and such signatures are used more widely in the OSS world)
>
> * RECORD.p7s is not described at all. I'm assuming this is intented to be a X.509 signature of RECORD in pkcs7 format. Why PKCS7 and not PEM? The latter seems to be easier to work with.

I am very confused about the idea that
not-downloading-the-archive-you-expected (pypi accounts getting
hacked, man-in-the-middle attacks, simply using the wrong index) is an
unrealistic threat.

It might help to think of the wheel signing scheme as a more powerful
version of the current #md5=digest instead of comparing it to PGP or
TLS. An md5 sum verifies the integrity of a single archive, the wheel
signing key verifies the integrity of any number of archives. Like the
archive digest, wheel just explains how to attach the signature to the
archive. A system for [automatically] trusting any particular key
could be built separately.

Wheel's signing scheme is similar to jarsigner. The big advantage over
PGP is that they are attached and less likely to get lost. PyPI still
supports detached signatures, even on wheel files, but they are
unpopular. Wheel gives you an additional different option.

Since the signature is over the unpacked contents, you can also change
the compression algorithm in the zipfile or append another signature
without invalidating the existing signature.

The simplified certificate model is inspired by SPKI/SDSI
(http://world.std.com/~cme/html/spki.html), Convergence
(http://convergence.io/) TACK (http://tack.io), and the general
discussion about the brokenness of the certificate authority system.
You get the raw public key without a claim that it represents anything
or anyone.

PKCS7 is the format that a US government user would be required to use
with their smartcard-based system.

I like the packagename[algorithm=key] syntax even though it started as
a hack. It fits into the existing pip requirements.txt syntax
perfectly, unlike packagename[extra]#algorithm=key, and it reads like
array indexing.

From dholth at gmail.com  Wed Oct 24 14:59:31 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 24 Oct 2012 08:59:31 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <8DC03FC0-5182-4979-B219-F1FA5E9EB56E@mac.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<8DC03FC0-5182-4979-B219-F1FA5E9EB56E@mac.com>
Message-ID: <CAG8k2+6fLwXz+WY+NgbLxRaCFLu-CPUS2Cisj3TjkSwKaGUa6w@mail.gmail.com>

On Wed, Oct 24, 2012 at 7:04 AM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
> On 18 Oct, 2012, at 19:29, Daniel Holth <dholth at gmail.com> wrote:
>
>> I'd like to submit the Wheel PEPs 425 (filename metadata), 426
>> (Metadata 1.3), and 427 (wheel itself) for acceptance. The format has
>> been stable since May and we are preparing a patch to support it in
>> pip, but we need to earn consensus before including it in the most
>> widely used installer.
>
> PEP 425:
>
> * "The version is py_version_nodot. CPython gets away with no dot, but if one is needed the underscore _ is used instead"
>
>    I don't particularly like replacing dots by underscores. That needed because you use the dot character in compressed tag sets, but why not use a comma to separate items in the compressed tag set?

> * "The platform tag is simply distutils.util.get_platform() with all hyphens - and periods . replaced with underscore _."
>
>    Why the replacement?  The need for replacement could be avoided by using a different separator between elements of a tag (for example "~" or "+"), and furthermore the platform tag is at a know
>    location, and hence the use of hyphens in the platform tag is harmless (use "python_tag, abi_tag, platform_tag = tag.split('-', 2)" to split the tag into its elements.

This is based on the longstanding convention of folding - and _
(hyphen and underscore) in built distribution filenames and using - to
separate parts.

> * "compressed tag sets"
>
>    Using '"," instead of "." to separate elements of the tag set takes away the need to replace dots in tag elements, and seems more natural to me (you'd also use comma to separate the elements
>    when you write them down in prose or python code.

I kindof like the ,

The + might transform into a space in URLs?

From larry at hastings.org  Wed Oct 24 18:04:08 2012
From: larry at hastings.org (Larry Hastings)
Date: Wed, 24 Oct 2012 09:04:08 -0700
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <k66glu$r11$1@ger.gmane.org>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org>
Message-ID: <50881178.4010606@hastings.org>

On 10/23/2012 09:29 AM, Georg Brandl wrote:
> Especially since you're suggesting a huge number of new files, I question the
> argument of better navigability.

FWIW I'm -1 on it too.  I don't see what the big deal is with "large" 
source files.  If you have difficulty finding your way around 
unicodeobject.c, that seems like more like a tooling issue to me, not a 
source code structural issue.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121024/6cbfa78c/attachment.html>

From guido at python.org  Wed Oct 24 21:19:14 2012
From: guido at python.org (Guido van Rossum)
Date: Wed, 24 Oct 2012 12:19:14 -0700
Subject: [Python-Dev] Some joker is trying to unsubscribe me
Message-ID: <CAP7+vJJa3HUXxwvJYf5FMgfRQ-o3XBLa_VErbD78=P5Q3=G4AA@mail.gmail.com>

I've received three messages in the past hour from mailman at
python.org notifying me of various attempts to receive a password
reminder or to remove me from python-dev. I hope they don't succeed.

-- 
--Guido van Rossum (python.org/~guido)

From soltysh at gmail.com  Wed Oct 24 21:45:14 2012
From: soltysh at gmail.com (Maciej Szulik)
Date: Wed, 24 Oct 2012 21:45:14 +0200
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <50874221.2090905@netwok.org>
References: <5064D399.5050208@netwok.org> <506B36DB.1090001@netwok.org>
	<CAPTjJmqGWfoCKKBHuJwJGpg-roNoQQ0NMt_HVDCAaau4U+4MQA@mail.gmail.com>
	<506B685C.3050608@netwok.org> <20121012175039.GA1284@chang>
	<50874221.2090905@netwok.org>
Message-ID: <5088454A.1060003@gmail.com>

On 10/24/2012 03:19 AM, ?ric Araujo wrote:
> Hello,
>
> Le 12/10/2012 13:50, Petri Lehtinen a ?crit :
>> It's two and a half weeks left, but I've not seen any announcements
>> yet!
> Indeed, work and other commitments took over, so we (Montr?al-Python)
> decided to move the bug day instead of announcing it late.  The date
> that would work for us is November 3rd.
>
> Brian, is it okay for Boston?
> Maciej, what about your group?
> Comitters, who could join on IRC?
>
> Sorry for the false start.
>

Eric,
We have a meeting tomorrow, I'll talk to guys, but because we're
starting Silesian Python Group, there won't be much interest yet.
I'll try to work on that ;) maybe some time in future I could organize
this kind of event and invite all of you to join us.
Nonetheless I'll try to join both events, on this and next Saturday
on IRC.

Maciej

From ncoghlan at gmail.com  Thu Oct 25 00:15:14 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 25 Oct 2012 08:15:14 +1000
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <50881178.4010606@hastings.org>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
Message-ID: <CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>

On Oct 25, 2012 2:06 AM, "Larry Hastings" <larry at hastings.org> wrote:
>
> On 10/23/2012 09:29 AM, Georg Brandl wrote:
>>
>> Especially since you're suggesting a huge number of new files, I
question the
>> argument of better navigability.
>
>
> FWIW I'm -1 on it too.  I don't see what the big deal is with "large"
source files.  If you have difficulty finding your way around
unicodeobject.c, that seems like more like a tooling issue to me, not a
source code structural issue.

OK, I need to weigh in after seeing this kind of reply. Large source files
are discouraged in general because they're a code smell that points
strongly towards a *lack of modularity* within a *complex piece of
functionality*.

Breaking such files up into separately compiled modules serves two purposes:
1. It proves that the code *isn't* a tangled monolithic mess;
2. It enlists the compilation toolchain's assistance in ensuring that
remains the case in the future.

I find complaints about the ease of searching within the file to be
misguided and irrelevant, as I can just as easily reply with "if searching
across multiple files is hard for you, use better tools, like grep, or
'Find in Files'".

Note that I also consider the "pro" argument about better navigability
inaccurate - the real gain is in *modularity*, making it clear to readers
which parts can be understood and worked on separately from each other.

We are not special snow flakes - good software engineering practice is
advisable for us as well, so a big +1 from me for breaking up the
monstrosity that is unicodeobject.c and lowering the barrier to entry for
hacking on the individual pieces. This should come with a large block
comment in unicodeobject.c explaining how the pieces are put back together
again.

However, -1 on the "faux modularity" idea of breaking up the files on disk,
but still exposing them to the compiler and linker as a monolithic block,
though. That would be completely missing the point of why large source
files are bad.

Regards,
Nick.

--
Sent from my phone, thus the relative brevity :)

>
>
> /arry
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121025/355c2b2c/attachment.html>

From thomas at python.org  Thu Oct 25 00:35:48 2012
From: thomas at python.org (Thomas Wouters)
Date: Thu, 25 Oct 2012 00:35:48 +0200
Subject: [Python-Dev] Some joker is trying to unsubscribe me
In-Reply-To: <CAP7+vJJa3HUXxwvJYf5FMgfRQ-o3XBLa_VErbD78=P5Q3=G4AA@mail.gmail.com>
References: <CAP7+vJJa3HUXxwvJYf5FMgfRQ-o3XBLa_VErbD78=P5Q3=G4AA@mail.gmail.com>
Message-ID: <CAPdQG2rszgTUK9MD013Jws+eW+=vpJ2XHbsEkLSxuW3w188PAQ@mail.gmail.com>

On Wed, Oct 24, 2012 at 9:19 PM, Guido van Rossum <guido at python.org> wrote:

> I've received three messages in the past hour from mailman at
> python.org notifying me of various attempts to receive a password
> reminder or to remove me from python-dev. I hope they don't succeed.


Are you asking us to CC you on all messages? I'm sure it could be arranged
:>

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121025/6cc494f7/attachment-0001.html>

From barry at python.org  Thu Oct 25 00:37:37 2012
From: barry at python.org (Barry Warsaw)
Date: Wed, 24 Oct 2012 18:37:37 -0400
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
Message-ID: <20121024183737.4fa6a64a@limelight.wooz.org>

On Oct 25, 2012, at 08:15 AM, Nick Coghlan wrote:

>OK, I need to weigh in after seeing this kind of reply. Large source files
>are discouraged in general because they're a code smell that points
>strongly towards a *lack of modularity* within a *complex piece of
>functionality*.

Modularity is good, and the file system structure of the project should
reflect that, but to be effective, it needs to be obvious.  It's pretty
obvious what's generally in intobject.c.  I've worked with code bases where
there's no rhyme nor reason as to what you'd find in a particular file, and
this really hurts.

It hurts even with good tools.  Remember that sometimes you don't even know
what you're looking for, so search tools may not be very useful.  For example,
sometimes you want to understand how all the pieces fit together, what the
holistic view of the subsystem is, or where the "entry points" are.  Search
tools are not very good at this, and if it's a subsystem you only interact
with occasionally, having a file system organization that makes things easier
to remember what you learned the last time you were there helps enormously.

Another point: rather than large files (or maybe in addition to them), large
functions can also be painful to navigate.  So just splitting a file into
subfiles may not be the only modularity improvement you can make.

While I'm personally -0 about splitting up unicodeobject.c, if the folks
advocating for it go ahead with it, I just ask that you do it very carefully,
with an eye toward the casual and newbie reader of our code base.

Cheers,
-Barry

From ncoghlan at gmail.com  Thu Oct 25 02:03:31 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 25 Oct 2012 10:03:31 +1000
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <20121024183737.4fa6a64a@limelight.wooz.org>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
	<20121024183737.4fa6a64a@limelight.wooz.org>
Message-ID: <CADiSq7e3VMQt9v8eLZO=Fxz4NSH8UHQZSWDF4Uv2HcDP+KXUzw@mail.gmail.com>

On Thu, Oct 25, 2012 at 8:37 AM, Barry Warsaw <barry at python.org> wrote:
> On Oct 25, 2012, at 08:15 AM, Nick Coghlan wrote:
>
>>OK, I need to weigh in after seeing this kind of reply. Large source files
>>are discouraged in general because they're a code smell that points
>>strongly towards a *lack of modularity* within a *complex piece of
>>functionality*.
>
> Modularity is good, and the file system structure of the project should
> reflect that, but to be effective, it needs to be obvious.  It's pretty
> obvious what's generally in intobject.c.  I've worked with code bases where
> there's no rhyme nor reason as to what you'd find in a particular file, and
> this really hurts.
>
> It hurts even with good tools.  Remember that sometimes you don't even know
> what you're looking for, so search tools may not be very useful.  For example,
> sometimes you want to understand how all the pieces fit together, what the
> holistic view of the subsystem is, or where the "entry points" are.  Search
> tools are not very good at this, and if it's a subsystem you only interact
> with occasionally, having a file system organization that makes things easier
> to remember what you learned the last time you were there helps enormously.

And if we were talking in the abstract, I think these would be
reasonable concerns to bring up. However, Victor's proposed division
*is* logical (especially if he goes down the path of a separate
subdirectory which will better support easy searching across all of
the unicode object related files), and I conditioned my +1 with the
requirement that a road map be provided in a leading block comment in
unicodeobject.c.

speed.python.org is also making progress, and once that is up and
running (which will happen well before any Python 3.4 release) it will
be possible to compare the numbers between 3.3 and trunk to help
determine the validity of any concerns regarding optimisations that
can be performed within a module but not across modules.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From stephen at xemacs.org  Thu Oct 25 06:22:03 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 25 Oct 2012 13:22:03 +0900
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
Message-ID: <87ip9zb2qc.fsf@uwakimon.sk.tsukuba.ac.jp>

Nick Coghlan writes:

 > OK, I need to weigh in after seeing this kind of reply. Large source files
 > are discouraged in general because they're a code smell that points
 > strongly towards a *lack of modularity* within a *complex piece of
 > functionality*.

Sure, but large numbers of tiny source files are also a code smell,
the smell of purist adherence to the literal principle of modularity
without application of judgment.

If you want to argue that the pragmatic point of view nevertheless is
to break up the file, I can see that, but I think Victor is going too
far.  (Full disclosure dept.: the call graph of the Emacs equivalents
is isomorphic to the Dungeon of Zork, so I may be a bit biased.)  You
really should speak to the question of "how many" and "what partition".

 > the real gain is in *modularity*, making it clear to readers which
 > parts can be understood and worked on separately from each other.

Yeah, so which do you think they are?  It seems to me that there are
three modules to be carved out of unicodeobject.c:

1.  The internal object management that is not exposed to Python:
    allocation, deallocation, and PEP 393 transformations.

2.  The public interface to Python implementation: methods and
    properties, including operators.

3.  Interaction with the outside world: codec implementations.  But
    conceptually, these really don't have anything to do with internal
    implementation of Unicode objects.  They're just functions that
    convert bytes to Unicode and vice versa.  In principle they can be
    written in terms of ord(), chr(), and bytes().  On the other hand,
    they're rather repetitive: "When you've seen one codec
    implementation, you've seen them all."  I see no harm in grouping
    them in one file, and possibly a gain from proximity: casual
    passers-by might see refactorings that reduce redundancy.

I'm not sure what to do with the charmap stuff.  In current CPython
head it seems incoherent to me: there's an IO codec, but there's also
unicode-to-unicode stuff (PyUnicode_Translate).  I haven't had time to
look at Victor's reorganization to see what he actually did with it,
but in terms of modularity, it seems to me that refactoring this stuff
would be a real win, as opposed to splitting the files which is
presentational improvement for the rest of the code which is pretty
modular.

As for Victor's proposal itself:

  1176 Objects/unicodecharmap.c
  1678 Objects/unicodecodecs.c
  1362 Objects/unicodeformat.c
   253 Objects/unicodeimpl.h
   733 Objects/unicodelegacy.c
  1836 Objects/unicodenew.c
  2777 Objects/unicodeobject.c
  2421 Objects/unicodeoperators.c
  1235 Objects/unicodeoscodecs.c
  1288 Objects/unicodeutfcodecs.c

As Victor himself admits, "unicodelegacy" and "unicodenew" are not
descriptive of what they contain.  In I18N discussions, "legacy" is
usually a deprectory reference to non-Unicode encodings, and I would
tend to guess this file contains codecs from the name.  A better name
might be "unicodedeprecated" (if what he really means is deprecated
APIs).

I don't understand why splitting out "unicodeoperators" is a great
idea; it's done nowhere else in CPython.  If that makes sense, why not
split out "unicodemethods" (for methods normally invoked explicitly
rather than by syntax) too?  N.B. For bytes, the corresponding file is
spelled "bytes_methods".

"unicodecodecs" vs "unicodeutfcodecs": Say what?  I would forever be
looking in the wrong one.

"unicodeoscodecs" suggests to me that these codecs are only usable on
some OSes.  If so, shouldn't the relevant OS be in the name?  If not,
the name is basically misleading IMO.

Why are any of these codecs here in unicodeobjectland in the first
place?  Sure, they're needed so that Python can find its own stuff,
but in principle *any* codec could be needed.  Is it just an heuristic
that the codecs needed for 99% of the world are here, and other codecs
live in separate modules?

Steve

From ncoghlan at gmail.com  Thu Oct 25 08:42:55 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 25 Oct 2012 16:42:55 +1000
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <87ip9zb2qc.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
	<87ip9zb2qc.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CADiSq7cVffXTYWXBfEqeVD3Maeqj+L6Wr--RE0-L2NZRuzFSXA@mail.gmail.com>

On Thu, Oct 25, 2012 at 2:22 PM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Nick Coghlan writes:
>
>  > OK, I need to weigh in after seeing this kind of reply. Large source files
>  > are discouraged in general because they're a code smell that points
>  > strongly towards a *lack of modularity* within a *complex piece of
>  > functionality*.
>
> Sure, but large numbers of tiny source files are also a code smell,
> the smell of purist adherence to the literal principle of modularity
> without application of judgment.

Absolutely. The classic example of this is Java's unfortunate
insistence on only-one-public-top-level-class-per-file. Bleh.

> If you want to argue that the pragmatic point of view nevertheless is
> to break up the file, I can see that, but I think Victor is going too
> far.  (Full disclosure dept.: the call graph of the Emacs equivalents
> is isomorphic to the Dungeon of Zork, so I may be a bit biased.)  You
> really should speak to the question of "how many" and "what partition".

Yes, I agree I was too hasty in calling the specifics of Victor's
current proposal a good idea. What raised my ire was the raft of
replies objecting to the refactoring *in principle* for completely
specious reasons like being able to search within a single file
instead of having to use tools that can search across multiple files.

unicodeobject.c is too big, and should be restructured to make any
natural modularity explicit, and provide an easier path for users that
want to understand how the unicode implementation works.

>  > the real gain is in *modularity*, making it clear to readers which
>  > parts can be understood and worked on separately from each other.
>
> Yeah, so which do you think they are?  It seems to me that there are
> three modules to be carved out of unicodeobject.c:
>
> 1.  The internal object management that is not exposed to Python:
>     allocation, deallocation, and PEP 393 transformations.
>
> 2.  The public interface to Python implementation: methods and
>     properties, including operators.
>
> 3.  Interaction with the outside world: codec implementations.  But
>     conceptually, these really don't have anything to do with internal
>     implementation of Unicode objects.  They're just functions that
>     convert bytes to Unicode and vice versa.  In principle they can be
>     written in terms of ord(), chr(), and bytes().  On the other hand,
>     they're rather repetitive: "When you've seen one codec
>     implementation, you've seen them all."  I see no harm in grouping
>     them in one file, and possibly a gain from proximity: casual
>     passers-by might see refactorings that reduce redundancy.

I suspect you and Victor are in a much better position to thrash out
the details than I am. It was the trend in the discussion to treat the
question as "split or don't split?" rather than "how should we split
it?" when a file that large should already contain some natural
splitting points if the implementation isn't a tangled monolithic
mess.

> Why are any of these codecs here in unicodeobjectland in the first
> place?  Sure, they're needed so that Python can find its own stuff,
> but in principle *any* codec could be needed.  Is it just an heuristic
> that the codecs needed for 99% of the world are here, and other codecs
> live in separate modules?

I believe it's a combination of history and whether or not they're
needed by the interpreter during the bootstrapping process before the
encodings namespace is importable.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From mal at egenix.com  Thu Oct 25 08:57:19 2012
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 25 Oct 2012 08:57:19 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <CADiSq7cVffXTYWXBfEqeVD3Maeqj+L6Wr--RE0-L2NZRuzFSXA@mail.gmail.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
	<87ip9zb2qc.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7cVffXTYWXBfEqeVD3Maeqj+L6Wr--RE0-L2NZRuzFSXA@mail.gmail.com>
Message-ID: <5088E2CF.4060000@egenix.com>

On 25.10.2012 08:42, Nick Coghlan wrote:
>> Why are any of these codecs here in unicodeobjectland in the first
>> place?  Sure, they're needed so that Python can find its own stuff,
>> but in principle *any* codec could be needed.  Is it just an heuristic
>> that the codecs needed for 99% of the world are here, and other codecs
>> live in separate modules?
> 
> I believe it's a combination of history and whether or not they're
> needed by the interpreter during the bootstrapping process before the
> encodings namespace is importable.

They are in unicodeobject.c so that the compilers can inline the
code in the various other places where they are used in the Unicode
implementation directly as necessary and because the codecs use
a lot of functions from the Unicode API (obviously), so the other
direction of inlining (Unicode API in codecs) is needed as well.

BTW: When discussing compiler optimizations, please remember that
there are more compilers out there than just GCC and also the fact
that not everyone is using the latest and greatest version of it.
Link time inlining will usually not be as efficient as compile time
optimization and we need every bit of performance we can get
for Unicode in Python 3.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 25 2012)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2012-09-27: Released eGenix PyRun 1.1.0 ...       http://egenix.com/go35
2012-09-26: Released mxODBC.Connect 2.0.1 ...     http://egenix.com/go34
2012-10-29: PyCon DE 2012, Leipzig, Germany ...             4 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Thu Oct 25 09:08:49 2012
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 25 Oct 2012 09:08:49 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <CADiSq7cVffXTYWXBfEqeVD3Maeqj+L6Wr--RE0-L2NZRuzFSXA@mail.gmail.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
	<87ip9zb2qc.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7cVffXTYWXBfEqeVD3Maeqj+L6Wr--RE0-L2NZRuzFSXA@mail.gmail.com>
Message-ID: <5088E581.3010000@egenix.com>

On 25.10.2012 08:42, Nick Coghlan wrote:
> unicodeobject.c is too big, and should be restructured to make any
> natural modularity explicit, and provide an easier path for users that
> want to understand how the unicode implementation works.

You can also achieve that goal by structuring the code in unicodeobject.c
in a more modular way. It is already structured in sections, but
there's always room for improvement, of course.

As mentioned before, it is impossible to split out various sections
into separate .c or .h files which then get included in the main
unicodeobject.c. If that's where consensus is going, I'm with Stephen
here in that such a separation should be done in higher level
chunks, rather than creating >10 new files.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 25 2012)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2012-09-27: Released eGenix PyRun 1.1.0 ...       http://egenix.com/go35
2012-09-26: Released mxODBC.Connect 2.0.1 ...     http://egenix.com/go34
2012-10-29: PyCon DE 2012, Leipzig, Germany ...             4 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From ronaldoussoren at mac.com  Thu Oct 25 11:07:13 2012
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 25 Oct 2012 11:07:13 +0200
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+6fLwXz+WY+NgbLxRaCFLu-CPUS2Cisj3TjkSwKaGUa6w@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<8DC03FC0-5182-4979-B219-F1FA5E9EB56E@mac.com>
	<CAG8k2+6fLwXz+WY+NgbLxRaCFLu-CPUS2Cisj3TjkSwKaGUa6w@mail.gmail.com>
Message-ID: <C6298E51-AABA-4230-88ED-5DFB82D4BD57@mac.com>


On 24 Oct, 2012, at 14:59, Daniel Holth <dholth at gmail.com> wrote:

> On Wed, Oct 24, 2012 at 7:04 AM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>> 
>> On 18 Oct, 2012, at 19:29, Daniel Holth <dholth at gmail.com> wrote:
>> 
>>> I'd like to submit the Wheel PEPs 425 (filename metadata), 426
>>> (Metadata 1.3), and 427 (wheel itself) for acceptance. The format has
>>> been stable since May and we are preparing a patch to support it in
>>> pip, but we need to earn consensus before including it in the most
>>> widely used installer.
>> 
>> PEP 425:
>> 
>> * "The version is py_version_nodot. CPython gets away with no dot, but if one is needed the underscore _ is used instead"
>> 
>>   I don't particularly like replacing dots by underscores. That needed because you use the dot character in compressed tag sets, but why not use a comma to separate items in the compressed tag set?
> 
>> * "The platform tag is simply distutils.util.get_platform() with all hyphens - and periods . replaced with underscore _."
>> 
>>   Why the replacement?  The need for replacement could be avoided by using a different separator between elements of a tag (for example "~" or "+"), and furthermore the platform tag is at a know
>>   location, and hence the use of hyphens in the platform tag is harmless (use "python_tag, abi_tag, platform_tag = tag.split('-', 2)" to split the tag into its elements.
> 
> This is based on the longstanding convention of folding - and _
> (hyphen and underscore) in built distribution filenames and using - to
> separate parts.

AFAIK distutils and setuptools do not replace hyphens in the platform name in the name of bdist files.

> 
>> * "compressed tag sets"
>> 
>>   Using '"," instead of "." to separate elements of the tag set takes away the need to replace dots in tag elements, and seems more natural to me (you'd also use comma to separate the elements
>>   when you write them down in prose or python code.
> 
> I kindof like the ,
> 
> The + might transform into a space in URLs?

You're right, + is not a good choice because that character must be quoted in URLs.

Ronald

From fijall at gmail.com  Thu Oct 25 11:18:53 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 25 Oct 2012 11:18:53 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <5088E2CF.4060000@egenix.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
	<87ip9zb2qc.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7cVffXTYWXBfEqeVD3Maeqj+L6Wr--RE0-L2NZRuzFSXA@mail.gmail.com>
	<5088E2CF.4060000@egenix.com>
Message-ID: <CAK5idxRufhDo=KBdEPfxeAec=y8SXVjXPC9vEVj=ySJkge2Ffw@mail.gmail.com>

On Thu, Oct 25, 2012 at 8:57 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> On 25.10.2012 08:42, Nick Coghlan wrote:
>>> Why are any of these codecs here in unicodeobjectland in the first
>>> place?  Sure, they're needed so that Python can find its own stuff,
>>> but in principle *any* codec could be needed.  Is it just an heuristic
>>> that the codecs needed for 99% of the world are here, and other codecs
>>> live in separate modules?
>>
>> I believe it's a combination of history and whether or not they're
>> needed by the interpreter during the bootstrapping process before the
>> encodings namespace is importable.
>
> They are in unicodeobject.c so that the compilers can inline the
> code in the various other places where they are used in the Unicode
> implementation directly as necessary and because the codecs use
> a lot of functions from the Unicode API (obviously), so the other
> direction of inlining (Unicode API in codecs) is needed as well.

I'm sorry to interrupt, but have you actually measured? What effect
the lack of said inlining has on *any* benchmark is definitely beyond
my ability to guess and I suspect is beyond the ability to guess of
anyone else on this list.

I challenge you to find a benchmark that is being significantly
affected (>15%) with the split proposed by Victor. It does not even
have to be a real-world one, although that would definitely buy it
more credibility.

Cheers,
fijal

From mal at egenix.com  Thu Oct 25 11:49:48 2012
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 25 Oct 2012 11:49:48 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <CAK5idxRufhDo=KBdEPfxeAec=y8SXVjXPC9vEVj=ySJkge2Ffw@mail.gmail.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
	<87ip9zb2qc.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7cVffXTYWXBfEqeVD3Maeqj+L6Wr--RE0-L2NZRuzFSXA@mail.gmail.com>
	<5088E2CF.4060000@egenix.com>
	<CAK5idxRufhDo=KBdEPfxeAec=y8SXVjXPC9vEVj=ySJkge2Ffw@mail.gmail.com>
Message-ID: <50890B3C.7020705@egenix.com>

On 25.10.2012 11:18, Maciej Fijalkowski wrote:
> On Thu, Oct 25, 2012 at 8:57 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> On 25.10.2012 08:42, Nick Coghlan wrote:
>>>> Why are any of these codecs here in unicodeobjectland in the first
>>>> place?  Sure, they're needed so that Python can find its own stuff,
>>>> but in principle *any* codec could be needed.  Is it just an heuristic
>>>> that the codecs needed for 99% of the world are here, and other codecs
>>>> live in separate modules?
>>>
>>> I believe it's a combination of history and whether or not they're
>>> needed by the interpreter during the bootstrapping process before the
>>> encodings namespace is importable.
>>
>> They are in unicodeobject.c so that the compilers can inline the
>> code in the various other places where they are used in the Unicode
>> implementation directly as necessary and because the codecs use
>> a lot of functions from the Unicode API (obviously), so the other
>> direction of inlining (Unicode API in codecs) is needed as well.
> 
> I'm sorry to interrupt, but have you actually measured? What effect
> the lack of said inlining has on *any* benchmark is definitely beyond
> my ability to guess and I suspect is beyond the ability to guess of
> anyone else on this list.
> 
> I challenge you to find a benchmark that is being significantly
> affected (>15%) with the split proposed by Victor. It does not even
> have to be a real-world one, although that would definitely buy it
> more credibility.

I think you misunderstood. What I described is the reason for having
the base codecs in unicodeobject.c.

I think we all agree that inlining has a positive effect on
performance. The scale of the effect depends on the used compiler
and platform.

Victor already mentioned that he'll check the impact of his
proposal, so let's wait for that.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 25 2012)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2012-09-27: Released eGenix PyRun 1.1.0 ...       http://egenix.com/go35
2012-09-26: Released mxODBC.Connect 2.0.1 ...     http://egenix.com/go34
2012-10-29: PyCon DE 2012, Leipzig, Germany ...             4 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From storchaka at gmail.com  Thu Oct 25 11:49:57 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 25 Oct 2012 12:49:57 +0300
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <CAK5idxRufhDo=KBdEPfxeAec=y8SXVjXPC9vEVj=ySJkge2Ffw@mail.gmail.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
	<87ip9zb2qc.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7cVffXTYWXBfEqeVD3Maeqj+L6Wr--RE0-L2NZRuzFSXA@mail.gmail.com>
	<5088E2CF.4060000@egenix.com>
	<CAK5idxRufhDo=KBdEPfxeAec=y8SXVjXPC9vEVj=ySJkge2Ffw@mail.gmail.com>
Message-ID: <k6b20r$cbj$1@ger.gmane.org>

On 25.10.12 12:18, Maciej Fijalkowski wrote:
> I challenge you to find a benchmark that is being significantly
> affected (>15%) with the split proposed by Victor. It does not even
> have to be a real-world one, although that would definitely buy it
> more credibility.

I see 10% slowdown for UTF-8 decoding for UCS2 strings, but 10% speedup 
for mostly-BMP UCS4 strings.  For encoding the situation is reversed 
(but up to +27%).  Charmap decoding speedups 10-30%.

GCC 4.4.3, 32-bit Linux.

https://bitbucket.org/storchaka/cpython-stuff/src/default/bench


From amauryfa at gmail.com  Thu Oct 25 11:59:55 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 25 Oct 2012 11:59:55 +0200
Subject: [Python-Dev] Some joker is trying to unsubscribe me
In-Reply-To: <CAP7+vJJa3HUXxwvJYf5FMgfRQ-o3XBLa_VErbD78=P5Q3=G4AA@mail.gmail.com>
References: <CAP7+vJJa3HUXxwvJYf5FMgfRQ-o3XBLa_VErbD78=P5Q3=G4AA@mail.gmail.com>
Message-ID: <CAGmFidbXe2dAUKFcJURRFkeQUsJX2z+fkwUpiaJHpDjfN6MJLQ@mail.gmail.com>

2012/10/24 Guido van Rossum <guido at python.org>:
> I've received three messages in the past hour from mailman at
> python.org notifying me of various attempts to receive a password
> reminder or to remove me from python-dev. I hope they don't succeed.

That's probably because most of your replies contain the full message,
which contains the "Unsubscribe" link, and some stupid bot followed it.

See the one below for my account. Please don't click it!

>
> --
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/amauryfa%40gmail.com

-- 
Amaury Forgeot d'Arc

From fijall at gmail.com  Thu Oct 25 12:07:41 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 25 Oct 2012 12:07:41 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <50890B3C.7020705@egenix.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
	<87ip9zb2qc.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7cVffXTYWXBfEqeVD3Maeqj+L6Wr--RE0-L2NZRuzFSXA@mail.gmail.com>
	<5088E2CF.4060000@egenix.com>
	<CAK5idxRufhDo=KBdEPfxeAec=y8SXVjXPC9vEVj=ySJkge2Ffw@mail.gmail.com>
	<50890B3C.7020705@egenix.com>
Message-ID: <CAK5idxSN9Y+TQyni6e2n8MZB7n01dbThHCGDWga1pSw0ys_6cA@mail.gmail.com>

>
> I think you misunderstood. What I described is the reason for having
> the base codecs in unicodeobject.c.
>
> I think we all agree that inlining has a positive effect on
> performance. The scale of the effect depends on the used compiler
> and platform.
>

Well. Inlining can have positive or negative effects, depending on
various details. Too much inlining causes more cache misses for
example. However, this is absolutely irrelevant if you don't create
benchmarks and run them. Guessing is seriously not a very good
optimization strategy.

Cheers,
fijal

From storchaka at gmail.com  Thu Oct 25 12:09:21 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Thu, 25 Oct 2012 13:09:21 +0300
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <50890B3C.7020705@egenix.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
	<87ip9zb2qc.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7cVffXTYWXBfEqeVD3Maeqj+L6Wr--RE0-L2NZRuzFSXA@mail.gmail.com>
	<5088E2CF.4060000@egenix.com>
	<CAK5idxRufhDo=KBdEPfxeAec=y8SXVjXPC9vEVj=ySJkge2Ffw@mail.gmail.com>
	<50890B3C.7020705@egenix.com>
Message-ID: <k6b34j$mjj$1@ger.gmane.org>

On 25.10.12 12:49, M.-A. Lemburg wrote:
> I think you misunderstood. What I described is the reason for having
> the base codecs in unicodeobject.c.

For example PyUnicode_FromStringAndSize and PyUnicode_FromString are 
thin wrappers around PyUnicode_DecodeUTF8Stateful.  I think this is a 
reason to keep this functions together.



From ncoghlan at gmail.com  Thu Oct 25 13:11:18 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 25 Oct 2012 21:11:18 +1000
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <CAK5idxSN9Y+TQyni6e2n8MZB7n01dbThHCGDWga1pSw0ys_6cA@mail.gmail.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
	<87ip9zb2qc.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7cVffXTYWXBfEqeVD3Maeqj+L6Wr--RE0-L2NZRuzFSXA@mail.gmail.com>
	<5088E2CF.4060000@egenix.com>
	<CAK5idxRufhDo=KBdEPfxeAec=y8SXVjXPC9vEVj=ySJkge2Ffw@mail.gmail.com>
	<50890B3C.7020705@egenix.com>
	<CAK5idxSN9Y+TQyni6e2n8MZB7n01dbThHCGDWga1pSw0ys_6cA@mail.gmail.com>
Message-ID: <CADiSq7dnbC23ZJatbjtrie7Dz1xY86R5-5P_Tb143e8RWKomWw@mail.gmail.com>

On Thu, Oct 25, 2012 at 8:07 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>>
>> I think you misunderstood. What I described is the reason for having
>> the base codecs in unicodeobject.c.
>>
>> I think we all agree that inlining has a positive effect on
>> performance. The scale of the effect depends on the used compiler
>> and platform.
>>
>
> Well. Inlining can have positive or negative effects, depending on
> various details. Too much inlining causes more cache misses for
> example. However, this is absolutely irrelevant if you don't create
> benchmarks and run them. Guessing is seriously not a very good
> optimization strategy.

Yep, that's why I made the point that speed.python.org should be a
going concern well before 3.4 release, and will be able to let us know
if we have a problem relative to 3.3.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From breamoreboy at yahoo.co.uk  Thu Oct 25 15:33:13 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 25 Oct 2012 14:33:13 +0100
Subject: [Python-Dev] Fwd: Python 3.3 can't sort memoryviews as they're
	unorderable
In-Reply-To: <CADiSq7cSWaceY72CqjXAsUq_YQ54VRi8ACaqT092C+UsE4WVww@mail.gmail.com>
References: <k60lsp$jok$1@ger.gmane.org> <k64mpo$uao$1@ger.gmane.org>
	<CADiSq7cSWaceY72CqjXAsUq_YQ54VRi8ACaqT092C+UsE4WVww@mail.gmail.com>
Message-ID: <k6betg$1vm$1@ger.gmane.org>

On 24/10/2012 13:19, Nick Coghlan wrote:
> (Oops, originally replied only to Mark)
>
> Is a 3x3 array greater or less than a 2x4 array or another 3x3 array?
>
> The contents of a 1D memory view may be sortable, but the "logical
> structure" part isn't, and neither is any multi-dimensional view.
>
> I'm surprised by the lack of inheritance support though - is that a
> regression from 3.2? If yes, that's definitely a bug to be fixed in a 3.3
> maintenance release, otherwise it's probably a feature request for 3.4.
>
> Cheers,
> Nick.
>
> --
> Sent from my phone, thus the relative brevity :)
>
>
>

The lack of inheritance support is the same in Python 2.7.3 so I doubt 
that there's any change there.

Quoting Nick Coghlan from http://bugs.python.org/issue15622#msg167957 
"PEP 3118 contains way too many errors (as has been found out the hard 
way) to be considered a normative document.".  I can't say that this 
gives me a great deal of confidence.  It strikes me that a lot of code 
has been written, tested and released without having anything like a 
requirement.  For example when is any given memoryview equal to or not 
equal to any other memoryview?

-- 
Cheers.

Mark Lawrence.


From stefan at bytereef.org  Thu Oct 25 16:06:25 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Thu, 25 Oct 2012 16:06:25 +0200
Subject: [Python-Dev] Fwd: Python 3.3 can't sort memoryviews as they're
 unorderable
In-Reply-To: <k6betg$1vm$1@ger.gmane.org>
References: <k60lsp$jok$1@ger.gmane.org> <k64mpo$uao$1@ger.gmane.org>
	<CADiSq7cSWaceY72CqjXAsUq_YQ54VRi8ACaqT092C+UsE4WVww@mail.gmail.com>
	<k6betg$1vm$1@ger.gmane.org>
Message-ID: <20121025140624.GA28959@sleipnir.bytereef.org>

Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> I can't say that this gives me a great deal of confidence. It strikes me
> that a lot of code has been written, tested and released without having
> anything like a requirement. For example when is any given memoryview
> equal to or not equal to any other memoryview?

A lot of documentation has been written and released as well. Use it.
In case you don't know: If a PEP and the current documentation diverge,
the documentation takes precedence.


Stefan Krah



From guido at python.org  Thu Oct 25 16:49:51 2012
From: guido at python.org (Guido van Rossum)
Date: Thu, 25 Oct 2012 07:49:51 -0700
Subject: [Python-Dev] Some joker is trying to unsubscribe me
In-Reply-To: <CAGmFidbXe2dAUKFcJURRFkeQUsJX2z+fkwUpiaJHpDjfN6MJLQ@mail.gmail.com>
References: <CAP7+vJJa3HUXxwvJYf5FMgfRQ-o3XBLa_VErbD78=P5Q3=G4AA@mail.gmail.com>
	<CAGmFidbXe2dAUKFcJURRFkeQUsJX2z+fkwUpiaJHpDjfN6MJLQ@mail.gmail.com>
Message-ID: <CAP7+vJ+VAxipDWF1OD=eh6TSqURG7e--dEuqu6iYWtQ6D3v1Pw@mail.gmail.com>

On Thu, Oct 25, 2012 at 2:59 AM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
> 2012/10/24 Guido van Rossum <guido at python.org>:
>> I've received three messages in the past hour from mailman at
>> python.org notifying me of various attempts to receive a password
>> reminder or to remove me from python-dev. I hope they don't succeed.
>
> That's probably because most of your replies contain the full message,
> which contains the "Unsubscribe" link, and some stupid bot followed it.
>
> See the one below for my account. Please don't click it!

Or, more likely, someone *else* posted that link in their quoting of
my message. Good hypothesis!

-- 
--Guido van Rossum (python.org/~guido)

From solipsis at pitrou.net  Thu Oct 25 16:52:55 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 25 Oct 2012 16:52:55 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <CADiSq7e3VMQt9v8eLZO=Fxz4NSH8UHQZSWDF4Uv2HcDP+KXUzw@mail.gmail.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
	<20121024183737.4fa6a64a@limelight.wooz.org>
	<CADiSq7e3VMQt9v8eLZO=Fxz4NSH8UHQZSWDF4Uv2HcDP+KXUzw@mail.gmail.com>
Message-ID: <k6bjo7$evl$1@ger.gmane.org>

Le 25/10/2012 02:03, Nick Coghlan a ?crit :
>
> speed.python.org is also making progress, and once that is up and
> running (which will happen well before any Python 3.4 release) it will
> be possible to compare the numbers between 3.3 and trunk to help
> determine the validity of any concerns regarding optimisations that
> can be performed within a module but not across modules.

Nobody needs speed.python.org to run benchmarks before and after a 
specific change, though.  Cloning http://hg.python.org/benchmarks and 
using the perf.py runner is everything that is needed.

Moreover, you would want to run benchmarks *before* committing and 
pushing the changes. We don't want the huge splitting to be recorded and 
then backed out in the repository history.

Regards

Antoine.



From solipsis at pitrou.net  Thu Oct 25 16:56:11 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 25 Oct 2012 16:56:11 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
Message-ID: <k6bjua$evl$2@ger.gmane.org>

Le 25/10/2012 00:15, Nick Coghlan a ?crit :
>
> However, -1 on the "faux modularity" idea of breaking up the files on
> disk, but still exposing them to the compiler and linker as a monolithic
> block, though. That would be completely missing the point of why large
> source files are bad.

I disagree with you. Source files are meant to be read by humans, we 
don't really care whether the compiler has a modular view of the source 
code.

Regards

Antoine.



From larry at hastings.org  Thu Oct 25 17:13:53 2012
From: larry at hastings.org (Larry Hastings)
Date: Thu, 25 Oct 2012 08:13:53 -0700
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
Message-ID: <50895731.7040505@hastings.org>


On 10/24/2012 03:15 PM, Nick Coghlan wrote:
> Breaking such files up into separately compiled modules serves two 
> purposes:
>
> 1. It proves that the code *isn't* a tangled monolithic mess;
> 2. It enlists the compilation toolchain's assistance in ensuring that 
> remains the case in the future.
>

Either the code is a "tangled monolithic mess" or it isn't.  If it is, 
then let's fix that, regardless of the size of the file.  If it isn't, I 
don't see breaking up the code among multiple files as providing any 
benefit.  And I see no need for the toolchain's assistance to help us do 
something without benefit.  The line count of the file is essentially 
unrelated to its inherent quality / maintainability.


> We are not special snow flakes - good software engineering practice is 
> advisable for us as well, so a big +1 from me for breaking up the 
> monstrosity that is unicodeobject.c and lowering the barrier to entry 
> for hacking on the individual pieces. This should come with a large 
> block comment in unicodeobject.c explaining how the pieces are put 
> back together again.
>

I'm all for good software engineering practice.  But can you cite 
objective reasons why large source files are provably bad?  Not "tangled 
monolithic messes", not poorly-factored code.  I agree that those are 
bad--but so far nobody has proposed that either of those is true about 
unicodeobject.c (unless you are implicitly doing so above), nor have 
they proposed credible remedies.  All I've seen is that unicodeobject.c 
is a large file, and some people want to break it up into smaller 
files.  I have yet to see anything but handwaving as justification.  For 
example, what is this barrier to entry you suggest exists to hacking on 
the str object, that will apparently be dispelled simply by splitting 
one file into multiple files?

Someone proposed breaking up unicodeobject.c into three distinct 
subsystems and putting those in separate files.  I still don't agree.  
It seems natural to me to have everything associated with the str object 
in one file, just as we do with every other object I can think of.  If 
this were a genuinely good idea, we should consider doing it with every 
similar object.  But nobody is proposing that.  My guess is because the 
other files in CPython are "small enough".  At which point we're right 
back to the primary motivation simply being the line count of 
unicodeobject.c, as a purely aesthetic and subjective judgment.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121025/cd0ecdbd/attachment.html>

From breamoreboy at yahoo.co.uk  Thu Oct 25 17:57:22 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Thu, 25 Oct 2012 16:57:22 +0100
Subject: [Python-Dev] Fwd: Python 3.3 can't sort memoryviews as they're
	unorderable
In-Reply-To: <20121025140624.GA28959@sleipnir.bytereef.org>
References: <k60lsp$jok$1@ger.gmane.org> <k64mpo$uao$1@ger.gmane.org>
	<CADiSq7cSWaceY72CqjXAsUq_YQ54VRi8ACaqT092C+UsE4WVww@mail.gmail.com>
	<k6betg$1vm$1@ger.gmane.org>
	<20121025140624.GA28959@sleipnir.bytereef.org>
Message-ID: <k6bnb9$l33$1@ger.gmane.org>

On 25/10/2012 15:06, Stefan Krah wrote:
> Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>> I can't say that this gives me a great deal of confidence. It strikes me
>> that a lot of code has been written, tested and released without having
>> anything like a requirement. For example when is any given memoryview
>> equal to or not equal to any other memoryview?
>
> A lot of documentation has been written and released as well. Use it.
> In case you don't know: If a PEP and the current documentation diverge,
> the documentation takes precedence.
>
>
> Stefan Krah
>
>

<Aussies please skip>
Thanks for completely snipping the context that I gave, it gives me the 
sense that my bodyline tactics have you on the back foot.
</Aussies please skip>

The documentation states

"__eq__(exporter) A memoryview and a PEP 3118 exporter are equal if 
their shapes are equivalent and if all corresponding values are equal 
when the operands? respective format codes are interpreted using struct 
syntax.".

Where is the requirement that states this is all that the implementation 
provides?  In the savaged PEP 3118?  Why can't I have a situation such 
that two memoryviews that I've created that meet the criteria above 
shouldn't be tested using the other comparison operators?  I'm guessing 
that I've missed something that's blatantly obvious to everybody except 
myself.  I can't find a rationale anywhere as to why I can't subclass 
memoryviews for my code, so I can't work around what I perceive as a 
glaring deficiency.  Is it a case whereby even consenting adults aren't 
allowed?

This strikes me as so different to the Python that I've been using for 
the last 10 years that it stood out, at least to me, like a sore thumb. 
  Perhaps I need yet another visit to the opticians?

-- 
Cheers.

Mark Lawrence.


From steve at pearwood.info  Thu Oct 25 18:18:47 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 26 Oct 2012 03:18:47 +1100
Subject: [Python-Dev] Fwd: Python 3.3 can't sort memoryviews as they're
 unorderable
In-Reply-To: <k6bnb9$l33$1@ger.gmane.org>
References: <k60lsp$jok$1@ger.gmane.org> <k64mpo$uao$1@ger.gmane.org>
	<CADiSq7cSWaceY72CqjXAsUq_YQ54VRi8ACaqT092C+UsE4WVww@mail.gmail.com>
	<k6betg$1vm$1@ger.gmane.org>
	<20121025140624.GA28959@sleipnir.bytereef.org>
	<k6bnb9$l33$1@ger.gmane.org>
Message-ID: <50896667.4060309@pearwood.info>

On 26/10/12 02:57, Mark Lawrence complained that he can't subclass memoryviews:

> I'm guessing that I've missed something that's blatantly obvious to
>everybody except myself. I can't find a rationale anywhere as to why
>I can't subclass memoryviews for my code, so I can't work around
>what I perceive as a glaring deficiency. Is it a case whereby even
>consenting adults aren't allowed?
>
> This strikes me as so different to the Python that I've been using
>for the last 10 years that it stood out, at least to me, like a sore
>thumb. Perhaps I need yet another visit to the opticians?

Possibly. There are many things that you can't subclass in Python.

NoneType
NotImplementedType
Ellipsis
functions
slices
frames
tracebacks

and probably others.


As annoying as it is when you run into something like this, it's hardly
without precedent. Hell, for about half of Python's existence, you
couldn't even subtype basic types like int, str and list!

Subclassing is not the only way to add functionality to a class. While
it is much less convenient with new-style classes than classic classes,
why don't you try delegation? Actually, since the problem you are
trying to solve is to sort a list of memoryviews, you don't even need
delegation. You just need a dirt-simple key function.

py> mv = memoryview
py> x = list(map(mv, (b'xyz', b'abc', b'pqr')))
py> x.sort(key=lambda x: x.obj)
py> [a.obj for a in x]
[b'abc', b'pqr', b'xyz']


What was the problem again?

:)



-- 
Steven

From josen.figueroa at unixmexico.org  Thu Oct 25 19:38:40 2012
From: josen.figueroa at unixmexico.org (Jose Figueroa)
Date: Thu, 25 Oct 2012 12:38:40 -0500
Subject: [Python-Dev] Who I am
Message-ID: <50897920.6050109@unixmexico.org>

Hello pythoners!

I am Jose Figueroa from Mexico. I work usually with C/C++, Ruby and PHP 
(yeah I know =( ) but I will start again with python because I got some 
free time after finishing a project.

I gonna use python for my Master Degree in Computer Sciences.

Thanks
Jose Figueroa



From phd at phdru.name  Thu Oct 25 20:08:27 2012
From: phd at phdru.name (Oleg Broytman)
Date: Thu, 25 Oct 2012 22:08:27 +0400
Subject: [Python-Dev] Who I am
In-Reply-To: <50897920.6050109@unixmexico.org>
References: <50897920.6050109@unixmexico.org>
Message-ID: <20121025180827.GA14113@iskra.aviel.ru>

Hello.

   This mailing list is to work on developing Python (adding new
features to Python itself and fixing bugs); if you are going to only use
Python but not develop it probably python-list/comp.lang.python mailing
list/news group is the best place. See http://www.python.org/community/
   In any case -- welcome!

On Thu, Oct 25, 2012 at 12:38:40PM -0500, Jose Figueroa <josen.figueroa at unixmexico.org> wrote:
> Hello pythoners!
> 
> I am Jose Figueroa from Mexico. I work usually with C/C++, Ruby and
> PHP (yeah I know =( ) but I will start again with python because I
> got some free time after finishing a project.
> 
> I gonna use python for my Master Degree in Computer Sciences.
> 
> Thanks
> Jose Figueroa

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From solipsis at pitrou.net  Thu Oct 25 23:39:19 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 25 Oct 2012 23:39:19 +0200
Subject: [Python-Dev] Split unicodeobject.c into subfiles
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
	<50895731.7040505@hastings.org>
Message-ID: <20121025233919.77111cb9@pitrou.net>

On Thu, 25 Oct 2012 08:13:53 -0700
Larry Hastings <larry at hastings.org> wrote:
> 
> I'm all for good software engineering practice.  But can you cite 
> objective reasons why large source files are provably bad?  Not "tangled 
> monolithic messes", not poorly-factored code.  I agree that those are 
> bad--but so far nobody has proposed that either of those is true about 
> unicodeobject.c (unless you are implicitly doing so above)

Well, "tangled monolithic mess" is quite true about unicodeobject.c,
IMO.
Seriously, I agree with Victor: navigating around unicodeobject.c is a
PITA. Perhaps it isn't if you are using emacs, or you have 35 fingers,
or just a lot of spare time, but in my experience it's painful.

Regards

Antoine.



From stephen at xemacs.org  Fri Oct 26 04:35:38 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 26 Oct 2012 11:35:38 +0900
Subject: [Python-Dev] Split unicodeobject.c into subfiles
In-Reply-To: <20121025233919.77111cb9@pitrou.net>
References: <CAMpsgwbQRZMugSprcM9CQGnraFG39AKpDAz-b1YM2j-ag=1Stg@mail.gmail.com>
	<CAPZV6o_0hANb7QbnJwF-j7oRak8oCBpvjsp0uvi+3XHWyOA0KA@mail.gmail.com>
	<k66glu$r11$1@ger.gmane.org> <50881178.4010606@hastings.org>
	<CADiSq7dvFyjMBBs3nBJbJTrShkMhDFbrjPXZnJ+b=PHS64zmDQ@mail.gmail.com>
	<50895731.7040505@hastings.org>
	<20121025233919.77111cb9@pitrou.net>
Message-ID: <87fw52ark5.fsf@uwakimon.sk.tsukuba.ac.jp>

Antoine Pitrou writes:

 > Well, "tangled monolithic mess" is quite true about unicodeobject.c,
 > IMO.

s/object.c// and your point remains valid.  Just reading the table of
contents for UTR#17 (http://www.unicode.org/reports/tr17/) should
convince you that it's not going to be easy to produce an elegant
implementation!

 > Seriously, I agree with Victor: navigating around unicodeobject.c is a
 > PITA. Perhaps it isn't if you are using emacs, or you have 35 fingers,
 > or just a lot of spare time, but in my experience it's painful.

Sure, but I don't know of a Unicode implementation which isn't.

I don't think that having a unicode/*.[ch] with a dozen files
(including the README etc) in it is going to make it much more
navigable.  If there are too many files, it's going to be a PITA to
maintain because there won't be an obvious place to put certain
functions.  Eg, I've already mentioned my suspicions about the charmap
code (I apologize for not reading Victor's code to confirm them).

I don't object in principle to splitting the unicodeobject.c.  At the
very least, with all due respect to MAL, XEmacs experience with coding
systems (the Emacs equivalent of Python codecs) suggests that there is
very little to be lost by moving the codec implementations to a
separate file from the Unicode object implementation.  (Here I'm
talking about codecs in the narrow sense of wire-format to Python3 str
and back, not the more general Python2 sense that included zip and
base64 and so on.  Ie, PyUnicode_Translate is not a codec in the
relevant sense.)

On the other hand, I wouldn't be surprised if (despite my earlier
suggestion) codecs and unicode object internals need a close
relationship.  (My intuition and sense of style says splitting codecs
from the low level memory management and PEP 393 stuff is a good idea,
but I'm not confident it would have no impact on performance.)

From ronaldoussoren at mac.com  Fri Oct 26 09:54:00 2012
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Fri, 26 Oct 2012 09:54:00 +0200
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+7kAKh2R7-BSwv-EwHggHc-87dLf35jCCbb5e1cr=jMVQ@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<B182D06E-27C5-46AB-9A3B-0AC182724FA9@mac.com>
	<CAG8k2+7kAKh2R7-BSwv-EwHggHc-87dLf35jCCbb5e1cr=jMVQ@mail.gmail.com>
Message-ID: <74DF21FC-447C-4CB0-A55B-D8CC839B165D@mac.com>


On 24 Oct, 2012, at 14:48, Daniel Holth <dholth at gmail.com> wrote:

> On Wed, Oct 24, 2012 at 7:28 AM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>> 
>> On 18 Oct, 2012, at 19:29, Daniel Holth <dholth at gmail.com> wrote:
>> 
>>> I'd like to submit the Wheel PEPs 425 (filename metadata), 426
>>> (Metadata 1.3), and 427 (wheel itself) for acceptance. The format has
>>> been stable since May and we are preparing a patch to support it in
>>> pip, but we need to earn consensus before including it in the most
>>> widely used installer.
>> 
>> PEP 427:
>> 
>> * The installation section mentions that .py files should be compiled to .pyc/.pyo files, and that "Uninstallers should be smart enough to remove .pyc even if it is not mentioned in RECORD.".
>> 
>>   Wouldn't it be better to add the compiled files to the RECORD file? That would break the digital signature, but I'm not sure if verifying the signature post-installation is useful (or if it's even
>>   intended to work).
> 
> The trouble with mentioning .pyc files in RECORD is that someone can
> install Python 3.4, and suddenly you have additional .pyc files,
> approximately __pycache__/pyfile.cp34.pyc. So you should remove more
> than what you installed anyway.
> 
> You can't verify the signature post-installation. #!python and RECORD
> have been rewritten at this point.
> 
>> * Why is urlsafe_b64encode_nopad used to encode the hash in the record file, instead of the normal hex encoding that's directly supported by the hash module and system tools?
> 
> It's nice and small. The encoder is just
> base64.urlsafe_b64encode(digest).rstrip('=')

But is the size difference really important? The wheel file itself is compressed, and the additional amount of space needed on installation shouldn't be a problem.  The advantage of using hexdigest is that both the "classic" MD5 checksum and the new tagged checksums you propose then use the same encoding for the signature.

> 
>> * The way to specify the required public key in package requirements in ugly (it looks like an abuse of setuptools' extras mechanism). Is there really no nicer way to specify this?
>> 
>> * As was noted before there is no threat model for the signature feature, which makes it hard to evaluate if the feature.  In particular, what is the advantage of this over PGP signatures of wheels? (PyPI already supports detached signatures, and such signatures are used more widely in the OSS world)
>> 
>> * RECORD.p7s is not described at all. I'm assuming this is intented to be a X.509 signature of RECORD in pkcs7 format. Why PKCS7 and not PEM? The latter seems to be easier to work with.
> 
> I am very confused about the idea that
> not-downloading-the-archive-you-expected (pypi accounts getting
> hacked, man-in-the-middle attacks, simply using the wrong index) is an
> unrealistic threat.

You don't mention the threats you try to protect against in the PEP.   Users are still somewhat vulnerable to the attacks to mention when the download new software, they still have to get the public key somewhere. In the example of using a requirements.txt file with public keys I'd still have to get that file from somewhere and maybe that location was attacked.

> 
> It might help to think of the wheel signing scheme as a more powerful
> version of the current #md5=digest instead of comparing it to PGP or
> TLS. An md5 sum verifies the integrity of a single archive, the wheel
> signing key verifies the integrity of any number of archives. Like the
> archive digest, wheel just explains how to attach the signature to the
> archive. A system for [automatically] trusting any particular key
> could be built separately.
> 
> Wheel's signing scheme is similar to jarsigner. The big advantage over
> PGP is that they are attached and less likely to get lost. PyPI still
> supports detached signatures, even on wheel files, but they are
> unpopular. Wheel gives you an additional different option.

RPM uses embedded PGP signatures, and those are easy enough to use.    PGP signatures on PyPI require a PGP installation on the users machine, your scheme at least has the advantage of not needing additional software. 

> 
> Since the signature is over the unpacked contents, you can also change
> the compression algorithm in the zipfile or append another signature
> without invalidating the existing signature.
> 
> The simplified certificate model is inspired by SPKI/SDSI
> (http://world.std.com/~cme/html/spki.html), Convergence
> (http://convergence.io/) TACK (http://tack.io), and the general
> discussion about the brokenness of the certificate authority system.

I've added these to my reading list. I know just enough of crypto/signatures to be dangerous, which might explain why I worry about something that isn't old and used a lot.

> You get the raw public key without a claim that it represents anything
> or anyone.

Simularly to the CA system :-)

> 
> PKCS7 is the format that a US government user would be required to use
> with their smartcard-based system.
> 
> I like the packagename[algorithm=key] syntax even though it started as
> a hack. It fits into the existing pip requirements.txt syntax
> perfectly, unlike packagename[extra]#algorithm=key, and it reads like
> array indexing.

I don't like the extras hack, but don't have a better solution. 

Ronald

From p.f.moore at gmail.com  Fri Oct 26 10:11:50 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 26 Oct 2012 09:11:50 +0100
Subject: [Python-Dev] [Distutils]  accept the wheel PEPs 425, 426, 427
In-Reply-To: <74DF21FC-447C-4CB0-A55B-D8CC839B165D@mac.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<B182D06E-27C5-46AB-9A3B-0AC182724FA9@mac.com>
	<CAG8k2+7kAKh2R7-BSwv-EwHggHc-87dLf35jCCbb5e1cr=jMVQ@mail.gmail.com>
	<74DF21FC-447C-4CB0-A55B-D8CC839B165D@mac.com>
Message-ID: <CACac1F8P0+PhMZZc3UbtN1LHv=cVvD5YzxEUT9UUGHgRQ3f6GQ@mail.gmail.com>

On 26 October 2012 08:54, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>>
>> It's nice and small. The encoder is just
>> base64.urlsafe_b64encode(digest).rstrip('=')
>
> But is the size difference really important? The wheel file itself is compressed, and the additional
> amount of space needed on installation shouldn't be a problem.  The advantage of using hexdigest
> is that both the "classic" MD5 checksum and the new tagged checksums you propose then use
> the same encoding for the signature.

I agree. This encoding seems to be a micro-optimisation with no real
justification. I'd prefer to see hexdigest used, as (a) it means md5
is not a special case, and (b) there's not a proliferation of 1-line
functions in use code doing that b64encode/strip dance.

With hexdigest, the syntax is just [algorithm=]hexdigest, where
algorithm defaults to md5. Much simpler to describe and work with.

Paul.

From rovitotv at gmail.com  Fri Oct 26 16:28:33 2012
From: rovitotv at gmail.com (Todd Rovito)
Date: Fri, 26 Oct 2012 10:28:33 -0400
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <5088454A.1060003@gmail.com>
References: <5064D399.5050208@netwok.org> <506B36DB.1090001@netwok.org>
	<CAPTjJmqGWfoCKKBHuJwJGpg-roNoQQ0NMt_HVDCAaau4U+4MQA@mail.gmail.com>
	<506B685C.3050608@netwok.org> <20121012175039.GA1284@chang>
	<50874221.2090905@netwok.org> <5088454A.1060003@gmail.com>
Message-ID: <CALZ1apyks1CJuv4y7x3hxm2NyYzGd0KYqv12CP-fnzUd=MOPLQ@mail.gmail.com>

On Wed, Oct 24, 2012 at 3:45 PM, Maciej Szulik <soltysh at gmail.com> wrote:
> On 10/24/2012 03:19 AM, ?ric Araujo wrote:
>>
>> Hello,
>>
>> Le 12/10/2012 13:50, Petri Lehtinen a ?crit :
>>>
>>> It's two and a half weeks left, but I've not seen any announcements
>>> yet!
>>
>> Indeed, work and other commitments took over, so we (Montr?al-Python)
>> decided to move the bug day instead of announcing it late.  The date
>> that would work for us is November 3rd.
>>
>> Brian, is it okay for Boston?
>> Maciej, what about your group?
>> Comitters, who could join on IRC?
>>
>> Sorry for the false start.
>>
>
> Eric,
> We have a meeting tomorrow, I'll talk to guys, but because we're
> starting Silesian Python Group, there won't be much interest yet.
> I'll try to work on that ;) maybe some time in future I could organize
> this kind of event and invite all of you to join us.
> Nonetheless I'll try to join both events, on this and next Saturday
> on IRC.

I will see if I can get some folks from Py Ohio to join in on November
3rd for sure I will be available to help.

From status at bugs.python.org  Fri Oct 26 18:07:25 2012
From: status at bugs.python.org (Python tracker)
Date: Fri, 26 Oct 2012 18:07:25 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20121026160725.8F9BB1CBE4@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2012-10-19 - 2012-10-26)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3824 ( +9)
  closed 24283 (+34)
  total  28107 (+43)

Open issues with patches: 1730 


Issues opened (32)
==================

#16274: test_asyncore failing on Solaris 10 2.7 (nitrogen/s10)
http://bugs.python.org/issue16274  reopened by trent

#16288: TextTestResult uses TestCase.__str__() which isn't customisabl
http://bugs.python.org/issue16288  opened by rbcollins

#16290: PyComplex_AsCComplex should allow __complex__ to return float.
http://bugs.python.org/issue16290  opened by mark.dickinson

#16291: Enable darwin-host cross compilation. Tested with build=linux,
http://bugs.python.org/issue16291  opened by Ray.Donnelly

#16292: Cross compilation fixes (general)
http://bugs.python.org/issue16292  opened by Ray.Donnelly

#16293: curses.ungetch raises OverflowError when given -1
http://bugs.python.org/issue16293  opened by Julian

#16296: Patch to fix building on Win32/64 under VS 2010
http://bugs.python.org/issue16296  opened by silverbacknet

#16298: httplib.HTTPResponse.read could potentially leave the socket o
http://bugs.python.org/issue16298  opened by eranrund

#16300: Windows raises ValueError for file:///file/not/exists instead 
http://bugs.python.org/issue16300  opened by orsenthil

#16301: localhost() and thishost() in urllib/request.py
http://bugs.python.org/issue16301  opened by orsenthil

#16304: re: Match Objects always have a boolean value of  True
http://bugs.python.org/issue16304  opened by vrutsky

#16306: Multiple error line for unknown command line parameter
http://bugs.python.org/issue16306  opened by tweksteen

#16307: multiprocess.pool.map_async callables not working
http://bugs.python.org/issue16307  opened by Bbb

#16308: Undocumented (?) behaviour change in argparse from 3.2.3 to 3.
http://bugs.python.org/issue16308  opened by telmich

#16309: "PYTHONPATH=" different from no PYTHONPATH at all
http://bugs.python.org/issue16309  opened by arigo

#16310: zipfile: allow surrogates in filenames
http://bugs.python.org/issue16310  opened by stefanholek

#16311: Use _PyUnicodeWriter API in text decoders
http://bugs.python.org/issue16311  opened by haypo

#16312: New command line option supported by unittest.main() for runni
http://bugs.python.org/issue16312  opened by piotr.dobrogost

#16316: Support xz compression in mimetypes module
http://bugs.python.org/issue16316  opened by serhiy.storchaka

#16317: Mention lzma module in tutorial
http://bugs.python.org/issue16317  opened by serhiy.storchaka

#16318: FTP_TLS in ftplib not supporting prot_p storlines in FTP7.5
http://bugs.python.org/issue16318  opened by Jeremy.Brock

#16320: Establish order in bytes/string dependencies
http://bugs.python.org/issue16320  opened by serhiy.storchaka

#16321: Move eq.h out of stringlib
http://bugs.python.org/issue16321  opened by serhiy.storchaka

#16322: time.tzname on Python 3.3.0 for Windows is decoded by wrong en
http://bugs.python.org/issue16322  opened by msmhrt

#16323: Wrong C API documentation for locale encoding
http://bugs.python.org/issue16323  opened by serhiy.storchaka

#16324: MIMEText __init__ does not support Charset instance
http://bugs.python.org/issue16324  opened by claudep

#16325: PEP??8 refers to reference to PEP 8
http://bugs.python.org/issue16325  opened by serhiy.storchaka

#16326: distutils build_ext fails to set library_dirs in 2.7.2 on Linu
http://bugs.python.org/issue16326  opened by Andy.Salnikov

#16327: subprocess.Popen leaks file descriptors on os.fork() failure
http://bugs.python.org/issue16327  opened by Mark.Gius

#16328: win_add2path.py sets wrong user path
http://bugs.python.org/issue16328  opened by Emil.Styrke

#16329: mimetypes does not support webm type
http://bugs.python.org/issue16329  opened by giampaolo.rodola

#16330: Use surrogate-related macros
http://bugs.python.org/issue16330  opened by serhiy.storchaka



Most recent 15 issues with no replies (15)
==========================================

#16330: Use surrogate-related macros
http://bugs.python.org/issue16330

#16326: distutils build_ext fails to set library_dirs in 2.7.2 on Linu
http://bugs.python.org/issue16326

#16323: Wrong C API documentation for locale encoding
http://bugs.python.org/issue16323

#16321: Move eq.h out of stringlib
http://bugs.python.org/issue16321

#16320: Establish order in bytes/string dependencies
http://bugs.python.org/issue16320

#16317: Mention lzma module in tutorial
http://bugs.python.org/issue16317

#16316: Support xz compression in mimetypes module
http://bugs.python.org/issue16316

#16312: New command line option supported by unittest.main() for runni
http://bugs.python.org/issue16312

#16298: httplib.HTTPResponse.read could potentially leave the socket o
http://bugs.python.org/issue16298

#16287: Sporadic test_utime() failures on Solaris
http://bugs.python.org/issue16287

#16283: ctypes.util.find_library does not find all DLLs anymore
http://bugs.python.org/issue16283

#16281: TODO in tailmatch(): it does not support backward in all cases
http://bugs.python.org/issue16281

#16272: C-API documentation clarification for tp_dictoffset
http://bugs.python.org/issue16272

#16253: Docs for PyArg_ParseTupleAndKeywords say it doesn't accept nes
http://bugs.python.org/issue16253

#16242: Pickle and __getattr__
http://bugs.python.org/issue16242



Most recent 15 issues waiting for review (15)
=============================================

#16330: Use surrogate-related macros
http://bugs.python.org/issue16330

#16329: mimetypes does not support webm type
http://bugs.python.org/issue16329

#16327: subprocess.Popen leaks file descriptors on os.fork() failure
http://bugs.python.org/issue16327

#16325: PEP??8 refers to reference to PEP 8
http://bugs.python.org/issue16325

#16324: MIMEText __init__ does not support Charset instance
http://bugs.python.org/issue16324

#16320: Establish order in bytes/string dependencies
http://bugs.python.org/issue16320

#16317: Mention lzma module in tutorial
http://bugs.python.org/issue16317

#16316: Support xz compression in mimetypes module
http://bugs.python.org/issue16316

#16311: Use _PyUnicodeWriter API in text decoders
http://bugs.python.org/issue16311

#16309: "PYTHONPATH=" different from no PYTHONPATH at all
http://bugs.python.org/issue16309

#16307: multiprocess.pool.map_async callables not working
http://bugs.python.org/issue16307

#16306: Multiple error line for unknown command line parameter
http://bugs.python.org/issue16306

#16296: Patch to fix building on Win32/64 under VS 2010
http://bugs.python.org/issue16296

#16292: Cross compilation fixes (general)
http://bugs.python.org/issue16292

#16291: Enable darwin-host cross compilation. Tested with build=linux,
http://bugs.python.org/issue16291



Top 10 most discussed issues (10)
=================================

#15494: Move test/support.py into a test.support subpackage
http://bugs.python.org/issue15494  17 msgs

#16327: subprocess.Popen leaks file descriptors on os.fork() failure
http://bugs.python.org/issue16327  15 msgs

#10527: multiprocessing.Pipe problem: "handle out of range in select()
http://bugs.python.org/issue10527  13 msgs

#16288: TextTestResult uses TestCase.__str__() which isn't customisabl
http://bugs.python.org/issue16288  12 msgs

#16308: Undocumented (?) behaviour change in argparse from 3.2.3 to 3.
http://bugs.python.org/issue16308  10 msgs

#16307: multiprocess.pool.map_async callables not working
http://bugs.python.org/issue16307   8 msgs

#14621: Hash function is not randomized properly
http://bugs.python.org/issue14621   7 msgs

#16245: Update html.entities.html5 dictionary and parseentities.py
http://bugs.python.org/issue16245   7 msgs

#16301: localhost() and thishost() in urllib/request.py
http://bugs.python.org/issue16301   7 msgs

#16322: time.tzname on Python 3.3.0 for Windows is decoded by wrong en
http://bugs.python.org/issue16322   7 msgs



Issues closed (33)
==================

#4997: xml.sax.saxutils.XMLGenerator should write to io.RawIOBase.
http://bugs.python.org/issue4997  closed by serhiy.storchaka

#5148: gzip.open breaks with  'U' flag
http://bugs.python.org/issue5148  closed by nadeem.vawda

#8792: Support Apache extensions to XML-RPC in xmlrpclib
http://bugs.python.org/issue8792  closed by bquinlan

#10405: IDLE breakpoint facility undocumented
http://bugs.python.org/issue10405  closed by ned.deily

#11271: concurrent.futures.ProcessPoolExecutor.map() doesn't batch fun
http://bugs.python.org/issue11271  closed by bquinlan

#12034: check_GetFinalPathNameByHandle() suboptimal
http://bugs.python.org/issue12034  closed by pitrou

#12692: test_urllib2net is triggering a ResourceWarning
http://bugs.python.org/issue12692  closed by nadeem.vawda

#12805: Optimizations for bytes.join() et. al
http://bugs.python.org/issue12805  closed by pitrou

#14394: Add speed improvement note to the decimal docs.
http://bugs.python.org/issue14394  closed by skrah

#14398: bz2.BZ2DEcompressor.decompress fail on large files
http://bugs.python.org/issue14398  closed by nadeem.vawda

#15040: stdlib compatibility with pypy: mailbox module
http://bugs.python.org/issue15040  closed by petri.lehtinen

#15772: Unresolved symbols in Windows 64-bit python
http://bugs.python.org/issue15772  closed by loewis

#15853: IDLE crashes selecting Preferences menu with OS X ActiveState 
http://bugs.python.org/issue15853  closed by ned.deily

#16013: small csv reader bug
http://bugs.python.org/issue16013  closed by serhiy.storchaka

#16116: Can not install C extension modules to inside a venv on Python
http://bugs.python.org/issue16116  closed by python-dev

#16210: combine the two type() definitions in built-in function docs
http://bugs.python.org/issue16210  closed by ezio.melotti

#16220: wsgiref does not call close() on iterable response
http://bugs.python.org/issue16220  closed by pitrou

#16241: "-X faulthandler" is not documented in -X option docs
http://bugs.python.org/issue16241  closed by asvetlov

#16262: srcdir != builddir builds fail, if hg is not installed
http://bugs.python.org/issue16262  closed by doko

#16264: test_logging failure on Windows 7 buildbot
http://bugs.python.org/issue16264  closed by pitrou

#16289: Update devguide for 3.4
http://bugs.python.org/issue16289  closed by ned.deily

#16294: 8 space indent in tutorial
http://bugs.python.org/issue16294  closed by python-dev

#16295: With VS2010 the select extension links against wsock32 instead
http://bugs.python.org/issue16295  closed by sbt

#16297: make csv.DictReader.__init__ use self.fieldnames
http://bugs.python.org/issue16297  closed by r.david.murray

#16299: __build__ as a temp build directory for setup.py
http://bugs.python.org/issue16299  closed by r.david.murray

#16302: Fix ResourceWarning in Lib/test/test_urllib.py
http://bugs.python.org/issue16302  closed by orsenthil

#16303: typo in error message in pstats.py
http://bugs.python.org/issue16303  closed by ezio.melotti

#16305: possible segfault in math.factorial
http://bugs.python.org/issue16305  closed by mark.dickinson

#16313: Support xz compression in shutil module
http://bugs.python.org/issue16313  closed by serhiy.storchaka

#16314: Support xz compression in distutils
http://bugs.python.org/issue16314  closed by serhiy.storchaka

#16315: Support xz compression in Mac build script
http://bugs.python.org/issue16315  closed by ned.deily

#16319: optional comma inside function argument list triggers syntax e
http://bugs.python.org/issue16319  closed by chris.jerdonek

#1248658: dir should accept dirproxies for __dict__
http://bugs.python.org/issue1248658  closed by ronaldoussoren

From brett at python.org  Fri Oct 26 21:14:08 2012
From: brett at python.org (Brett Cannon)
Date: Fri, 26 Oct 2012 15:14:08 -0400
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
Message-ID: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>

I re-ran the unladen benchmarks on my work machine and w/o the -b option
flipped on (i.e. more thorough benchmark numbers). I figured I would share
them now instead of after my PyCon Argentina talk in case people decide to
dig into the results now, find a pathological problem in CPython, and then
fix it before I give my presentation (if you have trouble running a
benchmark or it isn't available in the repo because it's one I hacked
together, just ask and I can help you run the benchmark if you want to try
to speed things up). I have colour-coded benchmarks based on whether it is
faster or slower in Python 3.3 (sorry for those of you who hate HTML email).

But the tl;dr message is that Python 3.3 looks good compared to Python 2.7
(the median benchmark score is 5% slower).

Worst benchmark is nosite_startup, best is telco. The benchmarks people
might want to analyze (i.e. more than 20% slower in Python 3.3) are
mako_v2, threaded_count, normal_startup, iterative_count, pathlib,
formatted_logging, and simple_logging.

###########################################

Report on Linux 3.2.5-gg987 #1 SMP Fri Sep 14 02:36:36 PDT 2012 x86_64
x86_64
Total CPU cores: 12

### 2to3 ###
9.320000 -> 8.980000: 1.04x faster

### call_method ###
Min: 0.417756 -> 0.355247: 1.18x faster
Avg: 0.419688 -> 0.356382: 1.18x faster
Significant (t=92.85)
Stddev: 0.00604 -> 0.00577: 1.0479x smaller
Timeline: b'http://tinyurl.com/8soruze'

### call_method_slots ###
Min: 0.417611 -> 0.358451: 1.17x faster
Avg: 0.420761 -> 0.359676: 1.17x faster
Significant (t=88.70)
Stddev: 0.00605 -> 0.00588: 1.0291x smaller
Timeline: b'http://tinyurl.com/8uu8234'

### call_method_unknown ###
Min: 0.459057 -> 0.359327: 1.28x faster
Avg: 0.462929 -> 0.360410: 1.28x faster
Significant (t=137.99)
Stddev: 0.00698 -> 0.00583: 1.1969x smaller
Timeline: b'http://tinyurl.com/9mo7h24'

### call_simple ###
Min: 0.341689 -> 0.265289: 1.29x faster
Avg: 0.343003 -> 0.266503: 1.29x faster
Significant (t=124.20)
Stddev: 0.00555 -> 0.00511: 1.0859x smaller
Timeline: b'http://tinyurl.com/9pnn7q4'

### chameleon ###
Min: 0.072232 -> 0.062713: 1.15x faster
Avg: 0.074588 -> 0.064261: 1.16x faster
Significant (t=33.74)
Stddev: 0.00284 -> 0.00245: 1.1599x smaller
Timeline: b'http://tinyurl.com/8my8afl'

### chaos ###
Min: 0.313727 -> 0.367015: 1.17x slower
Avg: 0.317568 -> 0.371473: 1.17x slower
Significant (t=-26.72)
Stddev: 0.00962 -> 0.01053: 1.0942x larger
Timeline: b'http://tinyurl.com/9y2u6kh'

### django ###
Min: 0.798331 -> 0.855461: 1.07x slower
Avg: 0.801109 -> 0.860996: 1.07x slower
Significant (t=-87.43)
Stddev: 0.00336 -> 0.00348: 1.0356x larger
Timeline: b'http://tinyurl.com/9sf95pq'

### fannkuch ###
Min: 1.364705 -> 1.327680: 1.03x faster
Avg: 1.380412 -> 1.337467: 1.03x faster
Significant (t=10.48)
Stddev: 0.02056 -> 0.02040: 1.0077x smaller
Timeline: b'http://tinyurl.com/9r2vq6g'

### fastpickle ###
Min: 0.763479 -> 0.805715: 1.06x slower
Avg: 0.770036 -> 0.810855: 1.05x slower
Significant (t=-12.73)
Stddev: 0.01618 -> 0.01589: 1.0180x smaller
Timeline: b'http://tinyurl.com/9rvqo4s'

### fastunpickle ###
Min: 0.588694 -> 0.663616: 1.13x slower
Avg: 0.596622 -> 0.672418: 1.13x slower
Significant (t=-23.22)
Stddev: 0.01503 -> 0.01752: 1.1656x larger
Timeline: b'http://tinyurl.com/9eggn34'

### float ###
Min: 0.363234 -> 0.344408: 1.05x faster
Avg: 0.376159 -> 0.354165: 1.06x faster
Significant (t=8.76)
Stddev: 0.01282 -> 0.01227: 1.0455x smaller
Timeline: b'http://tinyurl.com/8d6rcb8'

### formatted_logging ###
Min: 0.330988 -> 0.400309: 1.21x slower
Avg: 0.335522 -> 0.408920: 1.22x slower
Significant (t=-33.48)
Stddev: 0.00989 -> 0.01194: 1.2076x larger
Timeline: b'http://tinyurl.com/9ll7dqk'

### genshi ###
Min: 0.229140 -> 0.251766: 1.10x slower
Avg: 0.232124 -> 0.257252: 1.11x slower
Significant (t=-40.24)
Stddev: 0.00516 -> 0.00564: 1.0925x larger
Timeline: b'http://tinyurl.com/9dpuuaw'

### go ###
Min: 0.632778 -> 0.710382: 1.12x slower
Avg: 0.636143 -> 0.716748: 1.13x slower
Significant (t=-37.61)
Stddev: 0.00186 -> 0.01504: 8.0815x larger
Timeline: b'http://tinyurl.com/8s7vw74'

### hexiom2 ###
Min: 150.982155 -> 154.702444: 1.02x slower
Avg: 151.194622 -> 154.780953: 1.02x slower
Significant (t=-15.83)
Stddev: 0.30047 -> 0.11103: 2.7063x smaller
Timeline: b'http://tinyurl.com/8rkkduv'

### iterative_count ###
Min: 0.117036 -> 0.156752: 1.34x slower
Avg: 0.120802 -> 0.172218: 1.43x slower
Significant (t=-34.92)
Stddev: 0.00542 -> 0.00889: 1.6422x larger
Timeline: b'http://tinyurl.com/9x9rtnk'

### json_dump_v2 ###
Min: 3.449868 -> 3.522645: 1.02x slower
Avg: 3.467124 -> 3.541902: 1.02x slower
Significant (t=-13.20)
Stddev: 0.02701 -> 0.02960: 1.0959x larger
Timeline: b'http://tinyurl.com/8bsz64a'

### json_load ###
Min: 0.981740 -> 0.567611: 1.73x faster
Avg: 0.986729 -> 0.572975: 1.72x faster
Significant (t=128.95)
Stddev: 0.01796 -> 0.01386: 1.2955x smaller
Timeline: b'http://tinyurl.com/93txokx'

### mako_v2 ###
Min: 0.083660 -> 0.243323: 2.91x slower
Avg: 0.084634 -> 0.247875: 2.93x slower
Significant (t=-821.55)
Stddev: 0.00193 -> 0.00400: 2.0737x larger
Timeline: b'http://tinyurl.com/98n9fab'

### meteor_contest ###
Min: 0.257992 -> 0.232116: 1.11x faster
Avg: 0.262581 -> 0.236684: 1.11x faster
Significant (t=14.31)
Stddev: 0.00916 -> 0.00894: 1.0243x smaller
Timeline: b'http://tinyurl.com/8tpjt43'

### nbody ###
Min: 0.375414 -> 0.293685: 1.28x faster
Avg: 0.379489 -> 0.299794: 1.27x faster
Significant (t=42.71)
Stddev: 0.00997 -> 0.00864: 1.1537x smaller
Timeline: b'http://tinyurl.com/96aqtod'

### normal_startup ###
Min: 0.360002 -> 0.593214: 1.65x slower
Avg: 0.386755 -> 0.600625: 1.55x slower
Significant (t=-134.28)
Stddev: 0.01055 -> 0.00395: 2.6704x smaller
Timeline: b'http://tinyurl.com/9td8pna'

### nqueens ###
Min: 0.300390 -> 0.363904: 1.21x slower
Avg: 0.304282 -> 0.368003: 1.21x slower
Significant (t=-37.41)
Stddev: 0.00813 -> 0.00888: 1.0920x larger
Timeline: b'http://tinyurl.com/9zxyfcu'

### pathlib ###
Min: 0.106088 -> 0.138693: 1.31x slower
Avg: 0.107279 -> 0.139885: 1.30x slower
Significant (t=-133.12)
Stddev: 0.00256 -> 0.00290: 1.1324x larger
Timeline: b'http://tinyurl.com/9llvj6a'

### pidigits ###
Min: 0.351666 -> 0.341745: 1.03x faster
Avg: 0.354743 -> 0.344146: 1.03x faster
Significant (t=5.89)
Stddev: 0.00965 -> 0.00829: 1.1643x smaller
Timeline: b'http://tinyurl.com/8bkgrv4'

### raytrace ###
Min: 1.547054 -> 1.641147: 1.06x slower
Avg: 1.552614 -> 1.643716: 1.06x slower
Significant (t=-286.42)
Stddev: 0.00190 -> 0.00120: 1.5920x smaller
Timeline: b'http://tinyurl.com/9bmnbsd'

### regex_compile ###
Min: 0.494022 -> 0.537924: 1.09x slower
Avg: 0.497904 -> 0.541971: 1.09x slower
Significant (t=-18.23)
Stddev: 0.01177 -> 0.01239: 1.0523x larger
Timeline: b'http://tinyurl.com/cvdhrrm'

### regex_effbot ###
Min: 0.065431 -> 0.073393: 1.12x slower
Avg: 0.069753 -> 0.077338: 1.11x slower
Significant (t=-10.61)
Stddev: 0.00361 -> 0.00354: 1.0179x smaller
Timeline: b'http://tinyurl.com/cupb89m'

### regex_v8 ###
Min: 0.071053 -> 0.081441: 1.15x slower
Avg: 0.075075 -> 0.086167: 1.15x slower
Significant (t=-12.44)
Stddev: 0.00359 -> 0.00518: 1.4455x larger
Timeline: b'http://tinyurl.com/d9ly6x3'

### simple_logging ###
Min: 0.325386 -> 0.395093: 1.21x slower
Avg: 0.330235 -> 0.399825: 1.21x slower
Significant (t=-34.22)
Stddev: 0.00952 -> 0.01077: 1.1317x larger
Timeline: b'http://tinyurl.com/8sbqv85'

### startup_nosite ###
Min: 0.082137 -> 0.453112: 5.52x slower
Avg: 0.129994 -> 0.459361: 3.53x slower
Significant (t=-276.85)
Stddev: 0.01114 -> 0.00419: 2.6585x smaller
Timeline: b'http://tinyurl.com/932mzal'

### telco ###
Min: 0.810000 -> 0.010000: 81.00x faster
Avg: 0.823600 -> 0.015200: 54.18x faster
Significant (t=284.37)
Stddev: 0.01946 -> 0.00505: 3.8556x smaller
Timeline:
http://chart.apis.google.com/chart?cht=lc&chs=700x400&chxt=x,y,x,y&chxr=1,0,1.9&chco=FF0000,0000FF&chdl=../cpython-2.7-wide/bin/python2.7|../cpython-3.3/bin/python3.3&chds=0,1.9&chd=t:0.9,0.84,0.81,0.82,0.82,0.82,0.82,0.82,0.81,0.82,0.82,0.82,0.82,0.82,0.81,0.82,0.82,0.82,0.82,0.82,0.81,0.82,0.82,0.82,0.82,0.81,0.82,0.82,0.82,0.82,0.82,0.81,0.82,0.82,0.82,0.82,0.82,0.81,0.81,0.82,0.82,0.82,0.82,0.81,0.82,0.89,0.89,0.86,0.81,0.82|0.02,0.01,0.02,0.01,0.02,0.01,0.02,0.01,0.02,0.02,0.01,0.02,0.02,0.01,0.02,0.01,0.02,0.01,0.02,0.01,0.02,0.02,0.01,0.02,0.01,0.02,0.01,0.02,0.01,0.02,0.01,0.01,0.02,0.01,0.02,0.01,0.02,0.01,0.02,0.01,0.02,0.01,0.02,0.01,0.02,0.01,0.02,0.01,0.02,0.01&chxl=0:|1|10|20|30|40|50|2:||Iteration|3:||Time+(secs)&chtt=telco

### threaded_count ###
Min: 0.140653 -> 0.173500: 1.23x slower
Avg: 0.152514 -> 0.270779: 1.78x slower
Significant (t=-49.87)
Stddev: 0.00605 -> 0.01564: 2.5837x larger
Timeline: b'http://tinyurl.com/9w4u7el'

### unpack_sequence ###
Min: 0.000077 -> 0.000067: 1.15x faster
Avg: 0.000081 -> 0.000069: 1.18x faster
Significant (t=1163.57)
Stddev: 0.00000 -> 0.00000: 1.7412x larger
Timeline: b'http://tinyurl.com/8qdcjcr'

The following not significant results are hidden, use -v to show them:
html5lib, richards, silent_logging, spectral_norm.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121026/8cc1810d/attachment.html>

From robertsteckroth at gmail.com  Fri Oct 26 23:01:09 2012
From: robertsteckroth at gmail.com (Robert Steckroth)
Date: Fri, 26 Oct 2012 17:01:09 -0400
Subject: [Python-Dev] Anyone have a Universal setter example?
Message-ID: <CAL-VaPdEOoheOLi3Rpvvcifu+ZiW-CmDnQ0hypJEo=sO-iXzuw@mail.gmail.com>

Hello Gang, I am fresh to this list so, hello PYC gurus!
------------------------------------------------------------------------------
I have been getting into
the Python C API over the past few days and found it it to be a
wonderful framework with
a solid design. There are many questions I would like to pose and
answer in this community and
look forward to a productive relationship. Here are a few question I have:

1. Is it appropriate to post usage questions regarding the Python C
API to this list like the questions below?

2. Is there any examples of a universal setter function for
`PyGetSetDef`? The docs mention using
the closure pointer from the last element in the `PyGetSetDef`
structures. I am finding it hard
to get a grasp on how this is implemented. I understand why we use it
and what it does and
need a more modular setter function for a large object.

*From the docs* [http://docs.python.org/extending/newtypes.html]
> `The getter function is passed a Noddy object and a ?closure?, which is void pointer. In this case, the closure is ignored. (The closure supports an advanced usage in which definition data is passed to the getter and setter. This could, for example, be used to allow a single set of getter and setter functions that decide the attribute to get or set based on data in the closure.)`


3. Is it customary to use header files when building larger Python
modules/objects? I was successful in prototyping the source files,
but unable to get the module working after running a distutils
install. In short, is it encouraged to use header files?





-- 
Bust0ut, Surgemcgee: Systems Engineer ---
surgemcgee.com
Django_Teamplate3d

From phd at phdru.name  Fri Oct 26 23:09:32 2012
From: phd at phdru.name (Oleg Broytman)
Date: Sat, 27 Oct 2012 01:09:32 +0400
Subject: [Python-Dev] Anyone have a Universal setter example?
In-Reply-To: <CAL-VaPdEOoheOLi3Rpvvcifu+ZiW-CmDnQ0hypJEo=sO-iXzuw@mail.gmail.com>
References: <CAL-VaPdEOoheOLi3Rpvvcifu+ZiW-CmDnQ0hypJEo=sO-iXzuw@mail.gmail.com>
Message-ID: <20121026210932.GA21361@iskra.aviel.ru>

Hello.

   Let me remind you this mailing list is to work on developing Python
(adding new features to Python itself and fixing bugs); if you're having
problems using Python, please find another forum. Probably
python-list/comp.lang.python mailing list/news group is the best place;
there are Python developers who participate in it; you may get a faster,
and probably more complete, answer there. See
http://www.python.org/community/ for other lists/news groups/fora. Thank
you for understanding.

On Fri, Oct 26, 2012 at 05:01:09PM -0400, Robert Steckroth <robertsteckroth at gmail.com> wrote:
> 1. Is it appropriate to post usage questions regarding the Python C
> API to this list like the questions below?

  I think not until you ask a question in python-list/comp.lang.python
and don't get any reply.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From tseaver at palladion.com  Fri Oct 26 23:40:58 2012
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 26 Oct 2012 17:40:58 -0400
Subject: [Python-Dev] Anyone have a Universal setter example?
In-Reply-To: <CAL-VaPdEOoheOLi3Rpvvcifu+ZiW-CmDnQ0hypJEo=sO-iXzuw@mail.gmail.com>
References: <CAL-VaPdEOoheOLi3Rpvvcifu+ZiW-CmDnQ0hypJEo=sO-iXzuw@mail.gmail.com>
Message-ID: <508B036A.5000008@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/26/2012 05:01 PM, Robert Steckroth wrote:

> 1. Is it appropriate to post usage questions regarding the Python C 
> API to this list like the questions below?

The C-API SIG would likely be a better forum:

  cap-sig at python.org
  gmane:  gmane.comp.python.capi


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlCLA2oACgkQ+gerLs4ltQ6JnACg004jmBpUDRX1TBHaFNkg7JW0
7twAoI/s228R0TnaBpqiImJsjGg1hJKp
=IAPl
-----END PGP SIGNATURE-----


From storchaka at gmail.com  Sat Oct 27 11:30:53 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sat, 27 Oct 2012 12:30:53 +0300
Subject: [Python-Dev] cpython (3.2): bounds check for bad data (thanks
	amaury)
In-Reply-To: <3XpMlL1m20zPXZ@mail.python.org>
References: <3XpMlL1m20zPXZ@mail.python.org>
Message-ID: <508BA9CD.7030704@gmail.com>

On 27.10.12 03:05, philip.jenvey wrote:
> http://hg.python.org/cpython/rev/74d65c746f63
> changeset:   79953:74d65c746f63
> branch:      3.2
> parent:      79941:eb999002916c
> user:        Philip Jenvey <pjenvey at underboss.org>
> date:        Fri Oct 26 17:01:53 2012 -0700
> summary:
>    bounds check for bad data (thanks amaury)

> +        if (strlen(p) > 2 &&


First, it produces compiler warning:

Python/codecs.c: In function ?PyCodec_SurrogatePassErrors?:
Python/codecs.c:794: warning: pointer targets in passing argument 1 of ?strlen? differ in signedness
/usr/include/string.h:397: note: expected ?const char *? but argument is of type ?unsigned char *?

Second, it slowdown the code to 10%:

$ ./python-orig -m timeit -s 'b=b"\xed\xa0\xa0"+b"x"*10000'  'b.decode("utf-8", "surrogatepass")'
100000 loops, best of 3: 12.2 usec per loop
$ ./python -m timeit -s 'b=b"\xed\xa0\xa0"+b"x"*10000'  'b.decode("utf-8", "surrogatepass")'
100000 loops, best of 3: 13.3 usec per loop


I suggest to use the followed code instead:

         if (PyBytes_GET_SIZE(object) - start >= 3 &&



From arigo at tunes.org  Sat Oct 27 11:35:16 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 27 Oct 2012 11:35:16 +0200
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
Message-ID: <CAMSv6X2RSVYwjmy72Uh6Sc4_drqMGPg8KwA+hWVf+5kKdP+8dQ@mail.gmail.com>

Hi Brett,

On Fri, Oct 26, 2012 at 9:14 PM, Brett Cannon <brett at python.org> wrote:
> Worst benchmark is nosite_startup, best is telco.

May I express doubts about telco? :-)  It looks like the Python 3
version is simply not running:

> ### telco ###
> Min: 0.810000 -> 0.010000: 81.00x faster
> Avg: 0.823600 -> 0.015200: 54.18x faster


A bient?t,

Armin.

From storchaka at gmail.com  Sat Oct 27 11:43:51 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sat, 27 Oct 2012 12:43:51 +0300
Subject: [Python-Dev] cpython (3.2): bounds check for bad data (thanks
	amaury)
In-Reply-To: <508BA9CD.7030704@gmail.com>
References: <3XpMlL1m20zPXZ@mail.python.org> <508BA9CD.7030704@gmail.com>
Message-ID: <k6gacp$aqt$1@ger.gmane.org>

Oh, and there is yet one bug. "&&" should be used in the condition 
instead of "||".  Patch attached.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: codecs_surrogatepass.patch
Type: text/x-patch
Size: 787 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121027/c9dfaa27/attachment.bin>

From solipsis at pitrou.net  Sat Oct 27 11:53:52 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 27 Oct 2012 11:53:52 +0200
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
Message-ID: <20121027115352.35011ff9@pitrou.net>

On Fri, 26 Oct 2012 15:14:08 -0400
Brett Cannon <brett at python.org> wrote:
> 
> Worst benchmark is nosite_startup, best is telco. The benchmarks people
> might want to analyze (i.e. more than 20% slower in Python 3.3) are
> mako_v2, threaded_count, normal_startup, iterative_count, pathlib,
> formatted_logging, and simple_logging.

Well, did you check that mako_v2 wasn't subject to the Markupsafe
issue?

threaded_count and iterative_count are completely dumb.
Slower startup is due to the fact that Python 3 needs many more
modules to even start itself.

Regards

Antoine.



From storchaka at gmail.com  Sat Oct 27 12:09:16 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sat, 27 Oct 2012 13:09:16 +0300
Subject: [Python-Dev] cpython (3.2): bounds check for bad data (thanks
	amaury)
In-Reply-To: <k6gacp$aqt$1@ger.gmane.org>
References: <3XpMlL1m20zPXZ@mail.python.org> <508BA9CD.7030704@gmail.com>
	<k6gacp$aqt$1@ger.gmane.org>
Message-ID: <k6gbse$let$1@ger.gmane.org>

It's too large for Python-Dev quick remark. I open an issue in a 
bugtracker: http://bugs.python.org/issue16336.


From fijall at gmail.com  Sat Oct 27 12:12:28 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 27 Oct 2012 12:12:28 +0200
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CAMSv6X2RSVYwjmy72Uh6Sc4_drqMGPg8KwA+hWVf+5kKdP+8dQ@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<CAMSv6X2RSVYwjmy72Uh6Sc4_drqMGPg8KwA+hWVf+5kKdP+8dQ@mail.gmail.com>
Message-ID: <CAK5idxTMy-nS01TV84ftSa94JY7g=0TJM0=pFR+Jf0eh41-eGg@mail.gmail.com>

On Sat, Oct 27, 2012 at 11:35 AM, Armin Rigo <arigo at tunes.org> wrote:
> Hi Brett,
>
> On Fri, Oct 26, 2012 at 9:14 PM, Brett Cannon <brett at python.org> wrote:
>> Worst benchmark is nosite_startup, best is telco.
>
> May I express doubts about telco? :-)  It looks like the Python 3
> version is simply not running:
>
>> ### telco ###
>> Min: 0.810000 -> 0.010000: 81.00x faster
>> Avg: 0.823600 -> 0.015200: 54.18x faster
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com

I think the original explanation was cDecimal vs decimal.

From stefan at bytereef.org  Sat Oct 27 14:33:41 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Sat, 27 Oct 2012 14:33:41 +0200
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CAK5idxTMy-nS01TV84ftSa94JY7g=0TJM0=pFR+Jf0eh41-eGg@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<CAMSv6X2RSVYwjmy72Uh6Sc4_drqMGPg8KwA+hWVf+5kKdP+8dQ@mail.gmail.com>
	<CAK5idxTMy-nS01TV84ftSa94JY7g=0TJM0=pFR+Jf0eh41-eGg@mail.gmail.com>
Message-ID: <20121027123341.GA9537@sleipnir.bytereef.org>

Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Sat, Oct 27, 2012 at 11:35 AM, Armin Rigo <arigo at tunes.org> wrote:
> > May I express doubts about telco? :-)  It looks like the Python 3
> > version is simply not running:
> >
> >> ### telco ###
> >> Min: 0.810000 -> 0.010000: 81.00x faster
> >> Avg: 0.823600 -> 0.015200: 54.18x faster
> 
> I think the original explanation was cDecimal vs decimal.

Yes, the magnitude of the speedup looks correct. In an isolated benchmark
with the large input file [1] I'm getting 30x speedup for telco.


Stefan Krah

[1] http://www.bytereef.org/mpdecimal/quickstart.html#telco-benchmark - expon180-1e6b.zip



From bcannon at gmail.com  Sat Oct 27 15:20:36 2012
From: bcannon at gmail.com (Brett Cannon)
Date: Sat, 27 Oct 2012 09:20:36 -0400
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <20121027115352.35011ff9@pitrou.net>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
Message-ID: <CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>

I did check that markup safe as not installed. It might just be mako doing
something silly.

The threads tests are very synthetic.

And yes, there are more modules at startup. When was the last to,e we
looked at them to make sure we weren't doing needless I ports?
On Oct 27, 2012 5:56 AM, "Antoine Pitrou" <solipsis at pitrou.net> wrote:

> On Fri, 26 Oct 2012 15:14:08 -0400
> Brett Cannon <brett at python.org> wrote:
> >
> > Worst benchmark is nosite_startup, best is telco. The benchmarks people
> > might want to analyze (i.e. more than 20% slower in Python 3.3) are
> > mako_v2, threaded_count, normal_startup, iterative_count, pathlib,
> > formatted_logging, and simple_logging.
>
> Well, did you check that mako_v2 wasn't subject to the Markupsafe
> issue?
>
> threaded_count and iterative_count are completely dumb.
> Slower startup is due to the fact that Python 3 needs many more
> modules to even start itself.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121027/287a5ce5/attachment.html>

From ncoghlan at gmail.com  Sat Oct 27 17:22:20 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 28 Oct 2012 01:22:20 +1000
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
Message-ID: <CADiSq7cZL=bba_vi315GGOVfh7O+AxMWA0_A+quGg4ihbwHMbA@mail.gmail.com>

On Sat, Oct 27, 2012 at 11:20 PM, Brett Cannon <bcannon at gmail.com> wrote:
> I did check that markup safe as not installed. It might just be mako doing
> something silly.
>
> The threads tests are very synthetic.
>
> And yes, there are more modules at startup. When was the last to,e we looked
> at them to make sure we weren't doing needless I ports?

It's been quite a while.

>>> py3k - py27
set(['reprlib', 'heapq', '_collections', 'functools', '_bisect',
'copyreg', 'io', 'operator', '_heapq', '_io', '_thread',
'encodings.latin_1', 'collections', '_frozen_importlib',
'collections.abc', 'builtins', '_sysconfigdata', '_functools',
'keyword', '_imp', 'bisect', 'weakref', 'itertools', 'marshal'])

>>> py27 - py3k
set(['exceptions', 'copy_reg', 'warnings', 'UserDict', 'traceback',
'encodings.codecs', '__builtin__', 'linecache', '_abcoll',
'encodings.__builtin__', 'encodings.encodings', 'types'])

To check how many of those dependencies stemmed from collections, I
checked against the 2.7 version:

>>> py3k - py27_with_collections
set(['_functools', 'reprlib', '_thread', '_io', '_imp',
'_frozen_importlib', 'functools', 'weakref', 'collections.abc',
'encodings.latin_1', 'io', 'copyreg', 'builtins', 'marshal',
'_sysconfigdata'])

>>> py27_with_collections - py3k
set(['exceptions', 'copy_reg', 'thread', 'warnings', 'UserDict',
'traceback', 'encodings.codecs', '__builtin__', 'linecache',
'_abcoll', 'encodings.__builtin__', 'encodings.encodings', 'types'])

Implicitly bringing in _thread is a bit of a worry. Apparently 3.2 had
the same problem, though:

>>> py3k - py32
{'_imp', '_frozen_importlib', '_warnings', 'collections.abc',
'marshal', '_sysconfigdata'}

>>> py32 - py3k
{'_locale', 'locale', 'traceback', 'linecache', 'token', '_abcoll', 'tokenize'}


Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From solipsis at pitrou.net  Sat Oct 27 21:21:39 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 27 Oct 2012 21:21:39 +0200
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
Message-ID: <20121027212139.0d9d6fe2@pitrou.net>

On Sat, 27 Oct 2012 09:20:36 -0400
Brett Cannon <bcannon at gmail.com> wrote:
> I did check that markup safe as not installed. It might just be mako doing
> something silly.
> 
> The threads tests are very synthetic.
> 
> And yes, there are more modules at startup. When was the last to,e we
> looked at them to make sure we weren't doing needless I ports?

The last time was between 3.2 and 3.3. It will be hard to lower the
number of imported modules, given the current semantics (io, importlib,
unicode, site.py, sysconfig...). Python 2's view of the world was much
simpler (na?ve?) in comparison.

It would be interesting to know *where* the module import time gets
spent, on a lower level. My gut feeling is that execution of Python
module code is the main contributor.

Regards

Antoine.

From mark at hotpy.org  Sat Oct 27 22:40:26 2012
From: mark at hotpy.org (Mark Shannon)
Date: Sat, 27 Oct 2012 21:40:26 +0100
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <20121027212139.0d9d6fe2@pitrou.net>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net>
Message-ID: <508C46BA.8020906@hotpy.org>

On 27/10/12 20:21, Antoine Pitrou wrote:
> On Sat, 27 Oct 2012 09:20:36 -0400
> Brett Cannon <bcannon at gmail.com> wrote:
>> I did check that markup safe as not installed. It might just be mako doing
>> something silly.
>>
>> The threads tests are very synthetic.
>>
>> And yes, there are more modules at startup. When was the last to,e we
>> looked at them to make sure we weren't doing needless I ports?
>
> The last time was between 3.2 and 3.3. It will be hard to lower the
> number of imported modules, given the current semantics (io, importlib,
> unicode, site.py, sysconfig...). Python 2's view of the world was much
> simpler (na?ve?) in comparison.
>
> It would be interesting to know *where* the module import time gets
> spent, on a lower level. My gut feeling is that execution of Python
> module code is the main contributor.

I suspect that stating and loading the .pyc files is responsible for 
most of the overhead.
PyRun starts up quite a lot faster thanks to embedding all the modules 
in the executable: http://www.egenix.com/products/python/PyRun/

Freezing all the core modules into the executable should reduce start up 
time.

Cheers,
Mark


From timothy.c.delaney at gmail.com  Sat Oct 27 22:53:42 2012
From: timothy.c.delaney at gmail.com (Tim Delaney)
Date: Sun, 28 Oct 2012 07:53:42 +1100
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <508C46BA.8020906@hotpy.org>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
Message-ID: <CAN8CLgkttfmgJkt2E1H5TMkkbWwPza6Mv757AuWN6cppHV3XyQ@mail.gmail.com>

On 28 October 2012 07:40, Mark Shannon <mark at hotpy.org> wrote:

>
> I suspect that stating and loading the .pyc files is responsible for most
> of the overhead.
> PyRun starts up quite a lot faster thanks to embedding all the modules in
> the executable: http://www.egenix.com/**products/python/PyRun/<http://www.egenix.com/products/python/PyRun/>
>
> Freezing all the core modules into the executable should reduce start up
> time.
>

That suggests a test to me that the Cython guys might be interested in (or
may well have performed in the past). How much of the stdlib could be
compiled with Cython and used during the startup process? How much of an
effect would it have on startup times and these benchmarks if
Cython-compiled extensions were used?

I'm thinking here of elimination of .pyc interpretation and execution (stat
calls would be similar, probably slightly higher).

To be clear - I'm *not* suggesting Cython become part of the required build
toolchain. But *if* the Cython-compiled extensions prove to be
significantly faster I'm thinking maybe it could become a semi-supported
option (e.g. a HOWTO with the caveat "it worked on this particular system").

Tim Delaney
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121028/b3605974/attachment.html>

From martin at v.loewis.de  Sat Oct 27 22:58:26 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Sat, 27 Oct 2012 22:58:26 +0200
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CAN8CLgkttfmgJkt2E1H5TMkkbWwPza6Mv757AuWN6cppHV3XyQ@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAN8CLgkttfmgJkt2E1H5TMkkbWwPza6Mv757AuWN6cppHV3XyQ@mail.gmail.com>
Message-ID: <20121027225826.Horde.M0EjTUlCcOxQjEryQ2NWSMA@webmail.df.eu>


Zitat von Tim Delaney <timothy.c.delaney at gmail.com>:

> To be clear - I'm *not* suggesting Cython become part of the required build
> toolchain. But *if* the Cython-compiled extensions prove to be
> significantly faster I'm thinking maybe it could become a semi-supported
> option (e.g. a HOWTO with the caveat "it worked on this particular system").

This should compare to zipping the standard library, which has been a  
supported
configuration for a long time, and also avoids many stat calls.

Regards,
Martin



From solipsis at pitrou.net  Sat Oct 27 22:59:32 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 27 Oct 2012 22:59:32 +0200
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
Message-ID: <20121027225932.0d63bc68@pitrou.net>

On Sat, 27 Oct 2012 21:40:26 +0100
Mark Shannon <mark at hotpy.org> wrote:
> On 27/10/12 20:21, Antoine Pitrou wrote:
> >
> > It would be interesting to know *where* the module import time gets
> > spent, on a lower level. My gut feeling is that execution of Python
> > module code is the main contributor.
> 
> I suspect that stating and loading the .pyc files is responsible for 
> most of the overhead.
> PyRun starts up quite a lot faster thanks to embedding all the modules 
> in the executable: http://www.egenix.com/products/python/PyRun/

Any numbers?

Regards

Antoine.



From p.f.moore at gmail.com  Sat Oct 27 23:07:11 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 27 Oct 2012 22:07:11 +0100
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <20121027225826.Horde.M0EjTUlCcOxQjEryQ2NWSMA@webmail.df.eu>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAN8CLgkttfmgJkt2E1H5TMkkbWwPza6Mv757AuWN6cppHV3XyQ@mail.gmail.com>
	<20121027225826.Horde.M0EjTUlCcOxQjEryQ2NWSMA@webmail.df.eu>
Message-ID: <CACac1F8tF-fREvCH49TPKe2SLEUCVpj_yVpUn6_VBKpDbCy3qQ@mail.gmail.com>

On 27 October 2012 21:58,  <martin at v.loewis.de> wrote:
>
> Zitat von Tim Delaney <timothy.c.delaney at gmail.com>:
>
>
>> To be clear - I'm *not* suggesting Cython become part of the required
>> build
>> toolchain. But *if* the Cython-compiled extensions prove to be
>> significantly faster I'm thinking maybe it could become a semi-supported
>> option (e.g. a HOWTO with the caveat "it worked on this particular
>> system").
>
>
> This should compare to zipping the standard library, which has been a
> supported
> configuration for a long time, and also avoids many stat calls.

Interestingly, I just did a quick test of this: This is on my Windows
7 PC, running under Powershell. D:\Apps\Python33 is a standard
installation, whereas D:\Dev\P33 has a zipped stdlib:

PS 22:02 D:\Data
>foreach ($i in 1..10) { measure-command { D:\Apps\Python33\python.exe -c "raise SystemExit" } | % { $_.TotalSeconds } }
0.0737877
0.1014695
0.0950326
0.0910734
0.0689548
0.084994
0.0772204
0.0958197
0.0696385
0.0806066
PS 22:03 D:\Data
>foreach ($i in 1..10) { measure-command { D:\Dev\P33\python.exe -c "raise SystemExit" } | % { $_.TotalSeconds } }
0.1922151
0.1879894
0.2455766
0.2842425
0.1937161
0.2168928
0.2441508
0.1860206
0.1866409
0.1897004

Looks like the normal configuration is over twice as fast as the zipped one...

Paul.

From mark at hotpy.org  Sat Oct 27 23:11:01 2012
From: mark at hotpy.org (Mark Shannon)
Date: Sat, 27 Oct 2012 22:11:01 +0100
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <20121027225932.0d63bc68@pitrou.net>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<20121027225932.0d63bc68@pitrou.net>
Message-ID: <508C4DE5.8000808@hotpy.org>

On 27/10/12 21:59, Antoine Pitrou wrote:
> On Sat, 27 Oct 2012 21:40:26 +0100
> Mark Shannon <mark at hotpy.org> wrote:
>> On 27/10/12 20:21, Antoine Pitrou wrote:
>>>
>>> It would be interesting to know *where* the module import time gets
>>> spent, on a lower level. My gut feeling is that execution of Python
>>> module code is the main contributor.
>>
>> I suspect that stating and loading the .pyc files is responsible for
>> most of the overhead.
>> PyRun starts up quite a lot faster thanks to embedding all the modules
>> in the executable: http://www.egenix.com/products/python/PyRun/
>
> Any numbers?

No numbers, but I did see this talk:
http://2012.pyconuk.net/Talks/PyRun
The abstract claims that PyRun "has a greatly improved startup time 
compared to regular Python"

Cheers,
Mark


From solipsis at pitrou.net  Sat Oct 27 23:25:34 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 27 Oct 2012 23:25:34 +0200
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<20121027225932.0d63bc68@pitrou.net> <508C4DE5.8000808@hotpy.org>
Message-ID: <20121027232534.754d176d@pitrou.net>

On Sat, 27 Oct 2012 22:11:01 +0100
Mark Shannon <mark at hotpy.org> wrote:
> On 27/10/12 21:59, Antoine Pitrou wrote:
> > On Sat, 27 Oct 2012 21:40:26 +0100
> > Mark Shannon <mark at hotpy.org> wrote:
> >> On 27/10/12 20:21, Antoine Pitrou wrote:
> >>>
> >>> It would be interesting to know *where* the module import time gets
> >>> spent, on a lower level. My gut feeling is that execution of Python
> >>> module code is the main contributor.
> >>
> >> I suspect that stating and loading the .pyc files is responsible for
> >> most of the overhead.
> >> PyRun starts up quite a lot faster thanks to embedding all the modules
> >> in the executable: http://www.egenix.com/products/python/PyRun/
> >
> > Any numbers?
> 
> No numbers, but I did see this talk:
> http://2012.pyconuk.net/Talks/PyRun
> The abstract claims that PyRun "has a greatly improved startup time 
> compared to regular Python"

Sounds great ;-)

cheers

Antoine.



From brett at python.org  Sun Oct 28 00:06:05 2012
From: brett at python.org (Brett Cannon)
Date: Sat, 27 Oct 2012 18:06:05 -0400
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <508C46BA.8020906@hotpy.org>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
Message-ID: <CAP1=2W46ErE8LJ4nJnM_q7HWNVaWh5CQrg642EB_JwCO4sc++A@mail.gmail.com>

On Sat, Oct 27, 2012 at 4:40 PM, Mark Shannon <mark at hotpy.org> wrote:

> On 27/10/12 20:21, Antoine Pitrou wrote:
>
>> On Sat, 27 Oct 2012 09:20:36 -0400
>> Brett Cannon <bcannon at gmail.com> wrote:
>>
>>> I did check that markup safe as not installed. It might just be mako
>>> doing
>>> something silly.
>>>
>>> The threads tests are very synthetic.
>>>
>>> And yes, there are more modules at startup. When was the last to,e we
>>> looked at them to make sure we weren't doing needless I ports?
>>>
>>
>> The last time was between 3.2 and 3.3. It will be hard to lower the
>> number of imported modules, given the current semantics (io, importlib,
>> unicode, site.py, sysconfig...). Python 2's view of the world was much
>> simpler (na?ve?) in comparison.
>>
>> It would be interesting to know *where* the module import time gets
>> spent, on a lower level. My gut feeling is that execution of Python
>> module code is the main contributor.
>>
>
> I suspect that stating and loading the .pyc files is responsible for most
> of the overhead.
>

I really doubt that as the amount of stat calls is significantly reduced in
Python 3.3 compared to Python 3.2 (startup benchmarks show Python 3.3 is
roughly 1.66x faster than 3.2 thanks to caching filenames in a directory).
More modules means more work (e.g. I/O, executing the module, etc.).

The only way to lower stat call overhead is to simply not check if a
directory's contents changed during startup by assuming Python itself will
not write any new module files. Without benchmarking I don't know if it
would make that much of a difference, though.


> PyRun starts up quite a lot faster thanks to embedding all the modules in
> the executable: http://www.egenix.com/**products/python/PyRun/<http://www.egenix.com/products/python/PyRun/>
>
> Freezing all the core modules into the executable should reduce start up
> time.
>

 Sure, but working with a frozen module is a pain so it is not something to
take lightly.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121027/10d9101a/attachment.html>

From brett at python.org  Sun Oct 28 00:07:48 2012
From: brett at python.org (Brett Cannon)
Date: Sat, 27 Oct 2012 18:07:48 -0400
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CACac1F8tF-fREvCH49TPKe2SLEUCVpj_yVpUn6_VBKpDbCy3qQ@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAN8CLgkttfmgJkt2E1H5TMkkbWwPza6Mv757AuWN6cppHV3XyQ@mail.gmail.com>
	<20121027225826.Horde.M0EjTUlCcOxQjEryQ2NWSMA@webmail.df.eu>
	<CACac1F8tF-fREvCH49TPKe2SLEUCVpj_yVpUn6_VBKpDbCy3qQ@mail.gmail.com>
Message-ID: <CAP1=2W59zs59H2Xbs3QzhfGVNkFAMX2JNTh+KPv5KtK5NNnVOA@mail.gmail.com>

On Sat, Oct 27, 2012 at 5:07 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 27 October 2012 21:58,  <martin at v.loewis.de> wrote:
> >
> > Zitat von Tim Delaney <timothy.c.delaney at gmail.com>:
> >
> >
> >> To be clear - I'm *not* suggesting Cython become part of the required
> >> build
> >> toolchain. But *if* the Cython-compiled extensions prove to be
> >> significantly faster I'm thinking maybe it could become a semi-supported
> >> option (e.g. a HOWTO with the caveat "it worked on this particular
> >> system").
> >
> >
> > This should compare to zipping the standard library, which has been a
> > supported
> > configuration for a long time, and also avoids many stat calls.
>
> Interestingly, I just did a quick test of this: This is on my Windows
> 7 PC, running under Powershell. D:\Apps\Python33 is a standard
> installation, whereas D:\Dev\P33 has a zipped stdlib:
>
> PS 22:02 D:\Data
> >foreach ($i in 1..10) { measure-command { D:\Apps\Python33\python.exe -c
> "raise SystemExit" } | % { $_.TotalSeconds } }
> 0.0737877
> 0.1014695
> 0.0950326
> 0.0910734
> 0.0689548
> 0.084994
> 0.0772204
> 0.0958197
> 0.0696385
> 0.0806066
> PS 22:03 D:\Data
> >foreach ($i in 1..10) { measure-command { D:\Dev\P33\python.exe -c "raise
> SystemExit" } | % { $_.TotalSeconds } }
> 0.1922151
> 0.1879894
> 0.2455766
> 0.2842425
> 0.1937161
> 0.2168928
> 0.2441508
> 0.1860206
> 0.1866409
> 0.1897004
>
> Looks like the normal configuration is over twice as fast as the zipped
> one...
>

Are both debug builds (asking because of the path names)? CPython is now
significantly slower in a debug build thanks to the overhead it adds to any
Python code executing, which means importlib runs much slower.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121027/8e05c400/attachment.html>

From storchaka at gmail.com  Sun Oct 28 00:16:20 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 28 Oct 2012 01:16:20 +0300
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CACac1F8tF-fREvCH49TPKe2SLEUCVpj_yVpUn6_VBKpDbCy3qQ@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAN8CLgkttfmgJkt2E1H5TMkkbWwPza6Mv757AuWN6cppHV3XyQ@mail.gmail.com>
	<20121027225826.Horde.M0EjTUlCcOxQjEryQ2NWSMA@webmail.df.eu>
	<CACac1F8tF-fREvCH49TPKe2SLEUCVpj_yVpUn6_VBKpDbCy3qQ@mail.gmail.com>
Message-ID: <k6hmfm$n8d$1@ger.gmane.org>

On 28.10.12 00:07, Paul Moore wrote:
> Looks like the normal configuration is over twice as fast as the zipped one...

The normal configuration does 269 stats, but the zipped one does 12636 
seeks.


From storchaka at gmail.com  Sun Oct 28 00:39:42 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 28 Oct 2012 01:39:42 +0300
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CAP1=2W46ErE8LJ4nJnM_q7HWNVaWh5CQrg642EB_JwCO4sc++A@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAP1=2W46ErE8LJ4nJnM_q7HWNVaWh5CQrg642EB_JwCO4sc++A@mail.gmail.com>
Message-ID: <k6hnri$nn$1@ger.gmane.org>

On 28.10.12 01:06, Brett Cannon wrote:
> I really doubt that as the amount of stat calls is significantly reduced
> in Python 3.3 compared to Python 3.2 (startup benchmarks show Python 3.3
> is roughly 1.66x faster than 3.2 thanks to caching filenames in a
> directory).

$ strace ./python -c '' 2>&1 | grep -c stat

Python 2.7 - 161 stats
Python 3.2 - 555 stats
Python 3.3 - 243 stats



From solipsis at pitrou.net  Sun Oct 28 01:00:48 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 28 Oct 2012 01:00:48 +0200
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAP1=2W46ErE8LJ4nJnM_q7HWNVaWh5CQrg642EB_JwCO4sc++A@mail.gmail.com>
	<k6hnri$nn$1@ger.gmane.org>
Message-ID: <20121028010048.43547034@pitrou.net>

On Sun, 28 Oct 2012 01:39:42 +0300
Serhiy Storchaka <storchaka at gmail.com> wrote:

> On 28.10.12 01:06, Brett Cannon wrote:
> > I really doubt that as the amount of stat calls is significantly reduced
> > in Python 3.3 compared to Python 3.2 (startup benchmarks show Python 3.3
> > is roughly 1.66x faster than 3.2 thanks to caching filenames in a
> > directory).
> 
> $ strace ./python -c '' 2>&1 | grep -c stat
> 
> Python 2.7 - 161 stats
> Python 3.2 - 555 stats
> Python 3.3 - 243 stats

This will probably depend on the length of sys.path:

$ strace -e stat python2.7 -Sc "" 2>&1 | wc -l
35
$ strace -e stat python3.2 -Sc "" 2>&1 | wc -l
298
$ strace -e stat python3.3 -Sc "" 2>&1 | wc -l
106

$ strace -e stat python2.7 -c "" 2>&1 | wc -l
200
$ strace -e stat python3.2 -c "" 2>&1 | wc -l
726
$ strace -e stat python3.3 -c "" 2>&1 | wc -l
180

Regards

Antoine.



From greg at krypto.org  Sun Oct 28 04:38:58 2012
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 27 Oct 2012 20:38:58 -0700
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <20121028010048.43547034@pitrou.net>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAP1=2W46ErE8LJ4nJnM_q7HWNVaWh5CQrg642EB_JwCO4sc++A@mail.gmail.com>
	<k6hnri$nn$1@ger.gmane.org> <20121028010048.43547034@pitrou.net>
Message-ID: <CAGE7PNKFW4Y2JBnM3C-xBVmwdusfgDCrPO-1vHau4Cd63VfMyA@mail.gmail.com>

One word: profile.

Looking at stat counts alone rather than measuring the total time spent in
all types of system calls from strace and profiling is not really useful. ;)

Another thing to keep an eye out for within a startup profile:  how often
does the gc collect?  our default gc collection thresholds haven't been
tuned in ages afaik [or am i forgetting something] and I know of
pathological cases at work where simply doing a gc.disable() before
importing a bunch of modules (tons of generated protocol buffer code) and
re-enabling it afterwards speeds up this application's startup way more
significantly than seems healthy in 2.x... that could be related to the
particulars of the protobuf module code though.

-gps

On Sat, Oct 27, 2012 at 4:00 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sun, 28 Oct 2012 01:39:42 +0300
> Serhiy Storchaka <storchaka at gmail.com> wrote:
>
> > On 28.10.12 01:06, Brett Cannon wrote:
> > > I really doubt that as the amount of stat calls is significantly
> reduced
> > > in Python 3.3 compared to Python 3.2 (startup benchmarks show Python
> 3.3
> > > is roughly 1.66x faster than 3.2 thanks to caching filenames in a
> > > directory).
> >
> > $ strace ./python -c '' 2>&1 | grep -c stat
> >
> > Python 2.7 - 161 stats
> > Python 3.2 - 555 stats
> > Python 3.3 - 243 stats
>
> This will probably depend on the length of sys.path:
>
> $ strace -e stat python2.7 -Sc "" 2>&1 | wc -l
> 35
> $ strace -e stat python3.2 -Sc "" 2>&1 | wc -l
> 298
> $ strace -e stat python3.3 -Sc "" 2>&1 | wc -l
> 106
>
> $ strace -e stat python2.7 -c "" 2>&1 | wc -l
> 200
> $ strace -e stat python3.2 -c "" 2>&1 | wc -l
> 726
> $ strace -e stat python3.3 -c "" 2>&1 | wc -l
> 180
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121027/17035ea3/attachment.html>

From stefan_ml at behnel.de  Sun Oct 28 08:22:07 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 28 Oct 2012 08:22:07 +0100
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CAN8CLgkttfmgJkt2E1H5TMkkbWwPza6Mv757AuWN6cppHV3XyQ@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAN8CLgkttfmgJkt2E1H5TMkkbWwPza6Mv757AuWN6cppHV3XyQ@mail.gmail.com>
Message-ID: <k6imeu$rk1$1@ger.gmane.org>

Tim Delaney, 27.10.2012 22:53:
> On 28 October 2012 07:40, Mark Shannon wrote:
>> I suspect that stating and loading the .pyc files is responsible for most
>> of the overhead.
>> PyRun starts up quite a lot faster thanks to embedding all the modules in
>> the executable: http://www.egenix.com/**products/python/PyRun/<http://www.egenix.com/products/python/PyRun/>
>>
>> Freezing all the core modules into the executable should reduce start up
>> time.
>
> That suggests a test to me that the Cython guys might be interested in (or
> may well have performed in the past). How much of the stdlib could be
> compiled with Cython and used during the startup process?

We have a Jenkins job set up to run the CPython test suite with a compiled 
stdlib:

https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests-pyregr-stdlib/

Basically, we use pyximport as an import hook that tries to compile Python 
modules on import and then imports the shared library if it worked or the 
original Python module if it failed. A solution that explicitly runs over 
the stdlib and compiles it would be substantially cleaner and more stable.

I don't have numbers for Py3.4 because we currently have a hard crash in 
one of the tests on that platform when compiling recursively on import 
(likely meaning that one of the stdlib modules and/or tests would have to 
be excluded from compilation), but I get 434 automatically compiled stdlib 
modules for the latest Py2.7 branch out of 744 (excluding the test suite). 
And Py3.x code tends to pass as least as well through the compiler, often 
better.

Note that quite a number of modules are excluded accidentally because they 
are already imported as Python modules when Cython starts working. 
Compiling them explicitly would remove that limitation, maybe adding 
another (wild guess) 50 modules or so. Another few are not being compiled 
because the test module that uses them fails to compile. So missing shared 
libraries are not always due to failures to compile that particular Python 
module.

I didn't pay much attention to this part of our integration tests so far - 
a bit of debugging should get the Py3.4 build working.


> How much of an
> effect would it have on startup times and these benchmarks if
> Cython-compiled extensions were used?

Depends on what and how much code you use. If you compile everything into 
one big module that "imports" all of the stdlib when it gets loaded, you'd 
likely loose a lot of time because it would take a while to initialise all 
that useless code on startup. If you keep it separate, it would likely be a 
lot faster because you avoid the interpreter for most of the module startup.

Most Python code runs about 30% faster when compiled, some faster, some 
slower. If you want better numbers, you can start optimising the code by 
giving Cython static type hints. I did that for difflib a while ago, for 
example. Changing two methods made it some 50% faster back then:

http://blog.behnel.de/index.php?p=155

That particular module should compile without changes these days, and you 
can provide the type hints externally, i.e. without modifying the Python 
code itself.


> I'm thinking here of elimination of .pyc interpretation and execution (stat
> calls would be similar, probably slightly higher).

CPython checks for .so files before looking for .py files and imports are 
absolute by default in Py3, so there should be a slight reduction in stat 
calls. The net result then obviously also depends on how fast your shared 
library loader and linker is, etc., but I doubt that that path is any 
slower than loading and running a .pyc file.

BTW, you'd still get nice stack traces for compiled modules as long as your 
.py files lie right next to your .so files.


> To be clear - I'm *not* suggesting Cython become part of the required build
> toolchain. But *if* the Cython-compiled extensions prove to be
> significantly faster I'm thinking maybe it could become a semi-supported
> option (e.g. a HOWTO with the caveat "it worked on this particular system").

Sounds reasonable.

Stefan



From stefan_ml at behnel.de  Sun Oct 28 08:37:19 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 28 Oct 2012 08:37:19 +0100
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <k6imeu$rk1$1@ger.gmane.org>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAN8CLgkttfmgJkt2E1H5TMkkbWwPza6Mv757AuWN6cppHV3XyQ@mail.gmail.com>
	<k6imeu$rk1$1@ger.gmane.org>
Message-ID: <k6inbd$1hi$1@ger.gmane.org>

Stefan Behnel, 28.10.2012 08:22:
> Tim Delaney, 27.10.2012 22:53:
>> How much of an effect would it have on startup times and these benchmarks if
>> Cython-compiled extensions were used?
>
> Depends on what and how much code you use. If you compile everything into
> one big module that "imports" all of the stdlib when it gets loaded, you'd
> likely loose a lot of time because it would take a while to initialise all
> that useless code on startup. If you keep it separate, it would likely be a
> lot faster because you avoid the interpreter for most of the module startup.
>
> Most Python code runs about 30% faster when compiled, some faster, some
> slower.

Some more unoptimised pure-Python benchmarks, just in case:

2.7:

https://sage.math.washington.edu:8091/hudson/job/cython-devel-pybenchmarks-py27/lastSuccessfulBuild/artifact/bench_chart.html

3.3:

https://sage.math.washington.edu:8091/hudson/job/cython-devel-pybenchmarks-py3k/lastSuccessfulBuild/artifact/bench_chart.html

Note that the 3.3 benchmarks are not entirely up to date, the last 
successful run was a month ago (likely due to the branch into 3.4 which we 
use since then). Didn't have time to fix them yet.

Note also that the variations are pretty high from run to run as the 
machine that executes them is not a dedicated benchmark server.

Stefan



From g.brandl at gmx.net  Sun Oct 28 08:46:16 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 28 Oct 2012 08:46:16 +0100
Subject: [Python-Dev] cpython (merge 3.2 -> 3.2): null merge
In-Reply-To: <3XhbVM67d5zR6c@mail.python.org>
References: <3XhbVM67d5zR6c@mail.python.org>
Message-ID: <k6inpf$20d$1@ger.gmane.org>

Not very important, but this is not a null merge, as you can see from the diff :)

Georg

Am 17.10.2012 16:33, schrieb andrew.svetlov:
> http://hg.python.org/cpython/rev/16493102f9b1
> changeset:   79798:16493102f9b1
> branch:      3.2
> parent:      79791:98f64cbed2ac
> parent:      79795:a8052ad9752b
> user:        Andrew Svetlov <andrew.svetlov at gmail.com>
> date:        Wed Oct 17 17:20:46 2012 +0300
> summary:
>   null merge
> 
> files:
>   Doc/using/cmdline.rst |  6 ++++--
>   Misc/ACKS             |  1 +
>   2 files changed, 5 insertions(+), 2 deletions(-)
> 
> 
> diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst
> --- a/Doc/using/cmdline.rst
> +++ b/Doc/using/cmdline.rst
> @@ -379,7 +379,10 @@
>  Environment variables
>  ---------------------
>  
> -These environment variables influence Python's behavior.
> +These environment variables influence Python's behavior, they are processed
> +before the command-line switches other than -E.  It is customary that
> +command-line switches override environmental variables where there is a
> +conflict.
>  
>  .. envvar:: PYTHONHOME
>  
> @@ -570,4 +573,3 @@
>  
>     If set, Python will print memory allocation statistics every time a new
>     object arena is created, and on shutdown.
> -
> diff --git a/Misc/ACKS b/Misc/ACKS
> --- a/Misc/ACKS
> +++ b/Misc/ACKS
> @@ -901,6 +901,7 @@
>  Saskia van Rossum
>  Donald Wallace Rouse II
>  Liam Routt
> +Todd Rovito
>  Craig Rowland
>  Clinton Roy
>  Paul Rubin
> 
> 
> 
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
> 



From g.brandl at gmx.net  Sun Oct 28 08:52:58 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 28 Oct 2012 08:52:58 +0100
Subject: [Python-Dev] devguide: Silence Sphinx warning
In-Reply-To: <3Xjm7V5n8YzRBk@mail.python.org>
References: <3Xjm7V5n8YzRBk@mail.python.org>
Message-ID: <k6io61$20d$2@ger.gmane.org>

Am 19.10.2012 14:06, schrieb nick.coghlan:
> http://hg.python.org/devguide/rev/08f963e19a3e
> changeset:   559:08f963e19a3e
> user:        Nick Coghlan <ncoghlan at gmail.com>
> date:        Fri Oct 19 22:06:19 2012 +1000
> summary:
>   Silence Sphinx warning
> 
> files:
>   setup.rst |  2 +-
>   1 files changed, 1 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/setup.rst b/setup.rst
> --- a/setup.rst
> +++ b/setup.rst
> @@ -184,7 +184,7 @@
>  '''''''
>  
>  **Python 3.3** and later use Microsoft Visual Studio 2010.  You can
> -download Microsoft Visual C++ 2010 Express `here
> +download Microsoft Visual C++ 2010 Express `from Microsoft's site
>  <http://www.microsoft.com/visualstudio/en-us/products/2010-editions/visual-cpp-express>`_.

JFTR, this is a Docutils warning, not a Sphinx warning, and can also be avoided
by using `...`__ (i.e. two underscores).  The one-underscore form creates a
"target" that you can then reference again from elsewhere; the two-underscore
form doesn't.

cheers,
Georg


From barry at python.org  Sun Oct 28 09:40:05 2012
From: barry at python.org (Barry Warsaw)
Date: Sun, 28 Oct 2012 04:40:05 -0400
Subject: [Python-Dev] [Python-checkins] peps: PEP 430 is Final.
In-Reply-To: <3XqBkH6G0DzMwG@mail.python.org>
References: <3XqBkH6G0DzMwG@mail.python.org>
Message-ID: <20121028044005.243e3511@resist>

On Oct 28, 2012, at 09:21 AM, georg.brandl wrote:

>  PEP 430 is Final.

From the PEP:

"The existing /py3k/ subpath would be remapped to the new /3/ subpath."

Does "remapped" mean redirects so as not to break the existing py3k urls?  If
so, then cool.

Cheers,
-Barry

From g.brandl at gmx.net  Sun Oct 28 10:38:58 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 28 Oct 2012 10:38:58 +0100
Subject: [Python-Dev] [Python-checkins] peps: PEP 430 is Final.
In-Reply-To: <20121028044005.243e3511@resist>
References: <3XqBkH6G0DzMwG@mail.python.org> <20121028044005.243e3511@resist>
Message-ID: <k6iucp$g91$1@ger.gmane.org>

Am 28.10.2012 09:40, schrieb Barry Warsaw:
> On Oct 28, 2012, at 09:21 AM, georg.brandl wrote:
> 
>>  PEP 430 is Final.
> 
> From the PEP:
> 
> "The existing /py3k/ subpath would be remapped to the new /3/ subpath."
> 
> Does "remapped" mean redirects so as not to break the existing py3k urls?  If
> so, then cool.

Yes, /py3k is now a permanent redirect to /3.

The other new URLs are not redirects anymore, so that /2.4/ no longer redirects
to /dev/.

Georg


From solipsis at pitrou.net  Sun Oct 28 12:11:10 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 28 Oct 2012 12:11:10 +0100
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAP1=2W46ErE8LJ4nJnM_q7HWNVaWh5CQrg642EB_JwCO4sc++A@mail.gmail.com>
	<k6hnri$nn$1@ger.gmane.org> <20121028010048.43547034@pitrou.net>
	<CAGE7PNKFW4Y2JBnM3C-xBVmwdusfgDCrPO-1vHau4Cd63VfMyA@mail.gmail.com>
Message-ID: <20121028121110.410887a3@pitrou.net>

On Sat, 27 Oct 2012 20:38:58 -0700
"Gregory P. Smith" <greg at krypto.org> wrote:
> One word: profile.
> 
> Looking at stat counts alone rather than measuring the total time spent in
> all types of system calls from strace and profiling is not really useful. ;)

Agreed, but I can't seem to cope properly with gprof. Any suggestion?

> Another thing to keep an eye out for within a startup profile:  how often
> does the gc collect?  our default gc collection thresholds haven't been
> tuned in ages afaik [or am i forgetting something] and I know of
> pathological cases at work where simply doing a gc.disable() before
> importing a bunch of modules (tons of generated protocol buffer code) and
> re-enabling it afterwards speeds up this application's startup way more
> significantly than seems healthy in 2.x... that could be related to the
> particulars of the protobuf module code though.

That's a good suggestion indeed.

Thanks

Antoine.



From solipsis at pitrou.net  Sun Oct 28 12:13:26 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 28 Oct 2012 12:13:26 +0100
Subject: [Python-Dev] cpython (merge 2.6 -> 2.7): merge with 2.6
References: <3Xq91S1rD9zR8Z@mail.python.org>
Message-ID: <20121028121326.16e54e79@pitrou.net>

On Sun, 28 Oct 2012 08:05:00 +0100 (CET)
georg.brandl <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/ee33671b2c6a
> changeset:   79995:ee33671b2c6a
> branch:      2.7
> parent:      79983:7ca30af90c11
> parent:      79994:4a17784f2fee
> user:        Georg Brandl <georg at python.org>
> date:        Sun Oct 28 08:06:11 2012 +0100
> summary:
>   merge with 2.6
> 
> files:
>   Doc/tools/sphinxext/static/version_switch.js |  18 +++++----
>   1 files changed, 10 insertions(+), 8 deletions(-)

Is it deliberate for our Javascript code to use 2-space indents?

Regards

Antoine.



From ncoghlan at gmail.com  Sun Oct 28 12:25:28 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 28 Oct 2012 21:25:28 +1000
Subject: [Python-Dev] [Python-ideas] docs.python.org
In-Reply-To: <k6iohl$7pi$1@ger.gmane.org>
References: <CABicbJ+65RAqsS8gzF4MCe3e=+bCKRc4-2hzTt1s4qxv=4JeTg@mail.gmail.com>
	<20121026224644.GA28636@cskk.homeip.net>
	<k6fifd$34v$1@ger.gmane.org>
	<81687767-DC69-4097-877B-0C5AEB471D28@gmail.com>
	<CADiSq7fDSUncafype0LW=dPKdwTfcU4w2+AiUSRQ_bjYiVfY+A@mail.gmail.com>
	<B80164DA-CCD5-481C-B489-2EE2D8A28DB3@gmail.com>
	<CADiSq7dbmRusBbLmjcGO6J3VQFogGjW46T43wDqXesa8xGHzqQ@mail.gmail.com>
	<508BBD5C.305@egenix.com>
	<CADiSq7ecMXzj1vcPo6sUJqXXZWHK=tu0R6C3YL7=N8XMu4GiBw@mail.gmail.com>
	<k6iohl$7pi$1@ger.gmane.org>
Message-ID: <CADiSq7cNzGQxAq+5P=32N21HAziDZFYVgwP18=QjtAH-OQrYEQ@mail.gmail.com>

On Sun, Oct 28, 2012 at 5:59 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Am 27.10.2012 16:40, schrieb Nick Coghlan:
>
>>>> 4. We add a notice like the one above to the home page of the 2.7
>>>> docs, announce it on the PSF blog, announce it far and wide
>>>
>>> We also need a solution for URLs that exist for Python 2, but
>>> not for Python 3. Those should be redirected to the Python 2
>>> resource automatically, e.g. URLs pointing to the Python 2 modules
>>> that were renamed in Python 3.
>>>
>>> BTW: Will you write up a PEP for this ?
>>
>> Committed as PEP 430, should show up
>> http://www.python.org/dev/peps/pep-0430 before too long.
>
> Well, with the approval I've seen here, I have absolutely no problem
> with appointing myself PEP Czar and accepting the PEP :)

Heh, asking you to do that was next on my list, so thanks. Did Guido
hide a mind reading device in the time machine? :)

> I'll work on fixing the Apache config.

Huzzah \o/

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sun Oct 28 12:26:46 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 28 Oct 2012 21:26:46 +1000
Subject: [Python-Dev] devguide: Silence Sphinx warning
In-Reply-To: <k6io61$20d$2@ger.gmane.org>
References: <3Xjm7V5n8YzRBk@mail.python.org>
	<k6io61$20d$2@ger.gmane.org>
Message-ID: <CADiSq7djfj182HBRNFR81aCh-6sCM0SN9xO=PEQhfcRU_HU_3w@mail.gmail.com>

On Sun, Oct 28, 2012 at 5:52 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> JFTR, this is a Docutils warning, not a Sphinx warning, and can also be avoided
> by using `...`__ (i.e. two underscores).  The one-underscore form creates a
> "target" that you can then reference again from elsewhere; the two-underscore
> form doesn't.

Ah, useful trick to know, thanks.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From chris.jerdonek at gmail.com  Sun Oct 28 12:29:11 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sun, 28 Oct 2012 04:29:11 -0700
Subject: [Python-Dev] [Python-checkins] peps: PEP for updating the URL
	layout on docs.python.org
In-Reply-To: <3Xpl5B1cv9zNm1@mail.python.org>
References: <3Xpl5B1cv9zNm1@mail.python.org>
Message-ID: <CAOTb1wf=s+vR9PK7zJoT7cUB-xC3c4+B8VkRfMcZdAb=JNw-Og@mail.gmail.com>

On Sat, Oct 27, 2012 at 7:36 AM, nick.coghlan
<python-checkins at python.org> wrote:
> http://hg.python.org/peps/rev/c7ba002ca91d
> changeset:   4568:c7ba002ca91d
> user:        Nick Coghlan <ncoghlan at gmail.com>
> date:        Sun Oct 28 00:36:36 2012 +1000
> summary:
>   PEP for updating the URL layout on docs.python.org

> +* ``http://docs.python.org/x/*``
> +* ``http://docs.python.org/x.y/*``
> +* ``http://docs.python.org/release/x.y.z/*``
> +* ``http://docs.python.org/devguide``
> +
> +The ``/x/`` URLs mean "give me the latest documentation for this release
> +series``. Differences relative to previous versions in the release series
> +will be available through "version added" and "version changed" markers.
> +
> +The ``/x.y/`` URLs mean "give me the latest documentation for this release".

It might be worth clarifying what "latest" means for both the "x" and
"x.y" forms.  For example, should "http://docs.python.org/3/" today
link to the released 3.3.0 version, the in-progress 3.3.1 version, or
the in-progress 3.4.0?

For the "x" form, I think we mean the second because that is what it
points to today, i.e. the in-development version of the highest
released minor version of the major version "x".

There's a slight mismatch with how we're doing it today because
"http://docs.python.org/3/" shows 3.3.0 in the title even though it's
the in-progress 3.3.1.  The title should perhaps reflect that it's
post 3.3.0 (and similarly for the 2.7 and 3.2 pages).

--Chris

From g.brandl at gmx.net  Sun Oct 28 12:34:54 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 28 Oct 2012 12:34:54 +0100
Subject: [Python-Dev] cpython (merge 2.6 -> 2.7): merge with 2.6
In-Reply-To: <20121028121326.16e54e79@pitrou.net>
References: <3Xq91S1rD9zR8Z@mail.python.org>
	<20121028121326.16e54e79@pitrou.net>
Message-ID: <k6j58s$2h3$2@ger.gmane.org>

Am 28.10.2012 12:13, schrieb Antoine Pitrou:
> On Sun, 28 Oct 2012 08:05:00 +0100 (CET)
> georg.brandl <python-checkins at python.org> wrote:
>> http://hg.python.org/cpython/rev/ee33671b2c6a
>> changeset:   79995:ee33671b2c6a
>> branch:      2.7
>> parent:      79983:7ca30af90c11
>> parent:      79994:4a17784f2fee
>> user:        Georg Brandl <georg at python.org>
>> date:        Sun Oct 28 08:06:11 2012 +0100
>> summary:
>>   merge with 2.6
>> 
>> files:
>>   Doc/tools/sphinxext/static/version_switch.js |  18 +++++----
>>   1 files changed, 10 insertions(+), 8 deletions(-)
> 
> Is it deliberate for our Javascript code to use 2-space indents?

Yes.

Georg


From ncoghlan at gmail.com  Sun Oct 28 13:08:36 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 28 Oct 2012 22:08:36 +1000
Subject: [Python-Dev] [Python-checkins] peps: PEP for updating the URL
	layout on docs.python.org
In-Reply-To: <k6j58g$2h3$1@ger.gmane.org>
References: <3Xpl5B1cv9zNm1@mail.python.org>
	<CAOTb1wf=s+vR9PK7zJoT7cUB-xC3c4+B8VkRfMcZdAb=JNw-Og@mail.gmail.com>
	<k6j58g$2h3$1@ger.gmane.org>
Message-ID: <CADiSq7f1p91X83uy-Dr_QYNCLt_0=oEFuP=NstJ45uMuX6XNcQ@mail.gmail.com>

On Sun, Oct 28, 2012 at 9:34 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Am 28.10.2012 12:29, schrieb Chris Jerdonek:
>> On Sat, Oct 27, 2012 at 7:36 AM, nick.coghlan
>> <python-checkins at python.org> wrote:
>>> http://hg.python.org/peps/rev/c7ba002ca91d
>>> changeset:   4568:c7ba002ca91d
>>> user:        Nick Coghlan <ncoghlan at gmail.com>
>>> date:        Sun Oct 28 00:36:36 2012 +1000
>>> summary:
>>>   PEP for updating the URL layout on docs.python.org
>>
>>> +* ``http://docs.python.org/x/*``
>>> +* ``http://docs.python.org/x.y/*``
>>> +* ``http://docs.python.org/release/x.y.z/*``
>>> +* ``http://docs.python.org/devguide``
>>> +
>>> +The ``/x/`` URLs mean "give me the latest documentation for this release
>>> +series``. Differences relative to previous versions in the release series
>>> +will be available through "version added" and "version changed" markers.
>>> +
>>> +The ``/x.y/`` URLs mean "give me the latest documentation for this release".
>>
>> It might be worth clarifying what "latest" means for both the "x" and
>> "x.y" forms.  For example, should "http://docs.python.org/3/" today
>> link to the released 3.3.0 version, the in-progress 3.3.1 version, or
>> the in-progress 3.4.0?
>
> I understand "latest" to mean "latest stable plus bugfixes".
> I.e., /3/ is 3.3.0+. /dev and /3.4 is 3.4a0.  It might need clarifying
> in the PEP.

Yes, I realised last night that it would be good to be more explicit
about this. The "release" hierarchy should be for versions exactly as
they were released, while the shorter ones draw directly from source
control (and thus may contain fixes which otherwise haven't been
released yet).

I updated the PEP to describe what Georg has put in place more
explicitly (initially I was going to propose a change to the way /dev/
URLs are handled, but then I realised what is already there made more
sense).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From chris.jerdonek at gmail.com  Sun Oct 28 13:19:26 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sun, 28 Oct 2012 05:19:26 -0700
Subject: [Python-Dev] [Python-checkins] peps: PEP for updating the URL
	layout on docs.python.org
In-Reply-To: <k6j58g$2h3$1@ger.gmane.org>
References: <3Xpl5B1cv9zNm1@mail.python.org>
	<CAOTb1wf=s+vR9PK7zJoT7cUB-xC3c4+B8VkRfMcZdAb=JNw-Og@mail.gmail.com>
	<k6j58g$2h3$1@ger.gmane.org>
Message-ID: <CAOTb1wdUOyj4ZbpEdnQ=LsmSqkxpo_jtqTt=7Chhv+AHmAhcbA@mail.gmail.com>

On Sun, Oct 28, 2012 at 4:34 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> Am 28.10.2012 12:29, schrieb Chris Jerdonek:
> ...
> I understand "latest" to mean "latest stable plus bugfixes".
> I.e., /3/ is 3.3.0+. /dev and /3.4 is 3.4a0.  It might need clarifying
> in the PEP.
> ...
>> There's a slight mismatch with how we're doing it today because
>> "http://docs.python.org/3/" shows 3.3.0 in the title even though it's
>> the in-progress 3.3.1.  The title should perhaps reflect that it's
>> post 3.3.0 (and similarly for the 2.7 and 3.2 pages).
>
> Well, that has always been the case, and it doesn't matter anyway, because
> generally there's nothing in 3.3.1, feature-wise, that won't have been in 3.3.0.

One reason to change would be to avoid possible confusion created on
pages like this--

http://docs.python.org/3.3/whatsnew/3.2.html

where it says--

Author: Raymond Hettinger
Release: 3.3.0
Date: October 27, 2012

Would there be any disadvantage to changing the in-development titles
to read something like 3.3.0+, etc?

--Chris

From g.brandl at gmx.net  Sun Oct 28 13:28:50 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 28 Oct 2012 13:28:50 +0100
Subject: [Python-Dev] [Python-checkins] peps: PEP for updating the URL
	layout on docs.python.org
In-Reply-To: <CAOTb1wdUOyj4ZbpEdnQ=LsmSqkxpo_jtqTt=7Chhv+AHmAhcbA@mail.gmail.com>
References: <3Xpl5B1cv9zNm1@mail.python.org>
	<CAOTb1wf=s+vR9PK7zJoT7cUB-xC3c4+B8VkRfMcZdAb=JNw-Og@mail.gmail.com>
	<k6j58g$2h3$1@ger.gmane.org>
	<CAOTb1wdUOyj4ZbpEdnQ=LsmSqkxpo_jtqTt=7Chhv+AHmAhcbA@mail.gmail.com>
Message-ID: <k6j8e0$p4h$1@ger.gmane.org>

Am 28.10.2012 13:19, schrieb Chris Jerdonek:
> On Sun, Oct 28, 2012 at 4:34 AM, Georg Brandl <g.brandl at gmx.net> wrote:
>> Am 28.10.2012 12:29, schrieb Chris Jerdonek:
>> ...
>> I understand "latest" to mean "latest stable plus bugfixes".
>> I.e., /3/ is 3.3.0+. /dev and /3.4 is 3.4a0.  It might need clarifying
>> in the PEP.
>> ...
>>> There's a slight mismatch with how we're doing it today because
>>> "http://docs.python.org/3/" shows 3.3.0 in the title even though it's
>>> the in-progress 3.3.1.  The title should perhaps reflect that it's
>>> post 3.3.0 (and similarly for the 2.7 and 3.2 pages).
>>
>> Well, that has always been the case, and it doesn't matter anyway, because
>> generally there's nothing in 3.3.1, feature-wise, that won't have been in 3.3.0.
> 
> One reason to change would be to avoid possible confusion created on
> pages like this--
> 
> http://docs.python.org/3.3/whatsnew/3.2.html
> 
> where it says--
> 
> Author: Raymond Hettinger
> Release: 3.3.0
> Date: October 27, 2012

Well, that block is a little silly anyway.  I would just delete the
"Release" and "Date" lines.

> Would there be any disadvantage to changing the in-development titles
> to read something like 3.3.0+, etc?

I'm not sure it would lower any confusion, instead of creating more.
("What is that + anyway? Do I need another version?" etc.)

Georg


From solipsis at pitrou.net  Sun Oct 28 13:30:35 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 28 Oct 2012 13:30:35 +0100
Subject: [Python-Dev] [Python-checkins] peps: PEP for updating the URL
 layout on docs.python.org
References: <3Xpl5B1cv9zNm1@mail.python.org>
	<CAOTb1wf=s+vR9PK7zJoT7cUB-xC3c4+B8VkRfMcZdAb=JNw-Og@mail.gmail.com>
	<k6j58g$2h3$1@ger.gmane.org>
	<CAOTb1wdUOyj4ZbpEdnQ=LsmSqkxpo_jtqTt=7Chhv+AHmAhcbA@mail.gmail.com>
Message-ID: <20121028133035.6eede217@pitrou.net>

On Sun, 28 Oct 2012 05:19:26 -0700
Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
> 
> One reason to change would be to avoid possible confusion created on
> pages like this--
> 
> http://docs.python.org/3.3/whatsnew/3.2.html
> 
> where it says--
> 
> Author: Raymond Hettinger
> Release: 3.3.0
> Date: October 27, 2012
> 
> Would there be any disadvantage to changing the in-development titles
> to read something like 3.3.0+, etc?

Well, first why does it mention 3.3.0 while it's the what's new for
3.2? That's totally confusing, this mention should simply be removed.
Also the date is not informative at all.

Regards

Antoine.



From g.brandl at gmx.net  Sun Oct 28 13:39:18 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 28 Oct 2012 13:39:18 +0100
Subject: [Python-Dev] [Python-checkins] peps: PEP for updating the URL
	layout on docs.python.org
In-Reply-To: <20121028133035.6eede217@pitrou.net>
References: <3Xpl5B1cv9zNm1@mail.python.org>
	<CAOTb1wf=s+vR9PK7zJoT7cUB-xC3c4+B8VkRfMcZdAb=JNw-Og@mail.gmail.com>
	<k6j58g$2h3$1@ger.gmane.org>
	<CAOTb1wdUOyj4ZbpEdnQ=LsmSqkxpo_jtqTt=7Chhv+AHmAhcbA@mail.gmail.com>
	<20121028133035.6eede217@pitrou.net>
Message-ID: <k6j91l$u6g$1@ger.gmane.org>

Am 28.10.2012 13:30, schrieb Antoine Pitrou:
> On Sun, 28 Oct 2012 05:19:26 -0700
> Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
>> 
>> One reason to change would be to avoid possible confusion created on
>> pages like this--
>> 
>> http://docs.python.org/3.3/whatsnew/3.2.html
>> 
>> where it says--
>> 
>> Author: Raymond Hettinger
>> Release: 3.3.0
>> Date: October 27, 2012
>> 
>> Would there be any disadvantage to changing the in-development titles
>> to read something like 3.3.0+, etc?
> 
> Well, first why does it mention 3.3.0 while it's the what's new for
> 3.2? That's totally confusing, this mention should simply be removed.
> Also the date is not informative at all.

Agreed, I've now removed them.

Georg


From chris.jerdonek at gmail.com  Sun Oct 28 13:54:47 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sun, 28 Oct 2012 05:54:47 -0700
Subject: [Python-Dev] [Python-checkins] peps: PEP for updating the URL
 layout on docs.python.org
In-Reply-To: <k6j91l$u6g$1@ger.gmane.org>
References: <3Xpl5B1cv9zNm1@mail.python.org>
	<CAOTb1wf=s+vR9PK7zJoT7cUB-xC3c4+B8VkRfMcZdAb=JNw-Og@mail.gmail.com>
	<k6j58g$2h3$1@ger.gmane.org>
	<CAOTb1wdUOyj4ZbpEdnQ=LsmSqkxpo_jtqTt=7Chhv+AHmAhcbA@mail.gmail.com>
	<20121028133035.6eede217@pitrou.net> <k6j91l$u6g$1@ger.gmane.org>
Message-ID: <CAOTb1wcWgRAkv2dDkU-3-SX52hKsHHcC0SadGUrjfJxeFjC2AQ@mail.gmail.com>

On Sun, Oct 28, 2012 at 5:39 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> Am 28.10.2012 13:30, schrieb Antoine Pitrou:
>> Well, first why does it mention 3.3.0 while it's the what's new for
>> 3.2? That's totally confusing, this mention should simply be removed.
>> Also the date is not informative at all.
>
> Agreed, I've now removed them.

Thanks a lot.  Many of the index pages linked from the table of
contents also have uninformative release and date lines, e.g.

http://docs.python.org/3/library/index.html

but they use ":Release: |version|" (minor version without micro
version) instead of ":Release: |release|", so they're less likely to
be construed as an actual release date (but still uninformative).

--Chris

From g.brandl at gmx.net  Sun Oct 28 14:05:13 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 28 Oct 2012 14:05:13 +0100
Subject: [Python-Dev] [Python-checkins] peps: PEP for updating the URL
	layout on docs.python.org
In-Reply-To: <CAOTb1wcWgRAkv2dDkU-3-SX52hKsHHcC0SadGUrjfJxeFjC2AQ@mail.gmail.com>
References: <3Xpl5B1cv9zNm1@mail.python.org>
	<CAOTb1wf=s+vR9PK7zJoT7cUB-xC3c4+B8VkRfMcZdAb=JNw-Og@mail.gmail.com>
	<k6j58g$2h3$1@ger.gmane.org>
	<CAOTb1wdUOyj4ZbpEdnQ=LsmSqkxpo_jtqTt=7Chhv+AHmAhcbA@mail.gmail.com>
	<20121028133035.6eede217@pitrou.net> <k6j91l$u6g$1@ger.gmane.org>
	<CAOTb1wcWgRAkv2dDkU-3-SX52hKsHHcC0SadGUrjfJxeFjC2AQ@mail.gmail.com>
Message-ID: <k6jai8$9a8$1@ger.gmane.org>

Am 28.10.2012 13:54, schrieb Chris Jerdonek:
> On Sun, Oct 28, 2012 at 5:39 AM, Georg Brandl <g.brandl at gmx.net> wrote:
>> Am 28.10.2012 13:30, schrieb Antoine Pitrou:
>>> Well, first why does it mention 3.3.0 while it's the what's new for
>>> 3.2? That's totally confusing, this mention should simply be removed.
>>> Also the date is not informative at all.
>>
>> Agreed, I've now removed them.
> 
> Thanks a lot.  Many of the index pages linked from the table of
> contents also have uninformative release and date lines, e.g.
> 
> http://docs.python.org/3/library/index.html
> 
> but they use ":Release: |version|" (minor version without micro
> version) instead of ":Release: |release|", so they're less likely to
> be construed as an actual release date (but still uninformative).

Right.  I think they are remnants from the LaTeX age; you can remove
them all if you want.

Georg


From hs at ox.cx  Sun Oct 28 14:25:40 2012
From: hs at ox.cx (Hynek Schlawack)
Date: Sun, 28 Oct 2012 14:25:40 +0100
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
	but this time more solid numbers)
In-Reply-To: <20121028121110.410887a3@pitrou.net>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAP1=2W46ErE8LJ4nJnM_q7HWNVaWh5CQrg642EB_JwCO4sc++A@mail.gmail.com>
	<k6hnri$nn$1@ger.gmane.org> <20121028010048.43547034@pitrou.net>
	<CAGE7PNKFW4Y2JBnM3C-xBVmwdusfgDCrPO-1vHau4Cd63VfMyA@mail.gmail.com>
	<20121028121110.410887a3@pitrou.net>
Message-ID: <A111ABBF-A262-48DE-92BA-74F7AB8F9583@ox.cx>


Am 28.10.2012 um 12:11 schrieb Antoine Pitrou <solipsis at pitrou.net>:

>> One word: profile.
>> 
>> Looking at stat counts alone rather than measuring the total time spent in
>> all types of system calls from strace and profiling is not really useful. ;)
> Agreed, but I can't seem to cope properly with gprof. Any suggestion?

http://oprofile.sourceforge.net/news/
http://valgrind.org/docs/manual/cl-manual.html

Are both useful.  gprof is virtually useless.

From yselivanov.ml at gmail.com  Sun Oct 28 15:43:06 2012
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Sun, 28 Oct 2012 10:43:06 -0400
Subject: [Python-Dev] [Python-ideas] docs.python.org
In-Reply-To: <k6iohl$7pi$1@ger.gmane.org>
References: <CABicbJ+65RAqsS8gzF4MCe3e=+bCKRc4-2hzTt1s4qxv=4JeTg@mail.gmail.com>
	<20121026224644.GA28636@cskk.homeip.net>
	<k6fifd$34v$1@ger.gmane.org>
	<81687767-DC69-4097-877B-0C5AEB471D28@gmail.com>
	<CADiSq7fDSUncafype0LW=dPKdwTfcU4w2+AiUSRQ_bjYiVfY+A@mail.gmail.com>
	<B80164DA-CCD5-481C-B489-2EE2D8A28DB3@gmail.com>
	<CADiSq7dbmRusBbLmjcGO6J3VQFogGjW46T43wDqXesa8xGHzqQ@mail.gmail.com>
	<508BBD5C.305@egenix.com>
	<CADiSq7ecMXzj1vcPo6sUJqXXZWHK=tu0R6C3YL7=N8XMu4GiBw@mail.gmail.com>
	<k6iohl$7pi$1@ger.gmane.org>
Message-ID: <25E481E9-CF7B-4C96-BD8D-8AE0270FB712@gmail.com>

On 2012-10-28, at 3:59 AM, Georg Brandl <g.brandl at gmx.net> wrote:

> Well, with the approval I've seen here, I have absolutely no problem
> with appointing myself PEP Czar and accepting the PEP :)

That's awesome!

-
Yury

From breamoreboy at yahoo.co.uk  Sun Oct 28 17:23:26 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 28 Oct 2012 16:23:26 +0000
Subject: [Python-Dev] [Python-checkins] peps: PEP for updating the URL
	layout on docs.python.org
In-Reply-To: <k6j91l$u6g$1@ger.gmane.org>
References: <3Xpl5B1cv9zNm1@mail.python.org>
	<CAOTb1wf=s+vR9PK7zJoT7cUB-xC3c4+B8VkRfMcZdAb=JNw-Og@mail.gmail.com>
	<k6j58g$2h3$1@ger.gmane.org>
	<CAOTb1wdUOyj4ZbpEdnQ=LsmSqkxpo_jtqTt=7Chhv+AHmAhcbA@mail.gmail.com>
	<20121028133035.6eede217@pitrou.net> <k6j91l$u6g$1@ger.gmane.org>
Message-ID: <k6jm1b$v9h$1@ger.gmane.org>

On 28/10/2012 12:39, Georg Brandl wrote:
> Am 28.10.2012 13:30, schrieb Antoine Pitrou:
>> On Sun, 28 Oct 2012 05:19:26 -0700
>> Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
>>>
>>> One reason to change would be to avoid possible confusion created on
>>> pages like this--
>>>
>>> http://docs.python.org/3.3/whatsnew/3.2.html
>>>
>>> where it says--
>>>
>>> Author: Raymond Hettinger
>>> Release: 3.3.0
>>> Date: October 27, 2012
>>>
>>> Would there be any disadvantage to changing the in-development titles
>>> to read something like 3.3.0+, etc?
>>
>> Well, first why does it mention 3.3.0 while it's the what's new for
>> 3.2? That's totally confusing, this mention should simply be removed.
>> Also the date is not informative at all.
>
> Agreed, I've now removed them.
>
> Georg
>

Is it worth removing this as well, it's a couple of lines down

"See also PEP 392 - Python 3.2 Release Schedule" ?

-- 
Cheers.

Mark Lawrence.


From g.brandl at gmx.net  Sun Oct 28 17:43:22 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 28 Oct 2012 17:43:22 +0100
Subject: [Python-Dev] [Python-checkins] peps: PEP for updating the URL
	layout on docs.python.org
In-Reply-To: <k6jm1b$v9h$1@ger.gmane.org>
References: <3Xpl5B1cv9zNm1@mail.python.org>
	<CAOTb1wf=s+vR9PK7zJoT7cUB-xC3c4+B8VkRfMcZdAb=JNw-Og@mail.gmail.com>
	<k6j58g$2h3$1@ger.gmane.org>
	<CAOTb1wdUOyj4ZbpEdnQ=LsmSqkxpo_jtqTt=7Chhv+AHmAhcbA@mail.gmail.com>
	<20121028133035.6eede217@pitrou.net>
	<k6j91l$u6g$1@ger.gmane.org> <k6jm1b$v9h$1@ger.gmane.org>
Message-ID: <k6jnb9$dlo$1@ger.gmane.org>

Am 28.10.2012 17:23, schrieb Mark Lawrence:
> On 28/10/2012 12:39, Georg Brandl wrote:
>> Am 28.10.2012 13:30, schrieb Antoine Pitrou:
>>> On Sun, 28 Oct 2012 05:19:26 -0700
>>> Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
>>>>
>>>> One reason to change would be to avoid possible confusion created on
>>>> pages like this--
>>>>
>>>> http://docs.python.org/3.3/whatsnew/3.2.html
>>>>
>>>> where it says--
>>>>
>>>> Author: Raymond Hettinger
>>>> Release: 3.3.0
>>>> Date: October 27, 2012
>>>>
>>>> Would there be any disadvantage to changing the in-development titles
>>>> to read something like 3.3.0+, etc?
>>>
>>> Well, first why does it mention 3.3.0 while it's the what's new for
>>> 3.2? That's totally confusing, this mention should simply be removed.
>>> Also the date is not informative at all.
>>
>> Agreed, I've now removed them.
>>
>> Georg
>>
> 
> Is it worth removing this as well, it's a couple of lines down
> 
> "See also PEP 392 - Python 3.2 Release Schedule" ?

Why?  PEP 392 is not out of date, and for those who want to know when
exactly 3.2 and its minor versions were released, it is useful.

Georg


From solipsis at pitrou.net  Sun Oct 28 18:38:22 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 28 Oct 2012 18:38:22 +0100
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAP1=2W46ErE8LJ4nJnM_q7HWNVaWh5CQrg642EB_JwCO4sc++A@mail.gmail.com>
	<k6hnri$nn$1@ger.gmane.org> <20121028010048.43547034@pitrou.net>
	<CAGE7PNKFW4Y2JBnM3C-xBVmwdusfgDCrPO-1vHau4Cd63VfMyA@mail.gmail.com>
Message-ID: <20121028183822.71c2c28e@pitrou.net>

On Sat, 27 Oct 2012 20:38:58 -0700
"Gregory P. Smith" <greg at krypto.org> wrote:
> 
> Another thing to keep an eye out for within a startup profile:  how often
> does the gc collect?  our default gc collection thresholds haven't been
> tuned in ages afaik [or am i forgetting something] and I know of
> pathological cases at work where simply doing a gc.disable() before
> importing a bunch of modules (tons of generated protocol buffer code) and
> re-enabling it afterwards speeds up this application's startup way more
> significantly than seems healthy in 2.x... that could be related to the
> particulars of the protobuf module code though.

http://bugs.python.org/issue16351 shows us that the number of
collections at 3.4 startup is tiny:

$ ./python -Sc "import gc; print(gc.get_stats())"
[{'collections': 6, 'uncollectable': 0, 'collected': 0},
{'collections': 0, 'uncollectable': 0, 'collected': 0},
{'collections': 0, 'uncollectable': 0, 'collected': 0}]

$ ./python -c "import gc; print(gc.get_stats())"
[{'collected': 0, 'uncollectable': 0, 'collections': 12},
{'collected': 0, 'uncollectable': 0, 'collections': 1},
{'collected': 0, 'uncollectable': 0, 'collections': 0}]


Notably, there are no full collections.

Regards

Antoine.



From chris.jerdonek at gmail.com  Sun Oct 28 19:22:31 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sun, 28 Oct 2012 11:22:31 -0700
Subject: [Python-Dev] [Python-checkins] peps: PEP for updating the URL
 layout on docs.python.org
In-Reply-To: <k6jai8$9a8$1@ger.gmane.org>
References: <3Xpl5B1cv9zNm1@mail.python.org>
	<CAOTb1wf=s+vR9PK7zJoT7cUB-xC3c4+B8VkRfMcZdAb=JNw-Og@mail.gmail.com>
	<k6j58g$2h3$1@ger.gmane.org>
	<CAOTb1wdUOyj4ZbpEdnQ=LsmSqkxpo_jtqTt=7Chhv+AHmAhcbA@mail.gmail.com>
	<20121028133035.6eede217@pitrou.net> <k6j91l$u6g$1@ger.gmane.org>
	<CAOTb1wcWgRAkv2dDkU-3-SX52hKsHHcC0SadGUrjfJxeFjC2AQ@mail.gmail.com>
	<k6jai8$9a8$1@ger.gmane.org>
Message-ID: <CAOTb1wcRm_kbG229hv=NBTt=fhPUERwF8iBm=JrmV8zhRkAoNQ@mail.gmail.com>

On Sun, Oct 28, 2012 at 6:05 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> Am 28.10.2012 13:54, schrieb Chris Jerdonek:
>> On Sun, Oct 28, 2012 at 5:39 AM, Georg Brandl <g.brandl at gmx.net> wrote:
>>> Am 28.10.2012 13:30, schrieb Antoine Pitrou:
>>>> Well, first why does it mention 3.3.0 while it's the what's new for
>>>> 3.2? That's totally confusing, this mention should simply be removed.
>>>> Also the date is not informative at all.
>>>
>>> Agreed, I've now removed them.
>>
>> Thanks a lot.  Many of the index pages linked from the table of
>> contents also have uninformative release and date lines, e.g.
>>
>> http://docs.python.org/3/library/index.html
>>
>> but they use ":Release: |version|" (minor version without micro
>> version) instead of ":Release: |release|", so they're less likely to
>> be construed as an actual release date (but still uninformative).
>
> Right.  I think they are remnants from the LaTeX age; you can remove
> them all if you want.

Great, I just removed them.

--Chris

From vinay_sajip at yahoo.co.uk  Sun Oct 28 19:24:36 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sun, 28 Oct 2012 18:24:36 +0000 (UTC)
Subject: [Python-Dev] configure produces a non-working Makefile in some
	scenarios, due to ASDLGEN
Message-ID: <loom.20121028T191937-98@post.gmane.org>

In some scenarios, configure produces a Makefile which fails because ASDLGEN
doesn't point to a working Python. In particular, it seems to assume that if
there is an executable called e.g. "python3.4" on the path, then that will be a
system Python.

In my perhaps unusual but IMO perfectly valid setup, I have various Python repos
set up like so:

$HOME/projects/python
                 |
                 +- default
                 |
                 +- 3.3
                 |
                 + and so on for 3.2, 3.1, 2.7

In order to facilitate testing some script against multiple Python versions, I
have shell scripts in $HOME/bin named python3.4, python3.3 etc. which just run
the programs in the relevant directories in $HOME/projects/python/. (I know I
can do this using aliases etc., but I think that's beside the point.)

When I run configure in the repo for the default branch, it appears to look for
a Python on the path named python3.4, finds one in $HOME/bin, and thus generates
an ASDLGEN value of "python3.4". If I happen to have a built Python in the
default repo, then the script in $HOME/bin will successfully run that, and all
appears well; but if I clean the default repo and re-run make, it fails at the
ASDLGEN step, because the $HOME/bin/python3.4 script fails, due to not being
able to invoke $HOME/projects/python/default/python.

Is this a bug in configure, or is my configuration regarded as too perverse to
support?

Regards,

Vinay Sajip


From solipsis at pitrou.net  Sun Oct 28 19:40:43 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 28 Oct 2012 19:40:43 +0100
Subject: [Python-Dev] configure produces a non-working Makefile in some
 scenarios, due to ASDLGEN
References: <loom.20121028T191937-98@post.gmane.org>
Message-ID: <20121028194043.09415a50@pitrou.net>

On Sun, 28 Oct 2012 18:24:36 +0000 (UTC)
Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> In some scenarios, configure produces a Makefile which fails because ASDLGEN
> doesn't point to a working Python. In particular, it seems to assume that if
> there is an executable called e.g. "python3.4" on the path, then that will be a
> system Python.
> 
> In my perhaps unusual but IMO perfectly valid setup, I have various Python repos
> set up like so:
> 
> $HOME/projects/python
>                  |
>                  +- default
>                  |
>                  +- 3.3
>                  |
>                  + and so on for 3.2, 3.1, 2.7
> 
> In order to facilitate testing some script against multiple Python versions, I
> have shell scripts in $HOME/bin named python3.4, python3.3 etc. which just run
> the programs in the relevant directories in $HOME/projects/python/. (I know I
> can do this using aliases etc., but I think that's beside the point.)
> 
> When I run configure in the repo for the default branch, it appears to look for
> a Python on the path named python3.4, finds one in $HOME/bin, and thus generates
> an ASDLGEN value of "python3.4". If I happen to have a built Python in the
> default repo, then the script in $HOME/bin will successfully run that, and all
> appears well; but if I clean the default repo and re-run make, it fails at the
> ASDLGEN step, because the $HOME/bin/python3.4 script fails, due to not being
> able to invoke $HOME/projects/python/default/python.
> 
> Is this a bug in configure, or is my configuration regarded as too perverse to
> support?

The latter IMO :)

Regards

Antoine.



From dholth at gmail.com  Sun Oct 28 19:47:43 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 28 Oct 2012 14:47:43 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
Message-ID: <CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>

I think Metadata 1.3 is done. Who would like to czar?
On Oct 22, 2012 12:53 PM, "Daniel Holth" <dholth at gmail.com> wrote:

> http://hg.python.org/peps/rev/50e8ea1a17a0
>
> Based on the other "required" field's absence in the wild, only
> "Metadata-Version", "Name", "Version", and "Summary" are required.
> Hopefully a clearer explanation of 0, 1, or many occurrences of each
> field.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121028/2dd45fa7/attachment.html>

From timothy.c.delaney at gmail.com  Sun Oct 28 20:48:16 2012
From: timothy.c.delaney at gmail.com (Tim Delaney)
Date: Mon, 29 Oct 2012 06:48:16 +1100
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <k6imeu$rk1$1@ger.gmane.org>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAN8CLgkttfmgJkt2E1H5TMkkbWwPza6Mv757AuWN6cppHV3XyQ@mail.gmail.com>
	<k6imeu$rk1$1@ger.gmane.org>
Message-ID: <CAN8CLgmw041P8V6SFa6uJfVC5dvx31j890rGGYa7x6yOF3-qEw@mail.gmail.com>

On 28 October 2012 18:22, Stefan Behnel <stefan_ml at behnel.de> wrote:

> How much of an
>
>> effect would it have on startup times and these benchmarks if
>> Cython-compiled extensions were used?
>>
>
> Depends on what and how much code you use. If you compile everything into
> one big module that "imports" all of the stdlib when it gets loaded, you'd
> likely loose a lot of time because it would take a while to initialise all
> that useless code on startup. If you keep it separate, it would likely be a
> lot faster because you avoid the interpreter for most of the module startup.
>

I was specifically thinking in terms of the tests Brett ran (that was the
full set on speed.python.org, wasn't it?), and having each stdlib module be
its own extension i.e. no big import module. A literal 1:1 replacement
where possible.


> I'm thinking here of elimination of .pyc interpretation and execution (stat
>
>> calls would be similar, probably slightly higher).
>>
>
> CPython checks for .so files before looking for .py files and imports are
> absolute by default in Py3, so there should be a slight reduction in stat
> calls. The net result then obviously also depends on how fast your shared
> library loader and linker is, etc., but I doubt that that path is any
> slower than loading and running a .pyc file.
>

D'oh. I knew that and still got it backwards.


> To be clear - I'm *not* suggesting Cython become part of the required build
>> toolchain. But *if* the Cython-compiled extensions prove to be
>> significantly faster I'm thinking maybe it could become a semi-supported
>> option (e.g. a HOWTO with the caveat "it worked on this particular
>> system").
>>
>
> Sounds reasonable.


I think a stdlib compile script + pre-packaged hints for the 3.3 release
would likely help both 3.3 and Cython acceptance.

Putting aside my development interest and looking at it purely from the PoV
of a Python *user*, I'd really like to see Cython on
speed.python.orgeventually (in two modes - one without hints as a
baseline and one with
hints). Of course the ideal situation would be to have every implementation
of Python 3.3 that is capable of running on the hardware contributing
numbers e.g. if/when Jython achieves 3.3 compatibility I'd love to see
numbers for it.

Tim Delaney
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121029/5932c56e/attachment.html>

From iacobcatalin at gmail.com  Sun Oct 28 21:53:33 2012
From: iacobcatalin at gmail.com (Catalin Iacob)
Date: Sun, 28 Oct 2012 21:53:33 +0100
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CACac1F8tF-fREvCH49TPKe2SLEUCVpj_yVpUn6_VBKpDbCy3qQ@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAN8CLgkttfmgJkt2E1H5TMkkbWwPza6Mv757AuWN6cppHV3XyQ@mail.gmail.com>
	<20121027225826.Horde.M0EjTUlCcOxQjEryQ2NWSMA@webmail.df.eu>
	<CACac1F8tF-fREvCH49TPKe2SLEUCVpj_yVpUn6_VBKpDbCy3qQ@mail.gmail.com>
Message-ID: <CAHg_5gpZLqfYzDJA-BHxSX8-Zqzk2M098TGZbauqwZtZEFvqbg@mail.gmail.com>

On Sat, Oct 27, 2012 at 11:07 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> Interestingly, I just did a quick test of this: This is on my Windows
> 7 PC, running under Powershell.

snip

> Looks like the normal configuration is over twice as fast as the zipped one...

This result is influenced by zipimport fseek-ing for every file in the
imported zip and fseek flushing buffers in Microsoft's CRT
implementation. There's a patch which avoids the seek in
http://bugs.python.org/issue8745. Reviews welcome!

With that patch the time taken to import is half of the current state
of things so according to your test that would make zipped and non
zipped configurations roughly equally fast.

From dholth at gmail.com  Sun Oct 28 22:48:31 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 28 Oct 2012 17:48:31 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<CAPZV6o8=Uk5zn+f7kGCs6cFQnq50Nd46yOpu9FMwu4p6ctbE4A@mail.gmail.com>
	<20121018211014.73d56057@pitrou.net>
	<CAG8k2+6oALhohqOePU-yJYBA=Hr=FjqzGJJ1g5Ah3pjR+fCuGA@mail.gmail.com>
	<CAPZV6o9nC+7CsOhvE5pyp_u_cKoaKhGzPJXNgFoTDqNkbkJZkg@mail.gmail.com>
	<CAG8k2+7krh3R5_C0Emn3mb1sy+072iUKCRS6X0N4VdwkJr27SA@mail.gmail.com>
	<CAG8k2+5YLwk1RGG2CqDnMQkijvPJJYHCO0SwEoJKVkak2vpTDw@mail.gmail.com>
	<87k3un5fwh.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+6Ck+zACr6eAOZGAShWWtFeVyqcWy=vTLG-PojvsZ+TZA@mail.gmail.com>
	<87a9vh694r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CAG8k2+4WAqNxd7pZ=h==WHvSq=f8UrizzUWPV70-rAC6bwQ_Cg@mail.gmail.com>
	<87626457t6.fsf@uwakimon.sk.tsukuba.ac.jp>
	<CADiSq7fgsz49X4X09L4QJPA+Gft17K1SmAUpwbnjKMj-4Hu34w@mail.gmail.com>
	<CAG8k2+68O-uca_k_=n9FiyDNo7DSNkg3nECCza_nVt9L4_LT2A@mail.gmail.com>
	<CAG8k2+6dtWoy5_dX2gVVQwyrt7d3X3pZxz-ZioELGMMoqMkTZA@mail.gmail.com>
Message-ID: <CAG8k2+7beH5ATZvhbfMUTdS6ASU4QMPtXWyRpDOnEkYof83LAA@mail.gmail.com>

Now with an implementation at the end and possibly better wording for
the required fields. I think it's feature complete.


PEP: 426
Title: Metadata for Python Software Packages 1.3
Version: $Revision$
Last-Modified: $Date$
Author: Daniel Holth <dholth at fastmail.fm>
Discussions-To: Distutils SIG
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 30 Aug 2012


Abstract
========

This PEP describes a mechanism for adding metadata to Python distributions.
It includes specifics of the field names, and their semantics and
usage.

This document specifies version 1.3 of the metadata format.
Version 1.0 is specified in PEP 241.
Version 1.1 is specified in PEP 314.
Version 1.2 is specified in PEP 345.

Version 1.3 of the metadata format adds fields designed to make
third-party packaging of Python Software easier and defines a
formal extension mechanism.  The fields are "Setup-Requires-Dist"
"Provides-Extra", and "Extension".  This version also adds the `extra`
variable to the `environment markers` specification and allows the
description to be placed into a payload section.

Metadata Files
==============

The syntax defined in this PEP is for use with Python distribution
metadata files. The file format is a simple UTF-8 encoded Key: value
format with no maximum line length, followed by a blank line and an
arbitrary payload.  It is parseable by the ``email`` module with an
appropriate ``email.policy.Policy()``.

When ``metadata`` is a Unicode string,
```email.parser.Parser().parsestr(metadata)`` is a serviceable parser.

There are two standard locations for these metadata files:

* the ``PKG-INFO`` file included in the base directory of Python
  source distribution archives (as created by the distutils ``sdist``
  command)
* the ``.dist-info/METADATA`` files in a Python installation database, as
  described in PEP 376.

Other tools involved in Python distribution may also use this format.

Encoding
========

Metadata 1.3 files are UTF-8 with the restriction that keys must be
ASCII. Parser implementations should be aware that older versions of
the Metadata specification do not specify an encoding.

Fields
======

This section specifies the names and semantics of each of the
supported metadata fields.

In a single Metadata 1.3 file, fields marked with "(optional)" may occur
0 or 1 times.  Fields marked with "(multiple use)" may be specified
0, 1 or more times.  Only "Metadata-Version", "Name", "Version", and
"Summary" must appear exactly once.  The fields may appear in any order
within the file.

Metadata-Version
::::::::::::::::

Version of the file format; "1.3" is the only legal value.

Example::

    Metadata-Version: 1.3


Name
::::

The name of the distribution.

Example::

    Name: BeagleVote


Version
:::::::

A string containing the distribution's version number.  This
field  must be in the format specified in PEP 386.

Example::

    Version: 1.0a2


Summary
:::::::

A one-line summary of what the distribution does.

Example::

    Summary: A module for collecting votes from beagles.


Platform (multiple use)
:::::::::::::::::::::::

A Platform specification describing an operating system supported by
the distribution which is not listed in the "Operating System" Trove
classifiers.
See "Classifier" below.

Examples::

    Platform: ObscureUnix
    Platform: RareDOS


Supported-Platform (multiple use)
:::::::::::::::::::::::::::::::::

Binary distributions containing a metadata file will use the
Supported-Platform field in their metadata to specify the OS and
CPU for which the binary distribution was compiled.  The semantics of
the Supported-Platform field are not specified in this PEP.

Example::

    Supported-Platform: RedHat 7.2
    Supported-Platform: i386-win32-2791


Description (optional, deprecated)
::::::::::::::::::::::::::::::::::

A longer description of the distribution that can run to several
paragraphs.  Software that deals with metadata should not assume
any maximum size for this field.

The contents of this field can be written using reStructuredText
markup [1]_.  For programs that work with the metadata, supporting
markup is optional; programs can also display the contents of the
field as-is.  This means that authors should be conservative in
the markup they use.

Since a line separator immediately followed by another line separator
indicates the end of the headers section, any line separators in the
description must be suffixed by whitespace to indicate continuation.

Since Metadata 1.3 the recommended place for the description is in the
payload section of the document, after the last header.  The description
does not need to be reformatted when it is included in the payload.


Keywords (optional)
:::::::::::::::::::

A list of additional keywords to be used to assist searching
for the distribution in a larger catalog.

Example::

    Keywords: dog puppy voting election


Home-page (optional)
::::::::::::::::::::

A string containing the URL for the distribution's home page.

Example::

    Home-page: http://www.example.com/~cschultz/bvote/


Download-URL (optional)
:::::::::::::::::::::::

A string containing the URL from which this version of the distribution
can be downloaded.  (This means that the URL can't be something like
".../BeagleVote-latest.tgz", but instead must be ".../BeagleVote-0.45.tgz".)


Author (optional)
:::::::::::::::::

A string containing the author's name at a minimum; additional
contact information may be provided.

Example::

    Author: C. Schultz, Universal Features Syndicate,
            Los Angeles, CA <cschultz at peanuts.example.com>


Author-email (optional)
:::::::::::::::::::::::

A string containing the author's e-mail address.  It contains a name
and e-mail address in the RFC 5322 recommended ``Address Specification``
format.

Example::

    Author-email: "C. Schultz" <cschultz at example.com>


Maintainer (optional)
:::::::::::::::::::::

A string containing the maintainer's name at a minimum; additional
contact information may be provided.

Note that this field is intended for use when a project is being
maintained by someone other than the original author:  it should be
omitted if it is identical to ``Author``.

Example::

    Maintainer: C. Schultz, Universal Features Syndicate,
            Los Angeles, CA <cschultz at peanuts.example.com>


Maintainer-email (optional)
:::::::::::::::::::::::::::

A string containing the maintainer's e-mail address.  It has the same
format as ``Author-email``.

Note that this field is intended for use when a project is being
maintained by someone other than the original author:  it should be
omitted if it is identical to ``Author-email``.

Example::

    Maintainer-email: "C. Schultz" <cschultz at example.com>


License (optional)
::::::::::::::::::

Text indicating the license covering the distribution where the license
is not a selection from the "License" Trove classifiers. See
"Classifier" below.  This field may also be used to specify a
particular version of a license which is named via the ``Classifier``
field, or to indicate a variation or exception to such a license.

Examples::

    License: This software may only be obtained by sending the
            author a postcard, and then the user promises not
            to redistribute it.

    License: GPL version 3, excluding DRM provisions

The full text of the license would normally be included in a separate
file.


Classifier (multiple use)
:::::::::::::::::::::::::

Each entry is a string giving a single classification value
for the distribution.  Classifiers are described in PEP 301 [2].

Examples::

    Classifier: Development Status :: 4 - Beta
    Classifier: Environment :: Console (Text Based)


Requires-Dist (multiple use)
::::::::::::::::::::::::::::

Each entry contains a string naming some other distutils
project required by this distribution.

The format of a requirement string is identical to that of a
distutils project name (e.g., as found in the ``Name:`` field.
optionally followed by a version declaration within parentheses.

The distutils project names should correspond to names as found
on the `Python Package Index`_.

Version declarations must follow the rules described in
`Version Specifiers`_

Examples::

    Requires-Dist: pkginfo
    Requires-Dist: PasteDeploy
    Requires-Dist: zope.interface (>3.5.0)


Setup-Requires-Dist (multiple use)
::::::::::::::::::::::::::::::::::

Like Requires-Dist, but names dependencies needed while the
distributions's distutils / packaging `setup.py` / `setup.cfg` is run.
Commonly used to generate a manifest from version control.

Examples::

    Setup-Requires-Dist: custom_setup_command

Dependencies mentioned in `Setup-Requires-Dist` may be installed exclusively
for setup and are not guaranteed to be available at run time.


Provides-Dist (multiple use)
::::::::::::::::::::::::::::

Each entry contains a string naming a Distutils project which
is contained within this distribution.  This field *must* include
the project identified in the ``Name`` field, followed by the
version : Name (Version).

A distribution may provide additional names, e.g. to indicate that
multiple projects have been bundled together.  For instance, source
distributions of the ``ZODB`` project have historically included
the ``transaction`` project, which is now available as a separate
distribution.  Installing such a source distribution satisfies
requirements for both ``ZODB`` and ``transaction``.

A distribution may also provide a "virtual" project name, which does
not correspond to any separately-distributed project:  such a name
might be used to indicate an abstract capability which could be supplied
by one of multiple projects.  E.g., multiple projects might supply
RDBMS bindings for use by a given ORM:  each project might declare
that it provides ``ORM-bindings``, allowing other projects to depend
only on having at most one of them installed.

A version declaration may be supplied and must follow the rules described
in `Version Specifiers`_. The distribution's version number will be implied
if none is specified.

Examples::

    Provides-Dist: OtherProject
    Provides-Dist: AnotherProject (3.4)
    Provides-Dist: virtual_package


Obsoletes-Dist (multiple use)
:::::::::::::::::::::::::::::

Each entry contains a string describing a distutils project's distribution
which this distribution renders obsolete, meaning that the two projects
should not be installed at the same time.

Version declarations can be supplied.  Version numbers must be in the
format specified in `Version Specifiers`_.

The most common use of this field will be in case a project name
changes, e.g. Gorgon 2.3 gets subsumed into Torqued Python 1.0.
When you install Torqued Python, the Gorgon distribution should be
removed.

Examples::

    Obsoletes-Dist: Gorgon
    Obsoletes-Dist: OtherProject (<3.0)


Requires-Python (optional)
::::::::::::::::::::::::::

This field specifies the Python version(s) that the distribution is
guaranteed to be compatible with.

Version numbers must be in the format specified in `Version Specifiers`_.

Examples::

    Requires-Python: 2.5
    Requires-Python: >2.1
    Requires-Python: >=2.3.4
    Requires-Python: >=2.5,<2.7


Requires-External (multiple use)
::::::::::::::::::::::::::::::::

Each entry contains a string describing some dependency in the
system that the distribution is to be used.  This field is intended to
serve as a hint to downstream project maintainers, and has no
semantics which are meaningful to the ``distutils`` distribution.

The format of a requirement string is a name of an external
dependency, optionally followed by a version declaration within
parentheses.

Because they refer to non-Python software releases, version numbers
for this field are **not** required to conform to the format
specified in PEP 386:  they should correspond to the
version scheme used by the external dependency.

Notice that there's is no particular rule on the strings to be used.

Examples::

    Requires-External: C
    Requires-External: libpng (>=1.5)


Project-URL (multiple use)
::::::::::::::::::::::::::

A string containing a browsable URL for the project and a label for it,
separated by a comma.

Example::

    Bug Tracker, http://bitbucket.org/tarek/distribute/issues/

The label is a free text limited to 32 signs.


Provides-Extra (multiple use)
:::::::::::::::::::::::::::::

A string containing the name of an optional feature. Must be printable
ASCII, not containing whitespace, comma (,), or square brackets [].
May be used to make a dependency conditional on whether the optional
feature has been requested.

Example::

    Name: beaglevote
    Provides-Extra: pdf
    Requires-Dist: reportlab; extra == 'pdf'
    Requires-Dist: nose; extra == 'test'
    Requires-Dist: sphinx; extra == 'doc'

A second distribution requires an optional dependency by placing it
inside square brackets and can request multiple features by separating
them with a comma (,).

The full set of requirements is the union of the `Requires-Dist` sets
evaluated with `extra` set to `None` and then to the name of each
requested feature.

Example::

    Requires-Dist: beaglevote[pdf]
        -> requires beaglevote, reportlab

    Requires-Dist: beaglevote[test, doc]
        -> requires beaglevote, sphinx, nose

Two feature names `test` and `doc` are reserved to mark dependencies that
are needed for running automated tests and generating documentation,
respectively.

It is legal to specify `Provides-Extra` without referencing it in any
`Requires-Dist`. It is an error to request a feature name that has
not been declared with `Provides-Extra`.


Extension (multiple use)
::::::::::::::::::::::::

An ASCII string, not containing whitespace or the / character, that
indicates the presence of extended metadata. Additional tags defined by
an `Extension: Chili` must be of the form `Chili/Name`::

    Extension: Chili
    Chili/Type: Poblano
    Chili/Heat: Mild

An implementation might iterate over all the declared `Extension:`
fields to invoke the processors for those extensions.  As the order of
the fields is not used, the `Extension: Chili` field may appear before
or after its declared tags `Chili/Type:` etc.


Version Specifiers
==================

Version specifiers are a series of conditional operators and
version numbers, separated by commas.  Conditional operators
must be one of "<", ">", "<=", ">=", "==" and "!=".

Any number of conditional operators can be specified, e.g.
the string ">1.0, !=1.3.4, <2.0" is a legal version declaration.
The comma (",") is equivalent to the **and** operator.

Each version number must be in the format specified in PEP 386.

When a version is provided, it always includes all versions that
starts with the same value. For example the "2.5" version of Python
will include versions like "2.5.2" or "2.5.3". Pre and post releases
in that case are excluded. So in our example, versions like "2.5a1" are
not included when "2.5" is used. If the first version of the range is
required, it has to be explicitly given. In our example, it will be
"2.5.0".

Notice that some projects might omit the ".0" prefix for the first release
of the "2.5.x" series:

- 2.5
- 2.5.1
- 2.5.2
- etc.

In that case, "2.5.0" will have to be explicitly used to avoid any confusion
between the "2.5" notation that represents the full range. It is a recommended
practice to use schemes of the same length for a series to completely avoid
this problem.

Some Examples:

- ``Requires-Dist: zope.interface (3.1)``: any version that starts with 3.1,
  excluding post or pre-releases.
- ``Requires-Dist: zope.interface (==3.1)``: equivalent to ``Requires-Dist:
  zope.interface (3.1)``.
- ``Requires-Dist: zope.interface (3.1.0)``: any version that starts with
  3.1.0, excluding post or pre-releases. Since that particular project doesn't
  use more than 3 digits, it also means "only the 3.1.0 release".
- ``Requires-Python: 3``: Any Python 3 version, no matter wich one, excluding
  post or pre-releases.
- ``Requires-Python: >=2.6,<3``: Any version of Python 2.6 or 2.7, including
  post releases of 2.6, pre and post releases of 2.7. It excludes pre releases
  of Python 3.
- ``Requires-Python: 2.6.2``: Equivalent to ">=2.6.2,<2.6.3". So this includes
  only Python 2.6.2. Of course, if Python was numbered with 4 digits, it would
  have include all versions of the 2.6.2 series.
- ``Requires-Python: 2.5.0``: Equivalent to ">=2.5.0,<2.5.1".
- ``Requires-Dist: zope.interface (3.1,!=3.1.3)``: any version that starts with
  3.1, excluding post or pre-releases of 3.1 *and* excluding any version that
  starts with "3.1.3". For this particular project, this means: "any version
  of the 3.1 series but not 3.1.3". This is equivalent to:
  ">=3.1,!=3.1.3,<3.2".

Environment markers
===================

An **environment marker** is a marker that can be added at the end of a
field after a semi-colon (";"), to add a condition about the execution
environment.

Here are some example of fields using such markers::

   Requires-Dist: pywin32 (>1.0); sys.platform == 'win32'
   Obsoletes-Dist: pywin31; sys.platform == 'win32'
   Requires-Dist: foo (1,!=1.3); platform.machine == 'i386'
   Requires-Dist: bar; python_version == '2.4' or python_version == '2.5'
   Requires-External: libxslt; 'linux' in sys.platform

The micro-language behind this is a simple subset of Python: it compares
only strings, with the ``==`` and ``in`` operators (and their opposites),
and with the ability to combine expressions. Parenthesis are supported
for grouping.

The pseudo-grammar is ::

    EXPR [in|==|!=|not in]?EXPR [or|and] ...

where ``EXPR`` belongs to any of those:

- python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1])
- python_full_version = sys.version.split()[0]
- os.name = os.name
- sys.platform = sys.platform
- platform.version = platform.version()
- platform.machine = platform.machine()
- platform.python_implementation = platform.python_implementation()
- a free string, like ``'2.4'``, or ``'win32'``
- extra = (name of requested feature) or None

Notice that ``in`` is restricted to strings, meaning that it is not possible
to use other sequences like tuples or lists on the right side.

The fields that benefit from this marker are:

- Requires-Python
- Requires-External
- Requires-Dist
- Setup-Requires-Dist
- Provides-Dist
- Obsoletes-Dist
- Classifier

(The `extra` variable is only meaningful for Requires-Dist.)

Summary of Differences From PEP 345
===================================

* Metadata-Version is now 1.3.

* Values are now expected to be UTF-8.

* A payload (containing the description) may appear after the headers.

* Added `extra` to environment markers.

* Most fields are now optional.

* Changed fields:

  - Requires-Dist
  - Description

* Added fields:

  - Setup-Requires-Dist
  - Provides-Extra
  - Extension

References
==========

This document specifies version 1.3 of the metadata format.
Version 1.0 is specified in PEP 241.
Version 1.1 is specified in PEP 314.
Version 1.2 is specified in PEP 345.

.. [1] reStructuredText markup:
   http://docutils.sourceforge.net/

.. _`Python Package Index`: http://pypi.python.org/pypi/

.. [2] PEP 301:
   http://www.python.org/dev/peps/pep-0301/

Appendix
========

Parsing and generating the Metadata 1.3 serialization format using
Python 3.3::

    # Metadata 1.3 demo
    from email.generator import Generator
    from email import header
    from email.parser import Parser
    from email.policy import Compat32
    from email.utils import _has_surrogates

    class MetadataPolicy(Compat32):
        max_line_length = 0
        continuation_whitespace = '\t'

        def _sanitize_header(self, name, value):
            if not isinstance(value, str):
                return value
            if _has_surrogates(value):
                raise NotImplementedError()
            else:
                return value

        def _fold(self, name, value, sanitize):
            body = ((self.linesep+self.continuation_whitespace)
                    .join(value.splitlines()))
            return ''.join((name, ': ', body, self.linesep))

    if __name__ == "__main__":
        import sys
        import textwrap

        pkg_info = """\
    Metadata-Version: 1.3
    Name: package
    Version: 0.1.0
    Summary: A package.
    Description: Description
        ===========


        A description of the package.

    """

        m = Parser(policy=MetadataPolicy()).parsestr(pkg_info)

        m['License'] = 'GPL'
        description = m['Description']
        description_lines = description.splitlines()
        m.set_payload(description_lines[0]
                + '\n'
                + textwrap.dedent('\n'.join(description_lines[1:]))
                + '\n')
        del m['Description']

        # Correct if sys.stdout.encoding == 'UTF-8':
        Generator(sys.stdout, maxheaderlen=0).flatten(m)

Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:

From cs at zip.com.au  Sun Oct 28 23:00:21 2012
From: cs at zip.com.au (Cameron Simpson)
Date: Mon, 29 Oct 2012 09:00:21 +1100
Subject: [Python-Dev] configure produces a non-working Makefile in some
 scenarios, due to ASDLGEN
In-Reply-To: <loom.20121028T191937-98@post.gmane.org>
References: <loom.20121028T191937-98@post.gmane.org>
Message-ID: <20121028220021.GA11541@cskk.homeip.net>

On 28Oct2012 18:24, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
| In some scenarios, configure produces a Makefile which fails because ASDLGEN
| doesn't point to a working Python. In particular, it seems to assume that if
| there is an executable called e.g. "python3.4" on the path, then that will be a
| system Python.

Or at least a valid python. Given that python installs create a pythonA.B
executable/symlink I would call that a reasonable assumption.

If I invoke the command "python2.7", I expect to get a python 2.7.x
instance. Indeed, I have scripts that expect that (eg my py26+ script
which looks for a sufficiently new python for particular tasks).

| In my perhaps unusual but IMO perfectly valid setup, I have various Python repos
| set up like so:
| 
| $HOME/projects/python
|                  |
|                  +- default
|                  |
|                  +- 3.3
|                  |
|                  + and so on for 3.2, 3.1, 2.7

That's sane.

| In order to facilitate testing some script against multiple Python versions, I
| have shell scripts in $HOME/bin named python3.4, python3.3 etc. which just run
| the programs in the relevant directories in $HOME/projects/python/. (I know I
| can do this using aliases etc., but I think that's beside the point.)

That is not sane. And aliases aren't quite beside the point, though
nearly so. At least aliases aren't used by shell scripts on the whole,
which executables in the $PATH are.

A python install includes a pythonA.B executable. You're providing
"pythonA.B" which is _not_ a python executable!

If you had a shell script in your ~/bin called "ls" that ran "rm", would
you consider it sensible? I wouldn't.

I'm sure you're well aware of the newbie UNIX mistake of calling a test
script "test", too:-) Same issue, really.

| When I run configure in the repo for the default branch, it appears to look for
| a Python on the path named python3.4, finds one in $HOME/bin, and thus generates
| an ASDLGEN value of "python3.4". If I happen to have a built Python in the
| default repo, then the script in $HOME/bin will successfully run that, and all
| appears well; but if I clean the default repo and re-run make, it fails at the
| ASDLGEN step, because the $HOME/bin/python3.4 script fails, due to not being
| able to invoke $HOME/projects/python/default/python.
| 
| Is this a bug in configure, or is my configuration regarded as too perverse to
| support?

I agree with Antoine; it is slightly perverse. But entirely because of the
executable names you've chosen.

When I run configure I take care to _trash_ my $PATH. Well, to make it
more conventional. Configure relies heavily on $PATH (and a host of
other ill-documented variables, like $ARCH or $CPU, I forget now; I have
wrapper scripts to toss tis stuff and hand me a fresh shell for exactly
this issue).

I have long found it useful to craft my environment before using
configure. Specificly, the $PATH _must_ find the expected executable
names appropriately. I your case it will not.

I would have test scripts do something of this flavour:

  PATH={your-test-area-here}/bin:/usr/bin:/bin:/usr/local/bin
  export PATH

to ensure that configure finds what it expects.

I would also rename your ~/bin/python3.4 et al scripts to something more
descriptive; I have over the past several years been naming my own
scripts progressively more verbosely. Maybe "test-python-3.4" and so
forth. And make some convenience aliases (_not_ conflicting with
"standard" names like "python3.4", maybe "testpy34" or something equally
short). And having made those convenience aliases, possibly even move
the test wrapper scripts our of my ~/bin depending on how widely that
get used by other higher level scripts.

Hoping this is more helpful than I've probably made it sound,
-- 
Cameron Simpson <cs at zip.com.au>

Nothing is impossible for the man who doesn't have to do it.

From nad at acm.org  Mon Oct 29 00:19:36 2012
From: nad at acm.org (Ned Deily)
Date: Sun, 28 Oct 2012 16:19:36 -0700
Subject: [Python-Dev] configure produces a non-working Makefile in some
	scenarios, due to ASDLGEN
References: <loom.20121028T191937-98@post.gmane.org>
	<20121028194043.09415a50@pitrou.net>
Message-ID: <nad-335FDB.16193628102012@news.gmane.org>

In article <20121028194043.09415a50 at pitrou.net>,
 Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sun, 28 Oct 2012 18:24:36 +0000 (UTC)
> Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> > In some scenarios, configure produces a Makefile which fails because 
> > ASDLGEN
> > doesn't point to a working Python. In particular, it seems to assume that 
> > if
> > there is an executable called e.g. "python3.4" on the path, then that will 
> > be a
> > system Python.
> > 
> > In my perhaps unusual but IMO perfectly valid setup, I have various Python 
> > repos
> > set up like so:
> > 
> > $HOME/projects/python
> >                  |
> >                  +- default
> >                  |
> >                  +- 3.3
> >                  |
> >                  + and so on for 3.2, 3.1, 2.7
> > 
> > In order to facilitate testing some script against multiple Python 
> > versions, I
> > have shell scripts in $HOME/bin named python3.4, python3.3 etc. which just 
> > run
> > the programs in the relevant directories in $HOME/projects/python/. (I know 
> > I
> > can do this using aliases etc., but I think that's beside the point.)
> > 
> > When I run configure in the repo for the default branch, it appears to look 
> > for
> > a Python on the path named python3.4, finds one in $HOME/bin, and thus 
> > generates
> > an ASDLGEN value of "python3.4". If I happen to have a built Python in the
> > default repo, then the script in $HOME/bin will successfully run that, and 
> > all
> > appears well; but if I clean the default repo and re-run make, it fails at 
> > the
> > ASDLGEN step, because the $HOME/bin/python3.4 script fails, due to not 
> > being
> > able to invoke $HOME/projects/python/default/python.
> > 
> > Is this a bug in configure, or is my configuration regarded as too perverse 
> > to
> > support?
> 
> The latter IMO :)

Not at all, IMO :)

This behavior is a change recently introduced by the changesets 
associated with http://bugs.python.org/issue16262.  Previously, the 
first `python` on $PATH was used if asdl_c.py needed to be run in the 
Makefile.  You now have the opportunity to override the behavior at 
./configure time by using the PYTHON variable to specify the path to the 
Python executable you want to use for the asdl_c.py step, as in:

   ./configure ... PYTHON=/usr/bin/python2.7

-- 
 Ned Deily,
 nad at acm.org


From brian at python.org  Mon Oct 29 05:05:24 2012
From: brian at python.org (Brian Curtin)
Date: Sun, 28 Oct 2012 23:05:24 -0500
Subject: [Python-Dev] Acceptance of PEPs
Message-ID: <CAD+XWwrHnbByio0ENrsrHCYwAzSw6RjhEm=W4BwNLtRCFbH+Wg@mail.gmail.com>

I just found out that PEP 430 came up again and was approved, but only
because of another website. After that I looked back through my email
only to find mention via python-checkins. Aren't PEPs typically acted
on here on python-dev?

I mention this because I was writing up a blog.python.org post about
the (somewhat recent) Sphinx theme change and the change to add a
version dropdown which was pushed yesterday. I was about to wrap it up
and schedule it for tomorrow when I found out about 430, which I'm now
including.


note: I have no problems with that PEP actually being approved. I've
wanted that change for 2 years.

From chris.jerdonek at gmail.com  Mon Oct 29 07:05:52 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sun, 28 Oct 2012 23:05:52 -0700
Subject: [Python-Dev] Acceptance of PEPs
In-Reply-To: <CAD+XWwrHnbByio0ENrsrHCYwAzSw6RjhEm=W4BwNLtRCFbH+Wg@mail.gmail.com>
References: <CAD+XWwrHnbByio0ENrsrHCYwAzSw6RjhEm=W4BwNLtRCFbH+Wg@mail.gmail.com>
Message-ID: <CAOTb1wcMU_9erENifAPxWoBzgZ0UqF9hGFcFSoe6v4d8weBRLw@mail.gmail.com>

On Sun, Oct 28, 2012 at 9:05 PM, Brian Curtin <brian at python.org> wrote:
> I just found out that PEP 430 came up again and was approved, but only
> because of another website. After that I looked back through my email
> only to find mention via python-checkins. Aren't PEPs typically acted
> on here on python-dev?

FWIW, the following e-mail was cross-posted to python-dev:

http://mail.python.org/pipermail/python-ideas/2012-October/017494.html

The message doesn't seem to show up in the python-dev archives though:

http://mail.python.org/pipermail/python-dev/2012-October/thread.html

Perhaps this is because python-dev was the "CC" rather than the "To"
(which was python-ideas).  Here is another message that does show up
in the python-dev archives (and which quotes the acceptance):

http://mail.python.org/pipermail/python-dev/2012-October/122458.html

--Chris

From g.brandl at gmx.net  Mon Oct 29 07:18:24 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 29 Oct 2012 07:18:24 +0100
Subject: [Python-Dev] Acceptance of PEPs
In-Reply-To: <CAD+XWwrHnbByio0ENrsrHCYwAzSw6RjhEm=W4BwNLtRCFbH+Wg@mail.gmail.com>
References: <CAD+XWwrHnbByio0ENrsrHCYwAzSw6RjhEm=W4BwNLtRCFbH+Wg@mail.gmail.com>
Message-ID: <k6l73e$4cg$1@ger.gmane.org>

Am 29.10.2012 05:05, schrieb Brian Curtin:
> I just found out that PEP 430 came up again and was approved, but only
> because of another website. After that I looked back through my email
> only to find mention via python-checkins. Aren't PEPs typically acted
> on here on python-dev?

Yes, that's why I cross-posted the acceptance email here.

> I mention this because I was writing up a blog.python.org post about
> the (somewhat recent) Sphinx theme change and the change to add a
> version dropdown which was pushed yesterday. I was about to wrap it up
> and schedule it for tomorrow when I found out about 430, which I'm now
> including.
> 
> 
> note: I have no problems with that PEP actually being approved. I've
> wanted that change for 2 years.

For most PEPs, a longer contemplation and discussion period is of course
required. But in this case I don't think a PEP would have been required
anyway, and it matched more or less what was already discussed this time
*and* the last time this came up (at which time I didn't act on it).  So
it seemed the best course of action to do it right now.

Georg




From chris.jerdonek at gmail.com  Mon Oct 29 07:44:54 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sun, 28 Oct 2012 23:44:54 -0700
Subject: [Python-Dev] [Python-checkins] peps: PEP for updating the URL
	layout on docs.python.org
In-Reply-To: <3Xpl5B1cv9zNm1@mail.python.org>
References: <3Xpl5B1cv9zNm1@mail.python.org>
Message-ID: <CAOTb1wdhQbg_+mw=qx31dFh9U1TLf19omzOi3_9XSTH4qKMpZQ@mail.gmail.com>

On Sat, Oct 27, 2012 at 7:36 AM, nick.coghlan
<python-checkins at python.org> wrote:
> http://hg.python.org/peps/rev/c7ba002ca91d
> changeset:   4568:c7ba002ca91d
> user:        Nick Coghlan <ncoghlan at gmail.com>
> date:        Sun Oct 28 00:36:36 2012 +1000
> summary:
>   PEP for updating the URL layout on docs.python.org
> ...
> +Implementation
> +==============
> +
> +The URLs on docs.python.org are controlled by the python.org infrastructure
> +team rather than through the CPython source repo, so acceptance and
> +implementation of the ideas in this PEP will be up to the team.

Related to this, just checking to see if anyone is taking
responsibility for following up with http://www.python.org.  I imagine
they have several links that may no longer point to the correct
version (e.g. the "Documentation" link under "Quick Links (2.7.3)" in
the left column of the home page now results in
http://docs.python.org/3/, and similarly for 2.x on
http://www.python.org/doc/ ).

--Chris

From barry at python.org  Mon Oct 29 07:52:42 2012
From: barry at python.org (Barry Warsaw)
Date: Mon, 29 Oct 2012 07:52:42 +0100
Subject: [Python-Dev] Acceptance of PEPs
In-Reply-To: <CAD+XWwrHnbByio0ENrsrHCYwAzSw6RjhEm=W4BwNLtRCFbH+Wg@mail.gmail.com>
References: <CAD+XWwrHnbByio0ENrsrHCYwAzSw6RjhEm=W4BwNLtRCFbH+Wg@mail.gmail.com>
Message-ID: <20121029075242.073173f7@resist>

On Oct 28, 2012, at 11:05 PM, Brian Curtin wrote:

>I just found out that PEP 430 came up again and was approved, but only
>because of another website. After that I looked back through my email
>only to find mention via python-checkins. Aren't PEPs typically acted
>on here on python-dev?
>
>I mention this because I was writing up a blog.python.org post about
>the (somewhat recent) Sphinx theme change and the change to add a
>version dropdown which was pushed yesterday. I was about to wrap it up
>and schedule it for tomorrow when I found out about 430, which I'm now
>including.

PEP 1 states that proposals must be sent to python-dev after discussion on
python-ideas.  As for acceptance, PEP 1 says:

    Once the authors have completed a PEP, they may request a review for
    style and consistency from the PEP editors.  However, the content and
    final acceptance of the PEP must be requested of the BDFL, usually via
    an email to the python-dev mailing list.

    ...

    When a PEP is Accepted, Rejected or Withdrawn, the PEP should be updated
    accordingly. In addition to updating the status field, at the very least
    the Resolution header should be added with a link to the relevant post
    in the python-dev mailing list archives.
    
So yes, python-dev needs to be the place where PEPs are discussed and
accepted.

Cheers,
-Barry

From vinay_sajip at yahoo.co.uk  Mon Oct 29 10:34:42 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 29 Oct 2012 09:34:42 +0000 (UTC)
Subject: [Python-Dev] configure produces a non-working Makefile in some
	scenarios, due to ASDLGEN
References: <loom.20121028T191937-98@post.gmane.org>
	<20121028194043.09415a50@pitrou.net>
	<nad-335FDB.16193628102012@news.gmane.org>
Message-ID: <loom.20121029T103305-128@post.gmane.org>

Ned Deily <nad <at> acm.org> writes:

> Makefile.  You now have the opportunity to override the behavior at 
> ./configure time by using the PYTHON variable to specify the path to the 
> Python executable you want to use for the asdl_c.py step, as in:
> 
>    ./configure ... PYTHON=/usr/bin/python2.7

Aha! Thanks for pointing that out.

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Mon Oct 29 10:40:30 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Mon, 29 Oct 2012 09:40:30 +0000 (UTC)
Subject: [Python-Dev] configure produces a non-working Makefile in some
	scenarios, due to ASDLGEN
References: <loom.20121028T191937-98@post.gmane.org>
	<20121028220021.GA11541@cskk.homeip.net>
Message-ID: <loom.20121029T103513-71@post.gmane.org>

Cameron Simpson <cs <at> zip.com.au> writes:

> Hoping this is more helpful than I've probably made it sound,

Sure. I know my setup was somewhat unusual, which is why I posted here rather
than the tracker, but it was a surprise when a long-working configuration
suddenly broke because of an unbuilt Python 3.4. Ned's post showed me why.

Regards,

Vinay Sajip



From brett at python.org  Mon Oct 29 14:01:38 2012
From: brett at python.org (Brett Cannon)
Date: Mon, 29 Oct 2012 09:01:38 -0400
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CAN8CLgmw041P8V6SFa6uJfVC5dvx31j890rGGYa7x6yOF3-qEw@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAN8CLgkttfmgJkt2E1H5TMkkbWwPza6Mv757AuWN6cppHV3XyQ@mail.gmail.com>
	<k6imeu$rk1$1@ger.gmane.org>
	<CAN8CLgmw041P8V6SFa6uJfVC5dvx31j890rGGYa7x6yOF3-qEw@mail.gmail.com>
Message-ID: <CAP1=2W6ipQ+VTxWZ8p07ae-V_L68d2j=_PZibanucDHefay_UA@mail.gmail.com>

On Sun, Oct 28, 2012 at 3:48 PM, Tim Delaney <timothy.c.delaney at gmail.com>wrote:

> On 28 October 2012 18:22, Stefan Behnel <stefan_ml at behnel.de> wrote:
>
>> How much of an
>>
>>> effect would it have on startup times and these benchmarks if
>>> Cython-compiled extensions were used?
>>>
>>
>> Depends on what and how much code you use. If you compile everything into
>> one big module that "imports" all of the stdlib when it gets loaded, you'd
>> likely loose a lot of time because it would take a while to initialise all
>> that useless code on startup. If you keep it separate, it would likely be a
>> lot faster because you avoid the interpreter for most of the module startup.
>>
>
> I was specifically thinking in terms of the tests Brett ran (that was the
> full set on speed.python.org, wasn't it?),
>

It's not the full set as not all of them can be run on Python 3, but it is
as many as can be run.

 -Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121029/574c879f/attachment.html>

From brett at python.org  Mon Oct 29 14:56:57 2012
From: brett at python.org (Brett Cannon)
Date: Mon, 29 Oct 2012 09:56:57 -0400
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
Message-ID: <CAP1=2W4pL0f=JkoqKtVJYMEcO0F3SfE+szHqMO-q=Rs+ECenDA@mail.gmail.com>

To see if the bad iterative_count and threaded_count results were
consistently bad, I ran the benchmark suite on my MacBook Pro to see how
"reliable" the benchmarks were. The output is below.

Basically 6 benchmarks (regex_effbot, queens, startup_nosite,
iterative_count, threaded_count, and telco) had a variance of more than 15%
performance between my 2 computers, although queens, iterative_count, and
threaded_count were the only ones that swung between neutral/good to bad
depending on the machine (the rest either want from bad to very bad, or
very good to more very good).

And before Antoine asks, I added a ``sys.modules['markupsafe'] = None` line
to the mako_v2 benchmark locally. =) Still need to either explicitly block
it or emit a warning in the code in the repo.


#########################################


Report on Darwin Darwin Kernel Version 12.2.0: Sat Aug 25 00:48:52 PDT
2012; root:xnu-2050.18.24~1/RELEASE_X86_64 x86_64 i386
Total CPU cores: 8

### 2to3 ###
10.321463 -> 9.525119: 1.08x faster

### call_method ###
Min: 0.466812 -> 0.417812: 1.12x faster
Avg: 0.483324 -> 0.427158: 1.13x faster
Significant (t=28.77)
Stddev: 0.01876 -> 0.01483: 1.2644x smaller
Timeline: b'http://tinyurl.com/8al5lmm'

### call_method_slots ###
Min: 0.484923 -> 0.409452: 1.18x faster
Avg: 0.487877 -> 0.413054: 1.18x faster
Significant (t=131.11)
Stddev: 0.00395 -> 0.00577: 1.4589x larger
Timeline: b'http://tinyurl.com/9zhpg6z'

### call_method_unknown ###
Min: 0.547050 -> 0.406866: 1.34x faster
Avg: 0.550721 -> 0.409359: 1.35x faster
Significant (t=328.32)
Stddev: 0.00415 -> 0.00325: 1.2795x smaller
Timeline: b'http://tinyurl.com/9wxoddz'

### call_simple ###
Min: 0.391213 -> 0.332055: 1.18x faster
Avg: 0.393563 -> 0.335362: 1.17x faster
Significant (t=127.15)
Stddev: 0.00363 -> 0.00427: 1.1764x larger
Timeline: b'http://tinyurl.com/8mmepzw'

### chameleon ###
Min: 0.078505 -> 0.070175: 1.12x faster
Avg: 0.083754 -> 0.071500: 1.17x faster
Significant (t=2.95)
Stddev: 0.05086 -> 0.00119: 42.8425x smaller
Timeline: b'http://tinyurl.com/8bz9hpl'

### chaos ###
Min: 0.353739 -> 0.423587: 1.20x slower
Avg: 0.356297 -> 0.428197: 1.20x slower
Significant (t=-108.44)
Stddev: 0.00200 -> 0.00424: 2.1147x larger
Timeline: b'http://tinyurl.com/98e56le'

### django ###
Min: 0.824149 -> 0.862750: 1.05x slower
Avg: 0.831614 -> 0.869112: 1.05x slower
Significant (t=-21.47)
Stddev: 0.01020 -> 0.00697: 1.4634x smaller
Timeline: b'http://tinyurl.com/8kz8owv'

### fannkuch ###
Min: 1.776913 -> 1.832973: 1.03x slower
Avg: 1.793116 -> 1.915348: 1.07x slower
Significant (t=-11.57)
Stddev: 0.01436 -> 0.07329: 5.1030x larger
Timeline: b'http://tinyurl.com/9ptae4z'

### fastpickle ###
Min: 0.810968 -> 0.739322: 1.10x faster
Avg: 0.818099 -> 0.745148: 1.10x faster
Significant (t=58.02)
Stddev: 0.00577 -> 0.00677: 1.1731x larger
Timeline: b'http://tinyurl.com/8l769dd'

### fastunpickle ###
Min: 0.644198 -> 0.659345: 1.02x slower
Avg: 0.647976 -> 0.666154: 1.03x slower
Significant (t=-18.96)
Stddev: 0.00343 -> 0.00584: 1.7020x larger
Timeline: b'http://tinyurl.com/93xn7el'

### float ###
Min: 0.420888 -> 0.363410: 1.16x faster
Avg: 0.432285 -> 0.376179: 1.15x faster
Significant (t=38.14)
Stddev: 0.00762 -> 0.00708: 1.0766x smaller
Timeline: b'http://tinyurl.com/8bjwka9'

### formatted_logging ###
Min: 0.325707 -> 0.413196: 1.27x slower
Avg: 0.329846 -> 0.418099: 1.27x slower
Significant (t=-119.89)
Stddev: 0.00397 -> 0.00337: 1.1787x smaller
Timeline: b'http://tinyurl.com/8ktbs49'

### genshi ###
Min: 0.254604 -> 0.269696: 1.06x slower
Avg: 0.258585 -> 0.275615: 1.07x slower
Significant (t=-33.39)
Stddev: 0.00283 -> 0.00557: 1.9704x larger
Timeline: b'http://tinyurl.com/8bqvcwl'

### go ###
Min: 0.676453 -> 0.745504: 1.10x slower
Avg: 0.681833 -> 0.752170: 1.10x slower
Significant (t=-48.67)
Stddev: 0.00520 -> 0.00880: 1.6917x larger
Timeline: b'http://tinyurl.com/9d6qj3y'

### hexiom2 ###
Min: 186.378727 -> 172.939507: 1.08x faster
Avg: 186.679821 -> 173.103242: 1.08x faster
Significant (t=39.61)
Stddev: 0.42581 -> 0.23156: 1.8389x smaller
Timeline: b'http://tinyurl.com/9mc3pmg'

### html5lib ###
Min: 11.827770 -> 11.239556: 1.05x faster
Avg: 11.858253 -> 11.370960: 1.04x faster
Significant (t=6.93)
Stddev: 0.02825 -> 0.15466: 5.4746x larger
Timeline: b'http://tinyurl.com/8vl952y'

### iterative_count ###
Min: 0.168182 -> 0.154105: 1.09x faster
Avg: 0.169512 -> 0.155952: 1.09x faster
Significant (t=50.77)
Stddev: 0.00139 -> 0.00128: 1.0899x smaller
Timeline: b'http://tinyurl.com/9eymjtf'

### json_dump_v2 ###
Min: 3.350528 -> 3.795307: 1.13x slower
Avg: 3.369661 -> 3.825400: 1.14x slower
Significant (t=-125.93)
Stddev: 0.01470 -> 0.02095: 1.4250x larger
Timeline: b'http://tinyurl.com/8wyn9qa'

### json_load ###
Min: 0.999717 -> 0.607549: 1.65x faster
Avg: 1.007319 -> 0.613016: 1.64x faster
Significant (t=289.24)
Stddev: 0.00673 -> 0.00690: 1.0240x larger
Timeline: b'http://tinyurl.com/8qxakdw'

### mako_v2 ###
Min: 0.094817 -> 0.279593: 2.95x slower
Avg: 0.096962 -> 0.286479: 2.95x slower
Significant (t=-866.63)
Stddev: 0.00182 -> 0.00454: 2.4945x larger
Timeline: b'http://tinyurl.com/9lufgwz'

### meteor_contest ###
Min: 0.276138 -> 0.243228: 1.14x faster
Avg: 0.279559 -> 0.246018: 1.14x faster
Significant (t=72.30)
Stddev: 0.00298 -> 0.00136: 2.1943x smaller
Timeline: b'http://tinyurl.com/8pj9dnc'

### nbody ###
Min: 0.421698 -> 0.320496: 1.32x faster
Avg: 0.425878 -> 0.323483: 1.32x faster
Significant (t=158.15)
Stddev: 0.00386 -> 0.00247: 1.5638x smaller
Timeline: b'http://tinyurl.com/9fy8dfg'

### normal_startup ###
Min: 0.612120 -> 0.876470: 1.43x slower
Avg: 0.618945 -> 0.885492: 1.43x slower
Significant (t=-280.36)
Stddev: 0.00422 -> 0.00523: 1.2397x larger
Timeline: b'http://tinyurl.com/98ap93d'

### nqueens ###
Min: 0.402125 -> 0.410580: 1.02x slower
Avg: 0.406403 -> 0.414676: 1.02x slower
Significant (t=-12.06)
Stddev: 0.00442 -> 0.00199: 2.2189x smaller
Timeline: b'http://tinyurl.com/8wd3lez'

### pathlib ###
Min: 0.132423 -> 0.164525: 1.24x slower
Avg: 0.136298 -> 0.168843: 1.24x slower
Significant (t=-49.05)
Stddev: 0.00763 -> 0.00720: 1.0586x smaller
Timeline: b'http://tinyurl.com/9o86dc5'

### pidigits ###
Min: 0.387690 -> 0.367871: 1.05x faster
Avg: 0.391308 -> 0.371194: 1.05x faster
Significant (t=32.69)
Stddev: 0.00369 -> 0.00230: 1.6066x smaller
Timeline: b'http://tinyurl.com/9med7ko'

### raytrace ###
Min: 1.650066 -> 1.808829: 1.10x slower
Avg: 1.660110 -> 1.832654: 1.10x slower
Significant (t=-25.26)
Stddev: 0.01165 -> 0.04687: 4.0224x larger
Timeline: b'http://tinyurl.com/8fmyhex'

### regex_compile ###
Min: 0.559449 -> 0.571906: 1.02x slower
Avg: 0.563738 -> 0.580054: 1.03x slower
Significant (t=-8.38)
Stddev: 0.00434 -> 0.01306: 3.0087x larger
Timeline: b'http://tinyurl.com/8g6xcmd'

### regex_effbot ###
Min: 0.074999 -> 0.097456: 1.30x slower
Avg: 0.076343 -> 0.099435: 1.30x slower
Significant (t=-39.79)
Stddev: 0.00147 -> 0.00383: 2.5994x larger
Timeline: b'http://tinyurl.com/9vfaeux'

### regex_v8 ###
Min: 0.087433 -> 0.104053: 1.19x slower
Avg: 0.088804 -> 0.105520: 1.19x slower
Significant (t=-39.48)
Stddev: 0.00115 -> 0.00277: 2.4122x larger
Timeline: b'http://tinyurl.com/8un7vfr'

### richards ###
Min: 0.247208 -> 0.222483: 1.11x faster
Avg: 0.251661 -> 0.225276: 1.12x faster
Significant (t=44.04)
Stddev: 0.00392 -> 0.00161: 2.4275x smaller
Timeline: b'http://tinyurl.com/8b2zv34'

### silent_logging ###
Min: 0.099170 -> 0.095099: 1.04x faster
Avg: 0.099713 -> 0.095892: 1.04x faster
Significant (t=33.32)
Stddev: 0.00045 -> 0.00068: 1.5062x larger
Timeline: b'http://tinyurl.com/9arurw6'

### simple_logging ###
Min: 0.316639 -> 0.392833: 1.24x slower
Avg: 0.320059 -> 0.396853: 1.24x slower
Significant (t=-120.31)
Stddev: 0.00224 -> 0.00392: 1.7450x larger
Timeline: b'http://tinyurl.com/95bfxu7'

### spectral_norm ###
Min: 0.434691 -> 0.379294: 1.15x faster
Avg: 0.437958 -> 0.383761: 1.14x faster
Significant (t=67.75)
Stddev: 0.00410 -> 0.00390: 1.0502x smaller
Timeline: b'http://tinyurl.com/98s9c56'

### startup_nosite ###
Min: 0.209685 -> 0.660867: 3.15x slower
Avg: 0.218654 -> 0.673249: 3.08x slower
Significant (t=-458.50)
Stddev: 0.00646 -> 0.00752: 1.1645x larger
Timeline: b'http://tinyurl.com/9zyerhn'

### telco ###
Min: 0.840453 -> 0.018312: 45.90x faster
Avg: 0.844250 -> 0.019255: 43.85x faster
Significant (t=1088.45)
Stddev: 0.00521 -> 0.00127: 4.0959x smaller
Timeline: b'http://tinyurl.com/924mje7'

### threaded_count ###
Min: 0.197525 -> 0.151649: 1.30x faster
Avg: 0.213657 -> 0.153572: 1.39x faster
Significant (t=52.58)
Stddev: 0.00779 -> 0.00214: 3.6451x smaller
Timeline: b'http://tinyurl.com/8mrrqla'

### unpack_sequence ###
Min: 0.000060 -> 0.000052: 1.16x faster
Avg: 0.000088 -> 0.000069: 1.29x faster
Significant (t=1118.61)
Stddev: 0.00000 -> 0.00000: 1.0022x larger
Timeline: b'http://tinyurl.com/9ejrega'
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121029/940ade15/attachment.html>

From solipsis at pitrou.net  Mon Oct 29 20:22:34 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 29 Oct 2012 20:22:34 +0100
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<CAP1=2W4pL0f=JkoqKtVJYMEcO0F3SfE+szHqMO-q=Rs+ECenDA@mail.gmail.com>
Message-ID: <20121029202234.5447b860@pitrou.net>

On Mon, 29 Oct 2012 09:56:57 -0400
Brett Cannon <brett at python.org> wrote:

> To see if the bad iterative_count and threaded_count results were
> consistently bad, I ran the benchmark suite on my MacBook Pro to see how
> "reliable" the benchmarks were. The output is below.
> 
> Basically 6 benchmarks (regex_effbot, queens, startup_nosite,
> iterative_count, threaded_count, and telco) had a variance of more than 15%
> performance between my 2 computers, although queens, iterative_count, and
> threaded_count were the only ones that swung between neutral/good to bad
> depending on the machine (the rest either want from bad to very bad, or
> very good to more very good).

This is using different compilers on the 2 computers, right?

Regards

Antoine.



From brett at python.org  Mon Oct 29 21:01:18 2012
From: brett at python.org (Brett Cannon)
Date: Mon, 29 Oct 2012 16:01:18 -0400
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <20121029202234.5447b860@pitrou.net>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<CAP1=2W4pL0f=JkoqKtVJYMEcO0F3SfE+szHqMO-q=Rs+ECenDA@mail.gmail.com>
	<20121029202234.5447b860@pitrou.net>
Message-ID: <CAP1=2W6kRSQkQoZzG5pMeg_vmkZFeepWXy+7gOVxLhgx_HGx0w@mail.gmail.com>

On Mon, Oct 29, 2012 at 3:22 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Mon, 29 Oct 2012 09:56:57 -0400
> Brett Cannon <brett at python.org> wrote:
>
> > To see if the bad iterative_count and threaded_count results were
> > consistently bad, I ran the benchmark suite on my MacBook Pro to see how
> > "reliable" the benchmarks were. The output is below.
> >
> > Basically 6 benchmarks (regex_effbot, queens, startup_nosite,
> > iterative_count, threaded_count, and telco) had a variance of more than
> 15%
> > performance between my 2 computers, although queens, iterative_count, and
> > threaded_count were the only ones that swung between neutral/good to bad
> > depending on the machine (the rest either want from bad to very bad, or
> > very good to more very good).
>
> This is using different compilers on the 2 computers, right?


Yes: gcc 4.6.3 on Linux and Clang 3.1 on OS X.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121029/387a364a/attachment.html>

From merwok at netwok.org  Mon Oct 29 21:22:32 2012
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Mon, 29 Oct 2012 16:22:32 -0400
Subject: [Python-Dev] Python Bug Day this Saturday announced
Message-ID: <508EE588.5070408@netwok.org>

Hi everybody,

I just sent the announcement for the bug day to python-list (apparently
pending approval), core-mentorship and montrealpython.  Core developers
who plan on being on IRC can add themselves to the list on
http://wiki.python.org/moin/PythonBugDay so that people can connect
nicknames with people.

The list by Petri at http://piratepad.net/pyconfi-sprint-issues can
still be updated.  Otherwise we?ll fall back to the usual roundup query
for easy bugs.

Cheers!

From petri at digip.org  Tue Oct 30 08:42:36 2012
From: petri at digip.org (Petri Lehtinen)
Date: Tue, 30 Oct 2012 09:42:36 +0200
Subject: [Python-Dev] Python Bug Day this Saturday announced
In-Reply-To: <508EE588.5070408@netwok.org>
References: <508EE588.5070408@netwok.org>
Message-ID: <20121030074235.GD1349@p29>

?ric Araujo wrote:
> The list by Petri at http://piratepad.net/pyconfi-sprint-issues can
> still be updated.  Otherwise we?ll fall back to the usual roundup query
> for easy bugs.

I've removed the issues for which a patch was submitted during the
PyCon Finland sprint, and retitled it as Python Bug Day issue list.

People found the list somewhat useful during our sprint. At least it
has some starting points to look at, before starting to dig through
the easy issues list.

From stefan_ml at behnel.de  Tue Oct 30 07:47:19 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 30 Oct 2012 07:47:19 +0100
Subject: [Python-Dev] Python 3.3 vs. Python 2.7 benchmark results (again,
 but this time more solid numbers)
In-Reply-To: <CAN8CLgmw041P8V6SFa6uJfVC5dvx31j890rGGYa7x6yOF3-qEw@mail.gmail.com>
References: <CAP1=2W74+-dqx8=SOD0QBThRuGY_B9ed4wmDLq_mo0x3_1W1eg@mail.gmail.com>
	<20121027115352.35011ff9@pitrou.net>
	<CAP1=2W7UOkGDSN9su7ApSVRM07p+MLhYG_6Vgi6LzUp=1XRPVw@mail.gmail.com>
	<20121027212139.0d9d6fe2@pitrou.net> <508C46BA.8020906@hotpy.org>
	<CAN8CLgkttfmgJkt2E1H5TMkkbWwPza6Mv757AuWN6cppHV3XyQ@mail.gmail.com>
	<k6imeu$rk1$1@ger.gmane.org>
	<CAN8CLgmw041P8V6SFa6uJfVC5dvx31j890rGGYa7x6yOF3-qEw@mail.gmail.com>
Message-ID: <k6o3jh$nvq$1@ger.gmane.org>

Tim Delaney, 28.10.2012 20:48:
> On 28 October 2012 18:22, Stefan Behnel wrote:
>>> How much of an effect would it have on startup times and these
>>> benchmarks if Cython-compiled extensions were used?
>>
>> Depends on what and how much code you use. If you compile everything into
>> one big module that "imports" all of the stdlib when it gets loaded, you'd
>> likely loose a lot of time because it would take a while to initialise all
>> that useless code on startup. If you keep it separate, it would likely be a
>> lot faster because you avoid the interpreter for most of the module startup.
>
> I was specifically thinking in terms of the tests Brett ran (that was the
> full set on speed.python.org, wasn't it?), and having each stdlib module be
> its own extension i.e. no big import module. A literal 1:1 replacement
> where possible.

There's also an intermediate solution of linking the top-N modules into the 
interpreter core and leaving the rest outside, but I'd rather go for the 
straight forward approach of having separate libs first.

Compiling all that can be compiled is easy enough. I fixed up a couple of 
things in Cython (so you need the latest github master) and then ran this 
setup.py script from the Lib directory with "build_ext -i":

"""
from distutils.core import setup
from Cython.Build import cythonize
from Cython.Compiler import Options

# improve Python compatibility by allowing some broken code
Options.error_on_unknown_names = False

import sys

setup(
     name = 'stuff',
     ext_modules = cythonize(
         ["**/*.py"],
         exclude=['**/test/**/*.py', '**/tests/**/*.py',
                  '**/__init__.py',
                  'idlelib/MultiCall.py'],
         exclude_failures=True,
         language_level=sys.version_info[0],
         compiler_directives=dict(auto_cpdef=True)
         ),
)
"""

Note that the extra compiler option above disables fatal compile errors on 
unknown (usually mistyped) names of which Cython hits a couple in the 
stdlib. pylint should find them as well, they're worth fixing.

The directive at the end enables automatic module internal C calls which 
usually gives a major speed-up by allowing the C compiler to see what happens.

With the above setup, Cython compiles 612 out of 620 Python modules for me, 
excluding test modules and __init__.py files. The rest fails to compile due 
to either compiler bugs or statically detected bugs in the Python code. 
I'll look through them when I find a bit of time.

One major problem I ran into is that the new importlib bootstrap module 
crashes with a RuntimeError("maximum recursion depth exceeded while calling 
a Python object)" when it hits compiled modules with import cycles (e.g. 
shutil and tarfile, or os and posixpath). I guess that's the kind of corner 
case you get when working code gets rewritten. Worth giving Py3.2 a try in 
comparison.


>>> To be clear - I'm *not* suggesting Cython become part of the required build
>>> toolchain. But *if* the Cython-compiled extensions prove to be
>>> significantly faster I'm thinking maybe it could become a semi-supported
>>> option (e.g. a HOWTO with the caveat "it worked on this particular
>>> system").
>>
>> Sounds reasonable.
>
> I think a stdlib compile script

... see above ...

> + pre-packaged hints for the 3.3 release
> would likely help both 3.3 and Cython acceptance.

That would certainly be a cool feature. This can often be as easy as 
putting a .pxd file next to the .py file that overrides the declarations of 
functions and classes with static types.


> Putting aside my development interest and looking at it purely from the PoV
> of a Python *user*, I'd really like to see Cython on
> speed.python.org eventually (in two modes - one without hints as a
> baseline and one with
> hints).

I think the above setup.py script, with appropriately adapted glob 
patterns, should do that trick well enough for now. Certainly better and 
simpler than my initial pyximport configuration. With the obvious caveat 
that it takes a bit longer to compile everything, not just the modules that 
are actually used. But that's only an install time issue.

Stefan


From dholth at gmail.com  Wed Oct 31 13:38:08 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 31 Oct 2012 08:38:08 -0400
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <8DC03FC0-5182-4979-B219-F1FA5E9EB56E@mac.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<8DC03FC0-5182-4979-B219-F1FA5E9EB56E@mac.com>
Message-ID: <CAG8k2+5=aS6i5uF_CTK4=iqHMdzb83pP_aqOhem1USCcM5ds0g@mail.gmail.com>

On Wed, Oct 24, 2012 at 7:04 AM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
> On 18 Oct, 2012, at 19:29, Daniel Holth <dholth at gmail.com> wrote:
>
>> I'd like to submit the Wheel PEPs 425 (filename metadata), 426
>> (Metadata 1.3), and 427 (wheel itself) for acceptance. The format has
>> been stable since May and we are preparing a patch to support it in
>> pip, but we need to earn consensus before including it in the most
>> widely used installer.
>
> PEP 425:
>
> * "The version is py_version_nodot. CPython gets away with no dot, but if one is needed the underscore _ is used instead"
>
>    I don't particularly like replacing dots by underscores. That needed because you use the dot character in compressed tag sets, but why not use a comma to separate items in the compressed tag set?
>
> * "The platform tag is simply distutils.util.get_platform() with all hyphens - and periods . replaced with underscore _."
>
>    Why the replacement?  The need for replacement could be avoided by using a different separator between elements of a tag (for example "~" or "+"), and furthermore the platform tag is at a know
>    location, and hence the use of hyphens in the platform tag is harmless (use "python_tag, abi_tag, platform_tag = tag.split('-', 2)" to split the tag into its elements.
>
> * "compressed tag sets"
>
>    Using '"," instead of "." to separate elements of the tag set takes away the need to replace dots in tag elements, and seems more natural to me (you'd also use comma to separate the elements
>    when you write them down in prose or python code.

I can't get excited about changing the convention. The
hyphen-to-underscore folding and - separated file parts is the same as
what setuptools uses. _ folding for . as well in the compat tags? It
may not be beautiful but it is very unlikely to cause ambiguity.

From ronaldoussoren at mac.com  Wed Oct 31 14:29:37 2012
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 31 Oct 2012 14:29:37 +0100
Subject: [Python-Dev] accept the wheel PEPs 425, 426, 427
In-Reply-To: <CAG8k2+5=aS6i5uF_CTK4=iqHMdzb83pP_aqOhem1USCcM5ds0g@mail.gmail.com>
References: <CAG8k2+5CjZ48tjGSoykhu_30Ui5jx0eFuQNqEK3hiD_MyEFcvA@mail.gmail.com>
	<8DC03FC0-5182-4979-B219-F1FA5E9EB56E@mac.com>
	<CAG8k2+5=aS6i5uF_CTK4=iqHMdzb83pP_aqOhem1USCcM5ds0g@mail.gmail.com>
Message-ID: <1486488E-6CA0-41CE-806C-EF5F0B71E833@mac.com>


On 31 Oct, 2012, at 13:38, Daniel Holth <dholth at gmail.com> wrote:

> On Wed, Oct 24, 2012 at 7:04 AM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>> 
>> On 18 Oct, 2012, at 19:29, Daniel Holth <dholth at gmail.com> wrote:
>> 
>>> I'd like to submit the Wheel PEPs 425 (filename metadata), 426
>>> (Metadata 1.3), and 427 (wheel itself) for acceptance. The format has
>>> been stable since May and we are preparing a patch to support it in
>>> pip, but we need to earn consensus before including it in the most
>>> widely used installer.
>> 
>> PEP 425:
>> 
>> * "The version is py_version_nodot. CPython gets away with no dot, but if one is needed the underscore _ is used instead"
>> 
>>   I don't particularly like replacing dots by underscores. That needed because you use the dot character in compressed tag sets, but why not use a comma to separate items in the compressed tag set?
>> 
>> * "The platform tag is simply distutils.util.get_platform() with all hyphens - and periods . replaced with underscore _."
>> 
>>   Why the replacement?  The need for replacement could be avoided by using a different separator between elements of a tag (for example "~" or "+"), and furthermore the platform tag is at a know
>>   location, and hence the use of hyphens in the platform tag is harmless (use "python_tag, abi_tag, platform_tag = tag.split('-', 2)" to split the tag into its elements.
>> 
>> * "compressed tag sets"
>> 
>>   Using '"," instead of "." to separate elements of the tag set takes away the need to replace dots in tag elements, and seems more natural to me (you'd also use comma to separate the elements
>>   when you write them down in prose or python code.
> 
> I can't get excited about changing the convention. The
> hyphen-to-underscore folding and - separated file parts is the same as
> what setuptools uses. _ folding for . as well in the compat tags? It
> may not be beautiful but it is very unlikely to cause ambiguity.

Beauty is also important.   

BTW. Setuptools does not fold '-' to underscore in the tag: xs_image-1.0-py2.7-linux-x86_64.egg

This would be "xs_image-1.0-cp27-cp27-linux_x86_64.whl" in your scheme (a binary package for CPython 2.7, platform is Linux x86_64). Replacing characters is especially ugly on OSX, compare "xs_image-1.0-cp27-cp27-macosx_10_3_fat.whl" and "xs_image-1.0-cp27-cp27-macosx-10.3-fat.whl".

As I've written before, it is not necessary to replace characters in the platform tag because it is the last part of the filename anyway. 

Ronald




From jeremy.kloth at gmail.com  Wed Oct 31 15:47:03 2012
From: jeremy.kloth at gmail.com (Jeremy Kloth)
Date: Wed, 31 Oct 2012 08:47:03 -0600
Subject: [Python-Dev] [Python-checkins] cpython: Issue #15478: Fix
 test_os on Windows (os.chown is missing)
In-Reply-To: <3Xrqbz5mFqzRCB@mail.python.org>
References: <3Xrqbz5mFqzRCB@mail.python.org>
Message-ID: <CAGvrs3Js_vhvj3Qq4Vjv3UgvkdhMeDYkQbiuiDp3Q58nCe=ygw@mail.gmail.com>

On Tue, Oct 30, 2012 at 6:07 PM, victor.stinner
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/01cc9fb52887
> changeset:   80068:01cc9fb52887
> user:        Victor Stinner <victor.stinner at gmail.com>
> date:        Wed Oct 31 01:04:10 2012 +0100
> summary:
>   Issue #15478: Fix test_os on Windows (os.chown is missing)
>
> files:
>   Lib/test/test_os.py |  3 ++-
>   1 files changed, 2 insertions(+), 1 deletions(-)
>
>
> diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
> --- a/Lib/test/test_os.py
> +++ b/Lib/test/test_os.py
> @@ -2069,7 +2069,6 @@
>          funcs = [
>              (os.chdir,),
>              (os.chmod, 0o777),
> -            (os.chown, 0, 0),
>              (os.lchown, 0, 0),
>              (os.listdir,),
>              (os.lstat,),
> @@ -2081,6 +2080,8 @@
>              (os.truncate, 0),
>              (os.unlink,),
>          ]
> +        if hasattr(os, "chown"):
> +            funcs.append((os.chown, 0, 0))
>          if sys.platform == "win32":
>              funcs.extend((
>                  (os._getfullpathname,),

Also missing on Windows is os.lchown.

From techtonik at gmail.com  Wed Oct 31 18:21:24 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 31 Oct 2012 19:21:24 +0200
Subject: [Python-Dev] Segmentaion fault with wrongly set PYTHONPATH on
	Windows
In-Reply-To: <87r4oq3wmg.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAPkN8xJNknfKw=J8TOfmK-5mMuiS-W470fNcbObPEXBU+YCKog@mail.gmail.com>
	<CAMpsgwY42sq1YJdQiDewM7S=wdX4LNF92j8szubL_Y0-xTZBsw@mail.gmail.com>
	<CAPkN8xJgEAVLM1D8Sgoq59Z7b2W-pRngWzD4bTqa7S09PNPpsw@mail.gmail.com>
	<50857665.6010909@python.org>
	<CAPkN8xLxTAabDtk-TCF4WQBw_6KtMqoQteJui_Kg9fEnHzZSyA@mail.gmail.com>
	<87r4oq3wmg.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAPkN8x+n=BXaoUoKPD3ZtrQ8aO36=2ZAGq7MRnDAtngpZWUpxg@mail.gmail.com>

On Tue, Oct 23, 2012 at 2:39 AM, Stephen J. Turnbull <
turnbull at sk.tsukuba.ac.jp> wrote:

>
> So it shuts down abnormally.  That's what an abort means, in
> programming as in rocket launches.  Users should be scared if this
> happens; somebody really screwed up.  (Unless it's themselves, and
> then they only have themselves to blame.)
>
>  > The error message could be
>  > improved though. Right now I get:
>  >
>  > E:\>python
>  > Fatal Python error: Py_Initialize: unable to load the file system codec
>  > ImportError: No module named 'encodings'
>  >
>  > This could be improved to:
>  >
>  > Fatal Python error: Py_Initialize: unable to find module named
> 'encodings'
>  > in 'C:\'
>
> That may be an improvement.  But in that case it might be worth
> explaining where "C:\" came from (in this case PYTHONHOME, I guess?)
>

Good idea.

Fatal Python error: Py_Initialize: unable to find module named 'encodings'
in PYTHONHOME ('C:\')
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121031/4bdc9678/attachment.html>

From techtonik at gmail.com  Wed Oct 31 18:44:29 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 31 Oct 2012 19:44:29 +0200
Subject: [Python-Dev] Sign of bytes
Message-ID: <CAPkN8x+-ZbhHmsXET4ybiUU692afc8odJxbm2teGvBz0KFrMWg@mail.gmail.com>

Hi,

I wonder why Python uses signed chars for bytes
http://docs.python.org/2/library/ctypes.html#ctypes.c_byte

This is a Java thing, but Java doesn't have unsigned types at all
http://en.wikipedia.org/wiki/Criticism_of_Java#Unsigned_integer_types

Windows implements BYTE as unsigned char, and it is in the same line as
WORD, DWORD etc. The way you look at memory contents in assembly.

byte type is also unsigned in .NET platform for all languages implementes,
and also has a sbyte counterpart.

When working with bytes in decimal system it is much more convenient to
operate with strictly positive values than with -128 - 127 (or is it -127
to 128?)


-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121031/c763b890/attachment.html>

From techtonik at gmail.com  Wed Oct 31 18:54:46 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 31 Oct 2012 19:54:46 +0200
Subject: [Python-Dev] Sign of bytes
In-Reply-To: <CAPkN8x+-ZbhHmsXET4ybiUU692afc8odJxbm2teGvBz0KFrMWg@mail.gmail.com>
References: <CAPkN8x+-ZbhHmsXET4ybiUU692afc8odJxbm2teGvBz0KFrMWg@mail.gmail.com>
Message-ID: <CAPkN8xLuvXyUsvsRV0F1i67ovF4tUv7xSRSHa65TTge15sMLog@mail.gmail.com>

The thing that made me wonder is here - http://bugs.python.org/issue16376
When I inspect contents of Windows structures, I get negative values that
are not present in MSDN.

-- 
anatoly t.


On Wed, Oct 31, 2012 at 7:44 PM, anatoly techtonik <techtonik at gmail.com>wrote:

> Hi,
>
> I wonder why Python uses signed chars for bytes
> http://docs.python.org/2/library/ctypes.html#ctypes.c_byte
>
> This is a Java thing, but Java doesn't have unsigned types at all
> http://en.wikipedia.org/wiki/Criticism_of_Java#Unsigned_integer_types
>
> Windows implements BYTE as unsigned char, and it is in the same line as
> WORD, DWORD etc. The way you look at memory contents in assembly.
>
> byte type is also unsigned in .NET platform for all languages implementes,
> and also has a sbyte counterpart.
>
> When working with bytes in decimal system it is much more convenient to
> operate with strictly positive values than with -128 - 127 (or is it -127
> to 128?)
>
>
> --
> anatoly t.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121031/2f36a1b5/attachment.html>

From python-dev at masklinn.net  Wed Oct 31 19:19:07 2012
From: python-dev at masklinn.net (Xavier Morel)
Date: Wed, 31 Oct 2012 19:19:07 +0100
Subject: [Python-Dev] Sign of bytes
In-Reply-To: <CAPkN8x+-ZbhHmsXET4ybiUU692afc8odJxbm2teGvBz0KFrMWg@mail.gmail.com>
References: <CAPkN8x+-ZbhHmsXET4ybiUU692afc8odJxbm2teGvBz0KFrMWg@mail.gmail.com>
Message-ID: <46B73E1C-A8F6-47B6-8A57-C354B5EFA8A8@masklinn.net>

On 2012-10-31, at 18:44 , anatoly techtonik wrote:
> I wonder why Python uses signed chars for bytes
> http://docs.python.org/2/library/ctypes.html#ctypes.c_byte

That's not Python, that's ctypes. struct[0] has no "bytes" it uses
"char" for everything.

If I had to guess, it would be because "char" is already an unsigned
char in ctypes, "signed_char" is way too long, "schar" looks like dog
vomit and there was no point in aliasing "byte" to "char. Which left
"byte" free for "signed char".

[0] http://docs.python.org/2/library/struct.html#format-characters

From techtonik at gmail.com  Wed Oct 31 20:57:28 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 31 Oct 2012 21:57:28 +0200
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
 variable referenced before assignment"
Message-ID: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>

Here is the code:

---[cut]-----------------------------

DEBUG = []
FONT_NAMES = []

def names():
  if len(DEBUG):
    print(len(DEBUG))
  if len(FONT_NAMES):
    print(len(FONT_NAMES))
  if len(FONT_NAMES)==0:
    FONT_NAMES = "query()"

names()
---[cut]-----------------------------

Here is the output:

Traceback (most recent call last):
  File "globalocal.py", line 13, in <module>
    names()
  File "globalocal.py", line 8, in names
    if len(FONT_NAMES):
UnboundLocalError: local variable 'FONT_NAMES' referenced before assignment


As you may see there is inconsistency between handling of line 6 -
"if len(DEBUG):" and line 8 - "if len(FONT_NAMES):". This is very magical
and hard to troubleshoot. I wonder if this message can be improved with a
pointer to the concept on when global variables become local?

-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20121031/7b9af961/attachment.html>

From rdmurray at bitdance.com  Wed Oct 31 21:28:06 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 31 Oct 2012 16:28:06 -0400
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
	variable referenced before assignment"
In-Reply-To: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
References: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
Message-ID: <20121031202807.13E8B25016F@webabinitio.net>

On Wed, 31 Oct 2012 21:57:28 +0200, anatoly techtonik <techtonik at gmail.com> wrote:
> Here is the code:
> 
> ---[cut]-----------------------------
> 
> DEBUG = []
> FONT_NAMES = []
> 
> def names():
>   if len(DEBUG):
>     print(len(DEBUG))
>   if len(FONT_NAMES):
>     print(len(FONT_NAMES))
>   if len(FONT_NAMES)==0:
>     FONT_NAMES = "query()"
> 
> names()
> ---[cut]-----------------------------
> 
> Here is the output:
> 
> Traceback (most recent call last):
>   File "globalocal.py", line 13, in <module>
>     names()
>   File "globalocal.py", line 8, in names
>     if len(FONT_NAMES):
> UnboundLocalError: local variable 'FONT_NAMES' referenced before assignment
> 
> 
> As you may see there is inconsistency between handling of line 6 -
> "if len(DEBUG):" and line 8 - "if len(FONT_NAMES):". This is very magical
> and hard to troubleshoot. I wonder if this message can be improved with a
> pointer to the concept on when global variables become local?

There is no inconsistency here.  Only FONT_NAMES is assigned a value
in the local scope.  "local variable referenced before assignment" *is*
a pointer to the concept of when global variables become local...perhaps
there is a better wording, do you have a suggestion?

--David

From amauryfa at gmail.com  Wed Oct 31 21:45:50 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 31 Oct 2012 21:45:50 +0100
Subject: [Python-Dev] Sign of bytes
In-Reply-To: <CAPkN8x+-ZbhHmsXET4ybiUU692afc8odJxbm2teGvBz0KFrMWg@mail.gmail.com>
References: <CAPkN8x+-ZbhHmsXET4ybiUU692afc8odJxbm2teGvBz0KFrMWg@mail.gmail.com>
Message-ID: <CAGmFidbXAD0xkyc2i-+YE4pzRYqB2B04xFzX4mcWM4_9Jvffeg@mail.gmail.com>

2012/10/31 anatoly techtonik <techtonik at gmail.com>:
> I wonder why Python uses signed chars for bytes
> http://docs.python.org/2/library/ctypes.html#ctypes.c_byte

c_int is signed, c_uint is unsigned.
similarly c_byte is signed, and c_ubyte is unsigned.

> Windows implements BYTE as unsigned char, and it is in the same line as
> WORD, DWORD etc. The way you look at memory contents in assembly.

In this case you should use ctypes.wintypes.BYTE
... which is unfortunately defined as c_byte!
This is the bug :-(

-- 
Amaury Forgeot d'Arc

From tjreedy at udel.edu  Wed Oct 31 22:01:07 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 31 Oct 2012 17:01:07 -0400
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
 variable referenced before assignment"
In-Reply-To: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
References: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
Message-ID: <k6s3j2$7bb$1@ger.gmane.org>

This post would have been more appropriate on python-list than 
python-dev. But to answer your implied questions...

On 10/31/2012 3:57 PM, anatoly techtonik wrote:
> Here is the code:
>
> ---[cut]-----------------------------
>
> DEBUG = []
> FONT_NAMES = []

This line has nothing to do with the behavior of the function that 
follows. The error message would be the name if it were deleted.

> def names():
>    if len(DEBUG):
>      print(len(DEBUG))
>    if len(FONT_NAMES):
>      print(len(FONT_NAMES))
>    if len(FONT_NAMES)==0:
>      FONT_NAMES = "query()"

This makes FONT_NAMES a local name *everywhere* within names.

> names()
> ---[cut]-----------------------------
>
> Here is the output:
>
> Traceback (most recent call last):
>    File "globalocal.py", line 13, in <module>
>      names()
>    File "globalocal.py", line 8, in names
>      if len(FONT_NAMES):
> UnboundLocalError: local variable 'FONT_NAMES' referenced before assignment
>
>
> As you may see there is inconsistency between handling of line 6 -
> "if len(DEBUG):" and line 8 - "if len(FONT_NAMES):".

No there is not.

> This is very  magical and hard to troubleshoot.

Names (not 'variables') within a function are deterministically 
classified at compile time as local, nonlocal, or global according to 
declarations and usage *within the function*. This classification has 
nothing to do with names in other namespaces and is done independently 
of other namespaces. The rules are described in the manuals.

> I wonder if this message can be
> improved with a pointer to the concept on when global variables become
> local?

This never happens. Global names stay global (until deleted). They may 
be masked by a local name with the same spelling (as in your example), 
but that is a different issue.

-- 
Terry Jan Reedy


From victor.stinner at gmail.com  Wed Oct 31 22:51:02 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 31 Oct 2012 22:51:02 +0100
Subject: [Python-Dev] [Python-checkins] cpython: Issue #15478: Fix
 test_os on Windows (os.chown is missing)
In-Reply-To: <CAGvrs3Js_vhvj3Qq4Vjv3UgvkdhMeDYkQbiuiDp3Q58nCe=ygw@mail.gmail.com>
References: <3Xrqbz5mFqzRCB@mail.python.org>
	<CAGvrs3Js_vhvj3Qq4Vjv3UgvkdhMeDYkQbiuiDp3Q58nCe=ygw@mail.gmail.com>
Message-ID: <CAMpsgwYOxxz55EpFdkeYmAn2-CyBVgmUewqEqttuLjt_gOSaeA@mail.gmail.com>

Hi,

2012/10/31 Jeremy Kloth <jeremy.kloth at gmail.com>:
> On Tue, Oct 30, 2012 at 6:07 PM, victor.stinner
> <python-checkins at python.org> wrote:
>> http://hg.python.org/cpython/rev/01cc9fb52887
>> changeset:   80068:01cc9fb52887
>> user:        Victor Stinner <victor.stinner at gmail.com>
>> date:        Wed Oct 31 01:04:10 2012 +0100
>> summary:
>>   Issue #15478: Fix test_os on Windows (os.chown is missing)
>>
>> (...)
>
> Also missing on Windows is os.lchown.

Fixed, but there were also more errors introduced by #15478. test_os
should now pass again on Windows.

Victor

From steve at pearwood.info  Wed Oct 31 23:15:57 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 01 Nov 2012 09:15:57 +1100
Subject: [Python-Dev] Improve error message "UnboundLocalError: local
 variable referenced before assignment"
In-Reply-To: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
References: <CAPkN8x+ne38WqqqutwknQb=YRVrxtkYodorZXTg6L0hR-3kc5A@mail.gmail.com>
Message-ID: <5091A31D.4000904@pearwood.info>

On 01/11/12 06:57, anatoly techtonik wrote:

[...]
> UnboundLocalError: local variable 'FONT_NAMES' referenced before assignment
>
>
> As you may see there is inconsistency between handling of line 6 -
> "if len(DEBUG):" and line 8 - "if len(FONT_NAMES):". This is very magical
> and hard to troubleshoot.

I wouldn't call it an inconsistency, because the rules for deciding whether
something is treated as local or global is consistently applied to all
functions and variables. I would use the word "difference" instead -- there
is a difference between line 6 and line 8 because there is a difference
between DEBUG (global) and FONT_NAMES (local).

As to whether this is "magical" behaviour, I suppose in some sense it is,
but if so, it is more like a light sprinkling of pixie dust rather than
full-blown dark voodoo magic.


> I wonder if this message can be improved with a
> pointer to the concept on when global variables become local?

If you have a suggestion for an improved message, please tell us. Or raise
an issue on the bug tracker.



-- 
Steven