From brett at python.org  Sat Sep  1 00:42:01 2012
From: brett at python.org (Brett Cannon)
Date: Fri, 31 Aug 2012 18:42:01 -0400
Subject: [Python-Dev] benchmarks: Pathlib works under Python 3.
In-Reply-To: <20120831184958.737b18b2@pitrou.net>
References: <3X7lT21BBTzQMV@mail.python.org>
	<20120831184958.737b18b2@pitrou.net>
Message-ID: <CAP1=2W4gEbEManO9XwY8ftqXhAF9XXx0x96tvkOmZ65ruGG7hA@mail.gmail.com>

On Fri, Aug 31, 2012 at 12:49 PM, Antoine Pitrou <solipsis at pitrou.net>wrote:

> On Fri, 31 Aug 2012 17:52:38 +0200 (CEST)
> brett.cannon <python-checkins at python.org> wrote:
> > http://hg.python.org/benchmarks/rev/873baf08045e
> > changeset:   162:873baf08045e
> > user:        Brett Cannon <brett at python.org>
> > date:        Fri Aug 31 11:52:30 2012 -0400
> > summary:
> >   Pathlib works under Python 3.
>
> ... but therefore you shouldn't run it under 2to3 (which may pessimize
> the code).  Benchmarks with a 3.x-compatible code base are listed under
> the "2n3" meta-benchmark name in perf.py:
>
> http://hg.python.org/benchmarks/file/873baf08045e/perf.py#l2000


Right, but the code still needs to get copied over (which it wasn't). I'll
start a new entry for code that needs to be copied but not translated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120831/3ec42b8d/attachment-0001.html>

From stephen at xemacs.org  Sat Sep  1 06:55:11 2012
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 01 Sep 2012 13:55:11 +0900
Subject: [Python-Dev] Edits to Metadata 1.2 to add extras
	(optional	ependencies)
In-Reply-To: <5040D964.8030202@v.loewis.de>
References: <20120828041552.GA21422@chang>
	<87wr0jtxzy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<503CD917.5050902@v.loewis.de>
	<C4B87D5AB77C404D945F2B1E0DB97482@gmail.com>
	<503CDEA4.9090800@v.loewis.de>
	<691A785F4B8F4AE09948ADCBDD7534C2@gmail.com>
	<20120828153811.CBDD32500FE@webabinitio.net>
	<503CF398.3020606@v.loewis.de>
	<20120828171541.GA9777@iskra.aviel.ru>
	<503D0B9C.6070308@v.loewis.de>
	<20120828183807.GA12554@iskra.aviel.ru>
	<CAG8k2+6scwXX8QVx_TgqAKHphhWq=2fT9062F_Zi-kpD3-7VDg@mail.gmail.com>
	<50409690.9030109@v.loewis.de>
	<C1D864B70CE64B088D681994F630010C@gmail.com>
	<466B4750-EC6C-4773-87BF-E742652DD958@gmail.com>
	<20120831124119.9763A25010D@webabinitio.net>
	<CADiSq7fVfmmdjJP+anvT_v7kH6=xHye7BhH3XNqxFZoytpa8ZQ@mail.gmail.com>
	<5040D964.8030202@v.loewis.de>
Message-ID: <87oblqtk9c.fsf@uwakimon.sk.tsukuba.ac.jp>

"Martin v. L?wis" writes:

 > Unfortunately, this conflicts with the desire to use UTF-8 in attribute
 > values - RFC 822 (and also 2822) don't support this, but require the
 > use oF MIME instead (Q or B encoding).

This can be achieved simply by extending the set of characters
permitted, as MIME did for message bodies.  I'd be cautious about RFC
5335, not just because it's experimental, but because there may be
other requirements we don't want to mess with.  (If RDM says
otherwise, listen to him.  I just know the RFC exists.)

 > RFC 2822 also has a continuation line semantics which traditionally 
 > conflicts with the metadata; in particular, line breaks cannot be 
 > represented (but are interpreted as continuation lines instead).

Of course line breaks can be represented, without any further change
to RFC 2822.  Just use Unicode LINE SEPARATOR.  You could even do it
within ASCII by adhering strictly to RFC 2822 syntax which interprets
continuation lines by removing exactly the CRLF pair.  Just use ASCII
TAB as the field separator.

There's a final dodge that occurs to me: the semantics you're talking
about are *lexical* semantics in the RFC 2822 context (line unfolding
and RFC 2047 decoding).  We could possibly in the context of the email
module treat Metadata as an intermediate post-lexical-decoding
pre-syntactic-analysis representation.  I don't know if that makes
sense in the context of using email module facilities to parse
Metadata.

Steve

From techtonik at gmail.com  Sat Sep  1 08:27:25 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Sat, 1 Sep 2012 09:27:25 +0300
Subject: [Python-Dev] [Python-ideas] itertools.chunks(iterable, size,
	fill=None)
In-Reply-To: <jsl3cb$knn$1@dough.gmane.org>
References: <CAPkN8xKq7sfWQt9P2Vaw6B_ZwqKtNLiGdavPT=Y84yBcV08uEg@mail.gmail.com>
	<jsl3cb$knn$1@dough.gmane.org>
Message-ID: <CAPkN8xKM_K=59-Q9jfso+uGLnF=b_zrjvmCzNjEoOcGg+4P0ZQ@mail.gmail.com>

I've run into the necessity of implementing chunks() again. Here is
the code I've made from scratch.

def chunks(seq, size):
  '''Cut sequence into chunks of given size. If `seq` length is
     not divisible by `size` without reminder, last chunk will
     have length less than size.

     >>> list( chunks([1,2,3,4,5,6,7], 3) )
     [[1, 2, 3], [4, 5, 6], [7]]
  '''
  endlen = len(seq)//size
  for i in range(endlen):
    yield [seq[i*size+n] for n in range(size)]
  if len(seq) % size:
    yield seq[endlen*size:]

--
anatoly t.


On Fri, Jun 29, 2012 at 11:32 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> On 26.06.2012 10:03, anatoly techtonik wrote:
>>
>> Now that Python 3 is all about iterators (which is a user killer
>> feature for Python according to StackOverflow -
>> http://stackoverflow.com/questions/tagged/python) would it be nice to
>> introduce more first class functions to work with them? One function
>> to be exact to split string into chunks.
>>
>>      itertools.chunks(iterable, size, fill=None)
>>
>> Which is the 33th most voted Python question on SO -
>>
>> http://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks-in-python/312464
>>
>> P.S. CC'ing to python-dev@ to notify about the thread in python-ideas.
>>
>
> Anatoly, so far there were no negative votes -- would you care to go
> another step and propose a patch?
>
>
> Georg
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas

From rdmurray at bitdance.com  Sat Sep  1 14:58:30 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Sat, 01 Sep 2012 08:58:30 -0400
Subject: [Python-Dev] Edits to Metadata 1.2 to add extras (optional
	ependencies)
In-Reply-To: <87oblqtk9c.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <20120828041552.GA21422@chang>
	<87wr0jtxzy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<503CD917.5050902@v.loewis.de>
	<C4B87D5AB77C404D945F2B1E0DB97482@gmail.com>
	<503CDEA4.9090800@v.loewis.de>
	<691A785F4B8F4AE09948ADCBDD7534C2@gmail.com>
	<20120828153811.CBDD32500FE@webabinitio.net>
	<503CF398.3020606@v.loewis.de>
	<20120828171541.GA9777@iskra.aviel.ru>
	<503D0B9C.6070308@v.loewis.de>
	<20120828183807.GA12554@iskra.aviel.ru>
	<CAG8k2+6scwXX8QVx_TgqAKHphhWq=2fT9062F_Zi-kpD3-7VDg@mail.gmail.com>
	<50409690.9030109@v.loewis.de>
	<C1D864B70CE64B088D681994F630010C@gmail.com>
	<466B4750-EC6C-4773-87BF-E742652DD958@gmail.com>
	<20120831124119.9763A25010D@webabinitio.net>
	<CADiSq7fVfmmdjJP+anvT_v7kH6=xHye7BhH3XNqxFZoytpa8ZQ@mail.gmail.com>
	<5040D964.8030202@v.loewis.de>
	<87oblqtk9c.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20120901125831.6EA782500FE@webabinitio.net>

On Sat, 01 Sep 2012 13:55:11 +0900, "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> "Martin v. L??wis" writes:
> 
>  > Unfortunately, this conflicts with the desire to use UTF-8 in attribute
>  > values - RFC 822 (and also 2822) don't support this, but require the
>  > use oF MIME instead (Q or B encoding).
> 
> This can be achieved simply by extending the set of characters
> permitted, as MIME did for message bodies.  I'd be cautious about RFC
> 5335, not just because it's experimental, but because there may be
> other requirements we don't want to mess with.  (If RDM says
> otherwise, listen to him.  I just know the RFC exists.)

That is essentially what that RFC does.  I haven't gone through it with
a fine-tooth yet, but that's why I say the parsing side mostly works
already: we allow unicode characters anywhere non-special-characters are
allowed during parsing.  The only issue is that we encode non-ASCII using
the normal rules during serialization, so we need a new policy control to
disable that.  I'm thinking it will be any easy addition...the hard part
for RFC5335 is doing that fine-tooth read and adding appropriate tests.

Alternatively, as Donald pointed out, you can use the Binary mode, where
the utf-8 bytes just go along for the ride.  In the context of the
metadata, I think that should produce the desired results, since there
should be no need to re-wrap metadata lines.  It will also preserve the
line endings *if* you don't use the new policies.  But that is why I
would prefer to use explicit RFC5335 support...I'd like the email
backward compatibility policy to go away some day :)  (On the gripping
hand, it will always be possible to recreate it as a custom policy.)

>  > RFC 2822 also has a continuation line semantics which traditionally 
>  > conflicts with the metadata; in particular, line breaks cannot be 
>  > represented (but are interpreted as continuation lines instead).
> 
> Of course line breaks can be represented, without any further change
> to RFC 2822.  Just use Unicode LINE SEPARATOR.  You could even do it
> within ASCII by adhering strictly to RFC 2822 syntax which interprets
> continuation lines by removing exactly the CRLF pair.  Just use ASCII
> TAB as the field separator.

Yes, that is what I was talking to Tarek about.  And since ReST source
shouldn't contain tabs, a tab would probably work as the separator,
if for some reason you didn't want to use LINE SEPARATOR.

> There's a final dodge that occurs to me: the semantics you're talking
> about are *lexical* semantics in the RFC 2822 context (line unfolding
> and RFC 2047 decoding).  We could possibly in the context of the email
> module treat Metadata as an intermediate post-lexical-decoding
> pre-syntactic-analysis representation.  I don't know if that makes
> sense in the context of using email module facilities to parse
> Metadata.

The policy has hooks that support this.  A policy gets handed the source
line complete with the line breaks, determines what gets stored in the
model, and also gets to control what gets handed back to the application
when a header is retrieved from the model.  The policy can also control
the header folding during serialization.  So preserving line separators
using a custom policy is not only possible, but should be fairly easy.

--David

From victor.stinner at gmail.com  Sat Sep  1 15:44:23 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Sat, 1 Sep 2012 15:44:23 +0200
Subject: [Python-Dev] Unicode support of the curses module in Python 3.3
Message-ID: <CAMpsgwbyRD9um-4RurrMumtFGz5rivfTdQ0amZf3EqKTCW7abw@mail.gmail.com>

Hi,

I changed many functions of the curses module in Python 3.3 to improve
its Unicode support:
 - new functions: curses.unget_wch() and window.get_wch()
 - new attribute: window.encoding
 - the default encoding is now the locale encoding instead of UTF-8
 - use the C functions *_wch() and *wstr() when available instead of
*ch() and *str() functions. For example, the Python function addstr()
calls waddwstr() and addch(str) calls wadd_wch() (addch(int) and
addch(bytes) are still calling waddch())

Most new features related to Unicode now depends if the Python curses
module is linked to the C libncursesw library or not... and the Python
module is not linked to libncursesw if the libreadline library is
linked to libncurses module. How the readline library is linked to
libncurses/libncursesw is not a new problem but it may become more
annoying than before. I hope that most Linux distro are/will link
readline to libncursesw.

For example, if the Python curses module is not linked to libncursesw,
get_wch() is not available and addch("?") raises an OverflowError if
the locale encoding is UTF-8 (because "?".encode("utf-8") is longer
than 1 byte).

I introduced two bugs: get_wch() didn't support keycodes (like
curses.KEY_UP) and addch() didn't work anymore with special characters
like curses.ACS_HLINE. These issues are referenced as #15785 and
#14223 in the bug tracker, and I pushed fixes: c58789634d22 and
27b5bd5f0e4c. I hope that Georg will accept them in Python 3.3 final!

I didn't find these bugs myself because I only used dummy scripts to
test my changes. Does anyone know "real world" applications using the
curses module and supporting Python 3? Can you please test them with
non-ASCII characters and the last development version of Python 3.3?

So please try to test the curses module before Python 3.3 final with
your favorite application!

Victor

From miki.tebeka at gmail.com  Sat Sep  1 16:42:19 2012
From: miki.tebeka at gmail.com (Miki Tebeka)
Date: Sat, 1 Sep 2012 07:42:19 -0700 (PDT)
Subject: [Python-Dev] [Python-ideas] itertools.chunks(iterable, size,
	fill=None)
In-Reply-To: <CAPkN8xKM_K=59-Q9jfso+uGLnF=b_zrjvmCzNjEoOcGg+4P0ZQ@mail.gmail.com>
References: <CAPkN8xKq7sfWQt9P2Vaw6B_ZwqKtNLiGdavPT=Y84yBcV08uEg@mail.gmail.com>
	<jsl3cb$knn$1@dough.gmane.org>
	<CAPkN8xKM_K=59-Q9jfso+uGLnF=b_zrjvmCzNjEoOcGg+4P0ZQ@mail.gmail.com>
Message-ID: <d1c0db48-cf7b-4d7f-bb8a-fe2f950c4a6c@googlegroups.com>

See the "grouper" example in http://docs.python.org/library/itertools.html

On Friday, August 31, 2012 11:28:33 PM UTC-7, anatoly techtonik wrote:
>
> I've run into the necessity of implementing chunks() again. Here is 
> the code I've made from scratch. 
>
> def chunks(seq, size): 
>   '''Cut sequence into chunks of given size. If `seq` length is 
>      not divisible by `size` without reminder, last chunk will 
>      have length less than size. 
>
>      >>> list( chunks([1,2,3,4,5,6,7], 3) ) 
>      [[1, 2, 3], [4, 5, 6], [7]] 
>   ''' 
>   endlen = len(seq)//size 
>   for i in range(endlen): 
>     yield [seq[i*size+n] for n in range(size)] 
>   if len(seq) % size: 
>     yield seq[endlen*size:] 
>
> -- 
> anatoly t. 
>
>
> On Fri, Jun 29, 2012 at 11:32 PM, Georg Brandl <g.br... at gmx.net<javascript:>> 
> wrote: 
> > On 26.06.2012 10:03, anatoly techtonik wrote: 
> >> 
> >> Now that Python 3 is all about iterators (which is a user killer 
> >> feature for Python according to StackOverflow - 
> >> http://stackoverflow.com/questions/tagged/python) would it be nice to 
> >> introduce more first class functions to work with them? One function 
> >> to be exact to split string into chunks. 
> >> 
> >>      itertools.chunks(iterable, size, fill=None) 
> >> 
> >> Which is the 33th most voted Python question on SO - 
> >> 
> >> 
> http://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks-in-python/312464 
> >> 
> >> P.S. CC'ing to python-dev@ to notify about the thread in python-ideas. 
> >> 
> > 
> > Anatoly, so far there were no negative votes -- would you care to go 
> > another step and propose a patch? 
> > 
> > 
> > Georg 
> > 
> > _______________________________________________ 
> > Python-ideas mailing list 
> > Python... at python.org <javascript:> 
> > http://mail.python.org/mailman/listinfo/python-ideas 
> _______________________________________________ 
> Python-ideas mailing list 
> Python... at python.org <javascript:> 
> http://mail.python.org/mailman/listinfo/python-ideas 
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120901/c9cd341e/attachment.html>

From steve at pearwood.info  Sun Sep  2 00:34:58 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 02 Sep 2012 08:34:58 +1000
Subject: [Python-Dev] Unicode support of the curses module in Python 3.3
In-Reply-To: <CAMpsgwbyRD9um-4RurrMumtFGz5rivfTdQ0amZf3EqKTCW7abw@mail.gmail.com>
References: <CAMpsgwbyRD9um-4RurrMumtFGz5rivfTdQ0amZf3EqKTCW7abw@mail.gmail.com>
Message-ID: <50428D92.7040509@pearwood.info>

On 01/09/12 23:44, Victor Stinner wrote:
> Hi,
>
> I changed many functions of the curses module in Python 3.3 to improve
> its Unicode support:
[...]

Thank you.


> For example, if the Python curses module is not linked to libncursesw,
> get_wch() is not available and addch("?") raises an OverflowError if
> the locale encoding is UTF-8 (because "?".encode("utf-8") is longer
> than 1 byte).

OverflowError? That is very surprising. I wouldn't guess that calling
addch could raise OverflowError.

Could you use a less surprising exception, or at least make sure that
it is clearly and obviously documented?


-- 
Steven

From victor.stinner at gmail.com  Sun Sep  2 12:44:47 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Sun, 2 Sep 2012 12:44:47 +0200
Subject: [Python-Dev] Unicode support of the curses module in Python 3.3
In-Reply-To: <50428D92.7040509@pearwood.info>
References: <CAMpsgwbyRD9um-4RurrMumtFGz5rivfTdQ0amZf3EqKTCW7abw@mail.gmail.com>
	<50428D92.7040509@pearwood.info>
Message-ID: <CAMpsgwZcQaUmkYVrC6VRLx49ZEu8JVRKcE14Xdp9qY6xruBx1A@mail.gmail.com>

>> For example, if the Python curses module is not linked to libncursesw,
>> get_wch() is not available and addch("?") raises an OverflowError if
>> the locale encoding is UTF-8 (because "?".encode("utf-8") is longer
>> than 1 byte).
>
> OverflowError? That is very surprising. I wouldn't guess that calling
> addch could raise OverflowError.
>
> Could you use a less surprising exception, or at least make sure that
> it is clearly and obviously documented?

Which exception would you expect? ValueError? Another error?

I used the same exception for addch(int) and addch(str). addch(int)
already raises an OverflowError on Python 3.2 if the value doesn't fit
in a C long type (which may be bigger than chtype, Python 3.3 is more
strict), so applications don't need to be changed for the "new" error
(on addch(str)).

Victor

From techtonik at gmail.com  Sun Sep  2 13:36:59 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Sun, 2 Sep 2012 14:36:59 +0300
Subject: [Python-Dev] [Python-ideas] itertools.chunks(iterable, size,
	fill=None)
In-Reply-To: <d1c0db48-cf7b-4d7f-bb8a-fe2f950c4a6c@googlegroups.com>
References: <CAPkN8xKq7sfWQt9P2Vaw6B_ZwqKtNLiGdavPT=Y84yBcV08uEg@mail.gmail.com>
	<jsl3cb$knn$1@dough.gmane.org>
	<CAPkN8xKM_K=59-Q9jfso+uGLnF=b_zrjvmCzNjEoOcGg+4P0ZQ@mail.gmail.com>
	<d1c0db48-cf7b-4d7f-bb8a-fe2f950c4a6c@googlegroups.com>
Message-ID: <CAPkN8xKNgYWSy=C2rpxyiP+jQgr7k5w61qpRagPSSCoHwHWcyQ@mail.gmail.com>

On Sat, Sep 1, 2012 at 5:42 PM, Miki Tebeka <miki.tebeka at gmail.com> wrote:
> See the "grouper" example in http://docs.python.org/library/itertools.html

As was discussed before, the problem is visibility of the solution,
not the implementation. If we can divide core Python API into levels
where 0 is the less important and 10 is more, then `chunks` should be
level above than it is now.
--
anatoly t.

From techtonik at gmail.com  Sun Sep  2 21:48:48 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Sun, 2 Sep 2012 22:48:48 +0300
Subject: [Python-Dev] Should 2to3 convert Exceptions from
Message-ID: <CAPkN8xLV_k1rrLfmxtQcu_g3LSN-HM_Lc6RpsQ+kxJLLr7ChDw@mail.gmail.com>

I work offline from remote location about 2000m above the sea level. There
is no internet connection here, so I can not use tracker online. I need a
Python editor here, and I have Spyder checkout. The problem is that my
installation has only Python3. I've tried using 2to3 from setup.py
(attached), but it fails when I execute:

$ sudo python3 setup.py install

stderr.log and stdout.log are attached. Is it the intended behavior?
It is also hard find "Porting Python 2 Code to Python 3" article, because
it is not referenced from "Porting to Python 3.x" chapters.



-- 
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120902/6f4e40af/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: setup.py
Type: text/x-python
Size: 5883 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120902/6f4e40af/attachment.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stderr.log
Type: text/x-log
Size: 7008 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120902/6f4e40af/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stdout.log
Type: text/x-log
Size: 4528 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120902/6f4e40af/attachment-0001.bin>

From regebro at gmail.com  Sun Sep  2 22:26:03 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 2 Sep 2012 22:26:03 +0200
Subject: [Python-Dev] Should 2to3 convert Exceptions from
In-Reply-To: <CAPkN8xLV_k1rrLfmxtQcu_g3LSN-HM_Lc6RpsQ+kxJLLr7ChDw@mail.gmail.com>
References: <CAPkN8xLV_k1rrLfmxtQcu_g3LSN-HM_Lc6RpsQ+kxJLLr7ChDw@mail.gmail.com>
Message-ID: <CAL0kPAWZWtoz=7G++gAKjOfuXgA-Kzj+GyO_XHqYwfch1ixRjg@mail.gmail.com>

Switched from python-dev to python-porting.

On Sun, Sep 2, 2012 at 9:48 PM, anatoly techtonik <techtonik at gmail.com> wrote:
> I work offline from remote location about 2000m above the sea level. There
> is no internet connection here, so I can not use tracker online. I need a
> Python editor here, and I have Spyder checkout. The problem is that my
> installation has only Python3. I've tried using 2to3 from setup.py
> (attached), but it fails when I execute:
>
> $ sudo python3 setup.py install
>
> stderr.log and stdout.log are attached. Is it the intended behavior?
> It is also hard find "Porting Python 2 Code to Python 3" article, because it
> is not referenced from "Porting to Python 3.x" chapters.

2to3 has not been run on the code that gives you errors. The traceback
seems incomplete, and gives no indication of where in the install you
get the errors. From the stdout it seems like it just installs the
code without using 2to3.

The only idea I have at this moment i sto make sure that the build
directories are empty.

//Lennart

From solipsis at pitrou.net  Mon Sep  3 12:11:23 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 3 Sep 2012 12:11:23 +0200
Subject: [Python-Dev] ABI stability between bugfix versions?
Message-ID: <20120903121123.1cf7b413@pitrou.net>


Hello,

To fix the crash in http://bugs.python.org/issue13992, I have to change
the definition of the Py_TRASHCAN_SAFE_BEGIN and Py_TRASHCAN_SAFE_END
macros in a way that makes them stop rely on global variables.

Despite being undocumented, these two macros are used in a couple of
third-party extension modules (*) such as greenlet and py-judy. This
means that such an extension compiled for 2.7.3 wouldn't work with
2.7.4, and vice-versa. Is there something we can do to alleviate the
issue? Add a note in the release notes?

Regards

Antoine.


(*) http://code.google.com/codesearch#search&q=Py_TRASHCAN_SAFE_BEGIN


-- 
Software development and contracting: http://pro.pitrou.net



From lists at cheimes.de  Mon Sep  3 15:59:59 2012
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 03 Sep 2012 15:59:59 +0200
Subject: [Python-Dev] Coverity scan
Message-ID: <5044B7DF.1040100@cheimes.de>

Hello everybody,

two weeks ago Stefan Krah asked for a current Coverity scan report.
Coverity has updated us to a new version with a new workflow. Before the
update Coverity pulled the code from our version control system. However
the latest version doesn't work that way. The code must be compiled with
the Coverity tool chain, published to a public web server and then
downloaded by Coverity.

http://mail.python.org/pipermail/python-committers/2012-August/002134.html

It's easy, doesn't take much effort and can easily be automated, but
somebody has to do it. The process should also be placed on the Python
infrastructure and I don't have access. Secondly somebody has to contact
Coverity to apply for an upload account. I tried my user account without
success.

It be nice if we get Coverity scans up and running this week to check
the upcoming release candidate for issues.

Christian

From solipsis at pitrou.net  Mon Sep  3 16:27:16 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 3 Sep 2012 16:27:16 +0200
Subject: [Python-Dev] Coverity scan
References: <5044B7DF.1040100@cheimes.de>
Message-ID: <20120903162716.6502bd60@pitrou.net>

On Mon, 03 Sep 2012 15:59:59 +0200
Christian Heimes <lists at cheimes.de> wrote:
> 
> It's easy, doesn't take much effort and can easily be automated, but
> somebody has to do it. The process should also be placed on the Python
> infrastructure and I don't have access. Secondly somebody has to contact
> Coverity to apply for an upload account. I tried my user account without
> success.

You could ask infrastructure at python.org for an account on an existing
machine (dinsdale perhaps, it looks much less loaded now that some
services have been migrated).

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From lists at cheimes.de  Mon Sep  3 17:10:20 2012
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 03 Sep 2012 17:10:20 +0200
Subject: [Python-Dev] Coverity scan
In-Reply-To: <20120903162716.6502bd60@pitrou.net>
References: <5044B7DF.1040100@cheimes.de> <20120903162716.6502bd60@pitrou.net>
Message-ID: <5044C85C.10409@cheimes.de>

Am 03.09.2012 16:27, schrieb Antoine Pitrou:
> You could ask infrastructure at python.org for an account on an existing
> machine (dinsdale perhaps, it looks much less loaded now that some
> services have been migrated).

Thanks Antoine! I've contacted the infrastructure team.

Christian




From techtonik at gmail.com  Mon Sep  3 22:18:11 2012
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 3 Sep 2012 23:18:11 +0300
Subject: [Python-Dev] Should 2to3 convert Exceptions from
In-Reply-To: <CAL0kPAWZWtoz=7G++gAKjOfuXgA-Kzj+GyO_XHqYwfch1ixRjg@mail.gmail.com>
References: <CAPkN8xLV_k1rrLfmxtQcu_g3LSN-HM_Lc6RpsQ+kxJLLr7ChDw@mail.gmail.com>
	<CAL0kPAWZWtoz=7G++gAKjOfuXgA-Kzj+GyO_XHqYwfch1ixRjg@mail.gmail.com>
Message-ID: <CAPkN8x+pWjCpGfY5VCJz4T3onBhHpS0bNVrY33LENaEyAvJr5g@mail.gmail.com>

On Sun, Sep 2, 2012 at 11:26 PM, Lennart Regebro <regebro at gmail.com> wrote:
> Switched from python-dev to python-porting.
>
> On Sun, Sep 2, 2012 at 9:48 PM, anatoly techtonik <techtonik at gmail.com> wrote:
>> I work offline from remote location about 2000m above the sea level. There
>> is no internet connection here, so I can not use tracker online. I need a
>> Python editor here, and I have Spyder checkout. The problem is that my
>> installation has only Python3. I've tried using 2to3 from setup.py
>> (attached), but it fails when I execute:
>>
>> $ sudo python3 setup.py install
>>
>> stderr.log and stdout.log are attached. Is it the intended behavior?
>> It is also hard find "Porting Python 2 Code to Python 3" article, because it
>> is not referenced from "Porting to Python 3.x" chapters.
>
> 2to3 has not been run on the code that gives you errors. The traceback
> seems incomplete, and gives no indication of where in the install you
> get the errors. From the stdout it seems like it just installs the
> code without using 2to3.
>
> The only idea I have at this moment i sto make sure that the build
> directories are empty.

Removing build/ directory helped indeed. I wonder why it doesn't
remove them automatically? IIRC there is no incremental build support.
--
anatoly t.

From martin at v.loewis.de  Mon Sep  3 22:37:31 2012
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 03 Sep 2012 22:37:31 +0200
Subject: [Python-Dev] ABI stability between bugfix versions?
In-Reply-To: <20120903121123.1cf7b413@pitrou.net>
References: <20120903121123.1cf7b413@pitrou.net>
Message-ID: <5045150B.4070008@v.loewis.de>

Am 03.09.2012 12:11, schrieb Antoine Pitrou:
> Despite being undocumented, these two macros are used in a couple of
> third-party extension modules (*) such as greenlet and py-judy. This
> means that such an extension compiled for 2.7.3 wouldn't work with
> 2.7.4, and vice-versa. Is there something we can do to alleviate the
> issue? Add a note in the release notes?

I think breaking existing extension modules in a bugfix release is
really not acceptable.

I also think that the breakage can be reduced by keeping the global
variable. There would then be two counters: the global used by
old modules, the per-thread counter used by recompiled modules.
I believe this would require separate versions of the helper functions
also.

In the worst case, this could cause the nesting to go twice as deep,
which is not a problem, since the nesting is only 50, anyway.

The question also is about the other direction: modules compiled
for 2.7.4 that are run on 2.7.3. I think this would be more problematic
in your patch, since the code would just access fields in the thread
state which aren't there. If the helper functions are called
differently, the extension would fail to load, which IMO is the better
reaction.

Regards,
Martin

From regebro at gmail.com  Mon Sep  3 22:59:26 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Mon, 3 Sep 2012 22:59:26 +0200
Subject: [Python-Dev] Should 2to3 convert Exceptions from
In-Reply-To: <CAPkN8x+pWjCpGfY5VCJz4T3onBhHpS0bNVrY33LENaEyAvJr5g@mail.gmail.com>
References: <CAPkN8xLV_k1rrLfmxtQcu_g3LSN-HM_Lc6RpsQ+kxJLLr7ChDw@mail.gmail.com>
	<CAL0kPAWZWtoz=7G++gAKjOfuXgA-Kzj+GyO_XHqYwfch1ixRjg@mail.gmail.com>
	<CAPkN8x+pWjCpGfY5VCJz4T3onBhHpS0bNVrY33LENaEyAvJr5g@mail.gmail.com>
Message-ID: <CAL0kPAX5FhSATV0tnyhhjYZhHVnPkKDOn8JgWGQXj9itoc+WLQ@mail.gmail.com>

On Mon, Sep 3, 2012 at 10:18 PM, anatoly techtonik <techtonik at gmail.com> wrote:
> On Sun, Sep 2, 2012 at 11:26 PM, Lennart Regebro <regebro at gmail.com> wrote:
>> Switched from python-dev to python-porting.
>>
>> On Sun, Sep 2, 2012 at 9:48 PM, anatoly techtonik <techtonik at gmail.com> wrote:
>>> I work offline from remote location about 2000m above the sea level. There
>>> is no internet connection here, so I can not use tracker online. I need a
>>> Python editor here, and I have Spyder checkout. The problem is that my
>>> installation has only Python3. I've tried using 2to3 from setup.py
>>> (attached), but it fails when I execute:
>>>
>>> $ sudo python3 setup.py install
>>>
>>> stderr.log and stdout.log are attached. Is it the intended behavior?
>>> It is also hard find "Porting Python 2 Code to Python 3" article, because it
>>> is not referenced from "Porting to Python 3.x" chapters.
>>
>> 2to3 has not been run on the code that gives you errors. The traceback
>> seems incomplete, and gives no indication of where in the install you
>> get the errors. From the stdout it seems like it just installs the
>> code without using 2to3.
>>
>> The only idea I have at this moment i sto make sure that the build
>> directories are empty.
>
> Removing build/ directory helped indeed. I wonder why it doesn't
> remove them automatically? IIRC there is no incremental build support.

It doesn't remove them to avoid having to recompile everything
everytime you run it. But it compares timestamps, so if you run build,
and then add the 2to3 support, it will see that all the py-files are
up to date, and not run 2to3 on them.

//Lennart

From solipsis at pitrou.net  Tue Sep  4 00:59:40 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 4 Sep 2012 00:59:40 +0200
Subject: [Python-Dev] ABI stability between bugfix versions?
In-Reply-To: <5045150B.4070008@v.loewis.de>
References: <20120903121123.1cf7b413@pitrou.net> <5045150B.4070008@v.loewis.de>
Message-ID: <20120904005940.12bb71e5@pitrou.net>

On Mon, 03 Sep 2012 22:37:31 +0200
"Martin v. L?wis" <martin at v.loewis.de> wrote:
> Am 03.09.2012 12:11, schrieb Antoine Pitrou:
> > Despite being undocumented, these two macros are used in a couple of
> > third-party extension modules (*) such as greenlet and py-judy. This
> > means that such an extension compiled for 2.7.3 wouldn't work with
> > 2.7.4, and vice-versa. Is there something we can do to alleviate the
> > issue? Add a note in the release notes?
> 
> I think breaking existing extension modules in a bugfix release is
> really not acceptable.
> 
> I also think that the breakage can be reduced by keeping the global
> variable. There would then be two counters: the global used by
> old modules, the per-thread counter used by recompiled modules.
> I believe this would require separate versions of the helper functions
> also.

Indeed. That sounds workable, despite the added complexity.

> The question also is about the other direction: modules compiled
> for 2.7.4 that are run on 2.7.3. I think this would be more problematic
> in your patch, since the code would just access fields in the thread
> state which aren't there. If the helper functions are called
> differently, the extension would fail to load, which IMO is the better
> reaction.

Right, that's a better behaviour than crashing.

Thank you,

Antoine.

From andrew.svetlov at gmail.com  Tue Sep  4 11:24:10 2012
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Tue, 4 Sep 2012 12:24:10 +0300
Subject: [Python-Dev] Tru64 support
Message-ID: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>

Unittests for subprocess module has weird comment:
We cannot use os.path.realpath to canonicalize the path, since it
doesn't expand Tru64 {memb} strings.
See bug 1063571.
http://hg.python.org/cpython/file/82ae284cd5f1/Lib/test/test_subprocess.py#l374
is link to comment,
http://bugs.python.org/issue1063571 ? issue in bugtracker.

Is it still up to date? Bug has been created in 2004.
I don't see Tru64 in list of available buildbots.

Do we need to care about this platform? And how to make sure what
existing code works fine for that?
Or maybe we can drop support for Tru64 as well as we already does it
for, say, VMS?

-- 
Thanks,
Andrew Svetlov

From chris.jerdonek at gmail.com  Tue Sep  4 12:12:28 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Tue, 4 Sep 2012 03:12:28 -0700
Subject: [Python-Dev] Tru64 support
In-Reply-To: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
Message-ID: <CAOTb1wc8=GdDYTpbMU=zWCxX0_=zm98cCFM41+01ChFCOUM=ag@mail.gmail.com>

On Tue, Sep 4, 2012 at 2:24 AM, Andrew Svetlov <andrew.svetlov at gmail.com> wrote:
> Unittests for subprocess module has weird comment:
> We cannot use os.path.realpath to canonicalize the path, since it
> doesn't expand Tru64 {memb} strings.
> See bug 1063571.
> http://hg.python.org/cpython/file/82ae284cd5f1/Lib/test/test_subprocess.py#l374
> is link to comment,
> http://bugs.python.org/issue1063571 ? issue in bugtracker.

And to expand on Andrew's comments, issue #15533 is the current issue
that this pertains to--

http://bugs.python.org/issue15533

--Chris

From victor.stinner at gmail.com  Tue Sep  4 14:02:53 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 4 Sep 2012 14:02:53 +0200
Subject: [Python-Dev] Tru64 support
In-Reply-To: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
Message-ID: <CAMpsgwZgQtJJxmb_-MqsXRyHby6+so5UyYT62Smy6efMy4DwCQ@mail.gmail.com>

2012/9/4 Andrew Svetlov <andrew.svetlov at gmail.com>:
> I don't see Tru64 in list of available buildbots.

Nope, there is no Tru64 buildbot.

> Do we need to care about this platform? And how to make sure what
> existing code works fine for that?

I don't know this OS. According to Wikipedia, it's a closed source OS
which only runs on DEC Alpha CPU. It doesn't look to be maintained:
only bugs are fixed since 2010.

> Or maybe we can drop support for Tru64 as well as we already does it
> for, say, VMS?

See the PEP 11 for unsupported platforms.
http://www.python.org/dev/peps/pep-0011/#no-longer-supported-platforms

"Name: VMS
Unsupported in: Python 3.3
Code removed in: Python 3.4"

Victor

From chris.jerdonek at gmail.com  Tue Sep  4 14:10:43 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Tue, 4 Sep 2012 05:10:43 -0700
Subject: [Python-Dev] Tru64 support
In-Reply-To: <CAMpsgwZgQtJJxmb_-MqsXRyHby6+so5UyYT62Smy6efMy4DwCQ@mail.gmail.com>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
	<CAMpsgwZgQtJJxmb_-MqsXRyHby6+so5UyYT62Smy6efMy4DwCQ@mail.gmail.com>
Message-ID: <CAOTb1wfciAkNLfNVyjJEBYpgcpXi_Uid3d4MFNM9mm4fX4GRJg@mail.gmail.com>

On Tue, Sep 4, 2012 at 5:02 AM, Victor Stinner <victor.stinner at gmail.com> wrote:
>> Or maybe we can drop support for Tru64 as well as we already does it
>> for, say, VMS?
>
> See the PEP 11 for unsupported platforms.
> http://www.python.org/dev/peps/pep-0011/#no-longer-supported-platforms
>
> "Name: VMS
> Unsupported in: Python 3.3
> Code removed in: Python 3.4"

And for Python 2.7 when adding new tests, does this mean we should
write those tests in a way that attempts to have them pass on Tru64
(e.g. by following the pattern of another test), or should we not
worry about it?

--Chris

From solipsis at pitrou.net  Tue Sep  4 14:11:22 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 4 Sep 2012 14:11:22 +0200
Subject: [Python-Dev] Tru64 support
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
Message-ID: <20120904141122.351032b4@pitrou.net>

On Tue, 4 Sep 2012 12:24:10 +0300
Andrew Svetlov <andrew.svetlov at gmail.com> wrote:
> Unittests for subprocess module has weird comment:
> We cannot use os.path.realpath to canonicalize the path, since it
> doesn't expand Tru64 {memb} strings.
> See bug 1063571.
> http://hg.python.org/cpython/file/82ae284cd5f1/Lib/test/test_subprocess.py#l374
> is link to comment,
> http://bugs.python.org/issue1063571 ? issue in bugtracker.
> 
> Is it still up to date? Bug has been created in 2004.
> I don't see Tru64 in list of available buildbots.
> 
> Do we need to care about this platform?

I don't think so. If some hypothetical Tru64 user cares, they can
probably open a bug on the tracker.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From andrew.svetlov at gmail.com  Tue Sep  4 14:19:38 2012
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Tue, 4 Sep 2012 15:19:38 +0300
Subject: [Python-Dev] Tru64 support
In-Reply-To: <20120904141122.351032b4@pitrou.net>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
	<20120904141122.351032b4@pitrou.net>
Message-ID: <CAL3CFcVLBao5CuwUu3o_z0EUqNmN1osqZBvzGJkje+4WnYNHMQ@mail.gmail.com>

On Tue, Sep 4, 2012 at 3:11 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> I don't think so. If some hypothetical Tru64 user cares, they can
> probably open a bug on the tracker.
>
Thanks.

-- 
Thanks,
Andrew Svetlov

From andrew.svetlov at gmail.com  Tue Sep  4 15:00:45 2012
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Tue, 4 Sep 2012 16:00:45 +0300
Subject: [Python-Dev] Tru64 support
In-Reply-To: <CAL3CFcVLBao5CuwUu3o_z0EUqNmN1osqZBvzGJkje+4WnYNHMQ@mail.gmail.com>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
	<20120904141122.351032b4@pitrou.net>
	<CAL3CFcVLBao5CuwUu3o_z0EUqNmN1osqZBvzGJkje+4WnYNHMQ@mail.gmail.com>
Message-ID: <CAL3CFcXW946ZYzJ_aVSJODeshu7a0Ukq0FRMjZ+OJB6ND1jLSg@mail.gmail.com>

Aha, ./Doc/whatsnew/3.0.rst declares dropping support for Tru64.
Several files in repo mention it, though.

On Tue, Sep 4, 2012 at 3:19 PM, Andrew Svetlov <andrew.svetlov at gmail.com> wrote:
> On Tue, Sep 4, 2012 at 3:11 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> I don't think so. If some hypothetical Tru64 user cares, they can
>> probably open a bug on the tracker.
>>
> Thanks.
>
> --
> Thanks,
> Andrew Svetlov



-- 
Thanks,
Andrew Svetlov

From rdmurray at bitdance.com  Tue Sep  4 15:05:25 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 04 Sep 2012 09:05:25 -0400
Subject: [Python-Dev] Tru64 support
In-Reply-To: <CAL3CFcVLBao5CuwUu3o_z0EUqNmN1osqZBvzGJkje+4WnYNHMQ@mail.gmail.com>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
	<20120904141122.351032b4@pitrou.net>
	<CAL3CFcVLBao5CuwUu3o_z0EUqNmN1osqZBvzGJkje+4WnYNHMQ@mail.gmail.com>
Message-ID: <20120904130525.CB382250066@webabinitio.net>

On Tue, 04 Sep 2012 15:19:38 +0300, Andrew Svetlov <andrew.svetlov at gmail.com> wrote:
> On Tue, Sep 4, 2012 at 3:11 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > I don't think so. If some hypothetical Tru64 user cares, they can
> > probably open a bug on the tracker.
> >
> Thanks.

I used to work on Tru64 systems many years ago.

According to the pages on HP's site (http://h30097.www3.hp.com/) that
Google found me (HP acquired the Alpha platform that Tru64 runs on,
among other things), the Alpha itself was discontinued in 2007, and
Tru64 has reached the end of "Mature Product Support with Sustaining
Engineering" this year.  From this point on it is just "Mature Product
Support".  Not sure exactly what that means, other than that Tru64 is
definitely a dying platform.

--David

From trent at snakebite.org  Tue Sep  4 17:56:25 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 4 Sep 2012 11:56:25 -0400
Subject: [Python-Dev] Tru64 support
In-Reply-To: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
Message-ID: <20120904155624.GA4430@snakebite.org>

On Tue, Sep 04, 2012 at 02:24:10AM -0700, Andrew Svetlov wrote:
> Is it still up to date? Bug has been created in 2004.
> I don't see Tru64 in list of available buildbots.
>
> Do we need to care about this platform? And how to make sure what
> existing code works fine for that?
>
> Or maybe we can drop support for Tru64 as well as we already does it
> for, say, VMS?

    The ES40 I've got sitting in the corner waiting for a rainy day just
    died a little inside.  (HP was also kind enough to donate unlimited
    Tru64 media/licenses a while back.)  As for whether or not it should
    be a platform CPython cares about -- I'd argue it falls in the same
    boat as IRIX/MIPS: not so much.  I'll be damned if Snakebite doesn't
    have Tru64/Alpha representation though :-)

    OpenVMS is an interesting one though; it's still actively supported
    by HP on Itanium.  I've got an Itanium earmarked marked for OpenVMS
    "for an even rainier day", but I wouldn't know OpenVMS is it smacked
    me in the head, so, not a high priority at the moment.

        Trent.

From tjreedy at udel.edu  Tue Sep  4 20:08:40 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 04 Sep 2012 14:08:40 -0400
Subject: [Python-Dev] Tru64 support
In-Reply-To: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
Message-ID: <k25g3c$3i4$1@ger.gmane.org>

On 9/4/2012 5:24 AM, Andrew Svetlov wrote:
> Unittests for subprocess module has weird comment:
> We cannot use os.path.realpath to canonicalize the path, since it
> doesn't expand Tru64 {memb} strings.
> See bug 1063571.
> http://hg.python.org/cpython/file/82ae284cd5f1/Lib/test/test_subprocess.py#l374
> is link to comment,
> http://bugs.python.org/issue1063571 ? issue in bugtracker.

It seems to me that the resolution of that issue should have been to 
either make os.path.realpath work correctly on Tru64, so we can freely 
use it as documented. If that is not possible, the doc should say not 
available ... and the test skipped on Tru64. I realizes that skipping 
tests is perhaps easier now.

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Tue Sep  4 20:11:07 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 04 Sep 2012 14:11:07 -0400
Subject: [Python-Dev] Tru64 support
In-Reply-To: <CAL3CFcXW946ZYzJ_aVSJODeshu7a0Ukq0FRMjZ+OJB6ND1jLSg@mail.gmail.com>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
	<20120904141122.351032b4@pitrou.net>
	<CAL3CFcVLBao5CuwUu3o_z0EUqNmN1osqZBvzGJkje+4WnYNHMQ@mail.gmail.com>
	<CAL3CFcXW946ZYzJ_aVSJODeshu7a0Ukq0FRMjZ+OJB6ND1jLSg@mail.gmail.com>
Message-ID: <k25g7v$3i4$2@ger.gmane.org>

On 9/4/2012 9:00 AM, Andrew Svetlov wrote:
> Aha, ./Doc/whatsnew/3.0.rst declares dropping support for Tru64.
> Several files in repo mention it, though.

That suggests that it should be added to PEP11. Martin is the PEP 
editor, so I guess it is his final decision.


-- 
Terry Jan Reedy


From martin at v.loewis.de  Wed Sep  5 00:11:14 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Wed, 05 Sep 2012 00:11:14 +0200
Subject: [Python-Dev] Tru64 support
In-Reply-To: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
Message-ID: <20120905001114.Horde.eP_PTElCcOxQRnyCCc_14kA@webmail.df.eu>


Zitat von Andrew Svetlov <andrew.svetlov at gmail.com>:

> http://hg.python.org/cpython/file/82ae284cd5f1/Lib/test/test_subprocess.py#l374
> is link to comment,
> http://bugs.python.org/issue1063571 ? issue in bugtracker.
>
> Is it still up to date? Bug has been created in 2004.
> I don't see Tru64 in list of available buildbots.
>
> Do we need to care about this platform?

That there is no buildbot is no indication. Instead, PEP 11 keeps
track of platforms we don't want to support anymore; Tru64 isn't listed.
For a long time, there had been proponents of Tru64 on this list which
would object to removing support.

> And how to make sure what existing code works fine for that?

We don't need to. It's perfectly fine if it breaks - we just can't
actively remove code to support the platform.

OTOH, if Tru64 was proposed for "unsupport", we could disable support
in 3.4, and remove support in 3.5. Perhaps that would be a good thing
to do.

> Or maybe we can drop support for Tru64 as well as we already does it
> for, say, VMS?

No, see PEP 11.

Regards,
Martin



From martin at v.loewis.de  Wed Sep  5 00:15:30 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Wed, 05 Sep 2012 00:15:30 +0200
Subject: [Python-Dev] Tru64 support
In-Reply-To: <CAOTb1wfciAkNLfNVyjJEBYpgcpXi_Uid3d4MFNM9mm4fX4GRJg@mail.gmail.com>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
	<CAMpsgwZgQtJJxmb_-MqsXRyHby6+so5UyYT62Smy6efMy4DwCQ@mail.gmail.com>
	<CAOTb1wfciAkNLfNVyjJEBYpgcpXi_Uid3d4MFNM9mm4fX4GRJg@mail.gmail.com>
Message-ID: <20120905001530.Horde.YtfLRElCcOxQRn2Cf-zV62A@webmail.df.eu>


Zitat von Chris Jerdonek <chris.jerdonek at gmail.com>:

> And for Python 2.7 when adding new tests

Hmm. Adding new tests to 2.7 is a questionable activity in the first
place - it should be done very carefully.

> does this mean we should
> write those tests in a way that attempts to have them pass on Tru64
> (e.g. by following the pattern of another test), or should we not
> worry about it?

2.7 should absolutely be able to support Tru64 (even though it currently
may not - that would be a bug). Any potential contributor to fix Tru64 support
should find it "easy" to add fixes.

So being cautious and following existing code sounds like a good idea.
The world wouldn't end if the test broke on Tru64, though.

Regards,
Martin



From martin at v.loewis.de  Wed Sep  5 00:18:18 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Wed, 05 Sep 2012 00:18:18 +0200
Subject: [Python-Dev] Tru64 support
In-Reply-To: <20120904155624.GA4430@snakebite.org>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
	<20120904155624.GA4430@snakebite.org>
Message-ID: <20120905001818.Horde.eOoOXUlCcOxQRn4qthF17RA@webmail.df.eu>


Zitat von Trent Nelson <trent at snakebite.org>:

>     OpenVMS is an interesting one though; it's still actively supported
>     by HP on Itanium.  I've got an Itanium earmarked marked for OpenVMS
>     "for an even rainier day", but I wouldn't know OpenVMS is it smacked
>     me in the head, so, not a high priority at the moment.

We had an offer several months ago that patches to 3.3 would "likely" (IIRC)
be contributed; this didn't materialize. So the PEP 11 removal of VMS support
stands.

Regards,
Martin



From martin at v.loewis.de  Wed Sep  5 00:29:45 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Wed, 05 Sep 2012 00:29:45 +0200
Subject: [Python-Dev] Tru64 support
In-Reply-To: <k25g7v$3i4$2@ger.gmane.org>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
	<20120904141122.351032b4@pitrou.net>
	<CAL3CFcVLBao5CuwUu3o_z0EUqNmN1osqZBvzGJkje+4WnYNHMQ@mail.gmail.com>
	<CAL3CFcXW946ZYzJ_aVSJODeshu7a0Ukq0FRMjZ+OJB6ND1jLSg@mail.gmail.com>
	<k25g7v$3i4$2@ger.gmane.org>
Message-ID: <20120905002945.Horde.ZijGaUlCcOxQRoDZLjNWBQA@webmail.df.eu>


Zitat von Terry Reedy <tjreedy at udel.edu>:

> On 9/4/2012 9:00 AM, Andrew Svetlov wrote:
>> Aha, ./Doc/whatsnew/3.0.rst declares dropping support for Tru64.
>> Several files in repo mention it, though.
>
> That suggests that it should be added to PEP11. Martin is the PEP  
> editor, so I guess it is his final decision.

I'd be curious if there is any public record on this removal. I'm
not sure it actually happened; socketmodule.c still has

#if defined(__digital__) && defined(__unix__)

and configure.in still has two mentionings of Tru64. Also,
cmathmodule.c still has support code for Tru64. So if support
was removed at all, this wasn't done very carefully.

I'm fine with declaring that the removal was *announced* in 3.0,
though, and actually performed in 3.4 (assuming this then actually
happens, contributions are welcome).

Regards,
Martin



From lists at cheimes.de  Wed Sep  5 14:43:17 2012
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 05 Sep 2012 14:43:17 +0200
Subject: [Python-Dev] Coverity scan
In-Reply-To: <5044B7DF.1040100@cheimes.de>
References: <5044B7DF.1040100@cheimes.de>
Message-ID: <504748E5.9060704@cheimes.de>

Am 03.09.2012 15:59, schrieb Christian Heimes:
> It be nice if we get Coverity scans up and running this week to check
> the upcoming release candidate for issues.

Updates:

- Noah has set up a VM for me on the PSF infrastructure. I've installed
the Coverity tools, build dependencies of Python and a hg clone of the
default branch. The instrumented coverage builds are working, too.

- Brett has contacted Coverity to establish me as a second
administrative contact besides him. Once it's done I'll request an
upload account to submit the coverage data.

I try to get everything in place by tomorrow so we have some time to
check for bugs before the next RC is deployed.

Stefan:
Has Brett already requested an account for you or shall I request one
for you?

Christian


From brett at python.org  Wed Sep  5 14:45:38 2012
From: brett at python.org (Brett Cannon)
Date: Wed, 5 Sep 2012 08:45:38 -0400
Subject: [Python-Dev] Coverity scan
In-Reply-To: <504748E5.9060704@cheimes.de>
References: <5044B7DF.1040100@cheimes.de> <504748E5.9060704@cheimes.de>
Message-ID: <CAP1=2W6aGSO-ZbpDvz2V8pFTm4duHQBYuTSmZFkQtyac1_jz6Q@mail.gmail.com>

On Wed, Sep 5, 2012 at 8:43 AM, Christian Heimes <lists at cheimes.de> wrote:

> Am 03.09.2012 15:59, schrieb Christian Heimes:
> > It be nice if we get Coverity scans up and running this week to check
> > the upcoming release candidate for issues.
>
> Updates:
>
> - Noah has set up a VM for me on the PSF infrastructure. I've installed
> the Coverity tools, build dependencies of Python and a hg clone of the
> default branch. The instrumented coverage builds are working, too.
>
> - Brett has contacted Coverity to establish me as a second
> administrative contact besides him. Once it's done I'll request an
> upload account to submit the coverage data.
>
> I try to get everything in place by tomorrow so we have some time to
> check for bugs before the next RC is deployed.
>
> Stefan:
> Has Brett already requested an account for you or shall I request one
> for you?
>

I have not for no other reason than I had not thought about it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120905/eee09214/attachment.html>

From dholth at gmail.com  Wed Sep  5 14:50:02 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 5 Sep 2012 08:50:02 -0400
Subject: [Python-Dev] Edits to Metadata 1.2 to add extras (optional
	dependencies)
In-Reply-To: <CAG8k2+4QPcDBQ19su9faSrb+te=0+7gDpgmxNbm-30gggZHTJA@mail.gmail.com>
References: <20120828041552.GA21422@chang>
	<87wr0jtxzy.fsf@uwakimon.sk.tsukuba.ac.jp>
	<503CD917.5050902@v.loewis.de>
	<C4B87D5AB77C404D945F2B1E0DB97482@gmail.com>
	<503CDEA4.9090800@v.loewis.de>
	<691A785F4B8F4AE09948ADCBDD7534C2@gmail.com>
	<20120828153811.CBDD32500FE@webabinitio.net>
	<503CF398.3020606@v.loewis.de>
	<20120828171541.GA9777@iskra.aviel.ru>
	<503D0B9C.6070308@v.loewis.de>
	<20120828183807.GA12554@iskra.aviel.ru>
	<CAG8k2+6scwXX8QVx_TgqAKHphhWq=2fT9062F_Zi-kpD3-7VDg@mail.gmail.com>
	<50409690.9030109@v.loewis.de>
	<0C702AFF-9D1C-4BB0-A247-A226FE4C3F5E@gmail.com>
	<50409B6F.9090609@v.loewis.de>
	<CAG8k2+4QPcDBQ19su9faSrb+te=0+7gDpgmxNbm-30gggZHTJA@mail.gmail.com>
Message-ID: <CAG8k2+7tyVeQAb7x7tOrFqmeroXG97anQ_Z1fj8e_MMkQQjdBg@mail.gmail.com>

A few more edits at

https://bitbucket.org/dholth/python-peps/diff/pep-0426.txt?diff2=058b058ee3f8&diff1=42ee0afd40ed

rfc822 is nixed in favor of 'parseable by email.parser.Parser with the
right policy'

Encoding is strictly utf-8 with the warning that older revisions have
no defined encoding (keys are still always ASCII)

Extra names are now 'printable ascii except , [ ]' instead of 'valid
Python identifiers'.

Extension names are now 'ascii, with no whitespace or /', with the /
separator between the extension name and the rest of the tag name.

No 'the order doesn't matter' clause yet.

From lists at cheimes.de  Wed Sep  5 14:59:40 2012
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 05 Sep 2012 14:59:40 +0200
Subject: [Python-Dev] Coverity scan
In-Reply-To: <CAP1=2W6aGSO-ZbpDvz2V8pFTm4duHQBYuTSmZFkQtyac1_jz6Q@mail.gmail.com>
References: <5044B7DF.1040100@cheimes.de> <504748E5.9060704@cheimes.de>
	<CAP1=2W6aGSO-ZbpDvz2V8pFTm4duHQBYuTSmZFkQtyac1_jz6Q@mail.gmail.com>
Message-ID: <50474CBC.2060302@cheimes.de>

Am 05.09.2012 14:45, schrieb Brett Cannon:
> I have not for no other reason than I had not thought about it.

Whatever, I wasn't even sure if Stefan has contacted you or asked for a
account in a public message. He might have proclaimed his wish in a
private mail.

Christian

From stefan at bytereef.org  Wed Sep  5 15:10:06 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Wed, 5 Sep 2012 15:10:06 +0200
Subject: [Python-Dev] Coverity scan
In-Reply-To: <504748E5.9060704@cheimes.de>
References: <5044B7DF.1040100@cheimes.de>
 <504748E5.9060704@cheimes.de>
Message-ID: <20120905131006.GA13283@sleipnir.bytereef.org>

Christian Heimes <lists at cheimes.de> wrote:
> I try to get everything in place by tomorrow so we have some time to
> check for bugs before the next RC is deployed.

Fantastic. Thanks for pushing this forward!


> Stefan:
> Has Brett already requested an account for you or shall I request one
> for you?

Not yet, please do if it's no problem.


Stefan Krah



From lists at cheimes.de  Wed Sep  5 18:35:00 2012
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 05 Sep 2012 18:35:00 +0200
Subject: [Python-Dev] Coverity scan
In-Reply-To: <504748E5.9060704@cheimes.de>
References: <5044B7DF.1040100@cheimes.de> <504748E5.9060704@cheimes.de>
Message-ID: <50477F34.6050108@cheimes.de>

Am 05.09.2012 14:43, schrieb Christian Heimes:
> I try to get everything in place by tomorrow so we have some time to
> check for bugs before the next RC is deployed.

The people at Coverity are even faster than I hoped. I'm now in the
possession of the Project password which mean I can upload the builds
and add new users. I've already added Stefan and uploaded an
instrumented build successfully:

Your request for analysis of Python has been completed.  The results
should be available now in the database: http://scan5.coverity.com:8080/


Have fun!
Christian


From stefan at bytereef.org  Wed Sep  5 18:50:07 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Wed, 5 Sep 2012 18:50:07 +0200
Subject: [Python-Dev] Coverity scan
In-Reply-To: <50477F34.6050108@cheimes.de>
References: <5044B7DF.1040100@cheimes.de> <504748E5.9060704@cheimes.de>
	<50477F34.6050108@cheimes.de>
Message-ID: <20120905165007.GA16065@sleipnir.bytereef.org>

Christian Heimes <lists at cheimes.de> wrote:
> The people at Coverity are even faster than I hoped. I'm now in the
> possession of the Project password which mean I can upload the builds
> and add new users. I've already added Stefan and uploaded an
> instrumented build successfully:
> 
> Your request for analysis of Python has been completed.  The results
> should be available now in the database: http://scan5.coverity.com:8080/

Thanks Christian, works perfectly!


Stefan Krah



From brett at python.org  Wed Sep  5 18:56:42 2012
From: brett at python.org (Brett Cannon)
Date: Wed, 5 Sep 2012 12:56:42 -0400
Subject: [Python-Dev] Coverity scan
In-Reply-To: <20120905165007.GA16065@sleipnir.bytereef.org>
References: <5044B7DF.1040100@cheimes.de> <504748E5.9060704@cheimes.de>
	<50477F34.6050108@cheimes.de>
	<20120905165007.GA16065@sleipnir.bytereef.org>
Message-ID: <CAP1=2W4M-SX2Qq19a-_+vRdyoX4oTkP86nFdSiN97maEFE_82w@mail.gmail.com>

And a thanks to Christian and Stefan for picking this up and running with
it. I have not been the best keeper of this stuff as of late, but now that
Christian, Stefan, and I all have admin access to the data we can spread
the load so that none of us become a bottleneck.

On Wed, Sep 5, 2012 at 12:50 PM, Stefan Krah <stefan at bytereef.org> wrote:

> Christian Heimes <lists at cheimes.de> wrote:
> > The people at Coverity are even faster than I hoped. I'm now in the
> > possession of the Project password which mean I can upload the builds
> > and add new users. I've already added Stefan and uploaded an
> > instrumented build successfully:
> >
> > Your request for analysis of Python has been completed.  The results
> > should be available now in the database: http://scan5.coverity.com:8080/
>
> Thanks Christian, works perfectly!
>
>
> Stefan Krah
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120905/6005fe31/attachment.html>

From lists at cheimes.de  Wed Sep  5 19:35:14 2012
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 05 Sep 2012 19:35:14 +0200
Subject: [Python-Dev] Coverity scan
In-Reply-To: <CAP1=2W4M-SX2Qq19a-_+vRdyoX4oTkP86nFdSiN97maEFE_82w@mail.gmail.com>
References: <5044B7DF.1040100@cheimes.de> <504748E5.9060704@cheimes.de>
	<50477F34.6050108@cheimes.de>
	<20120905165007.GA16065@sleipnir.bytereef.org>
	<CAP1=2W4M-SX2Qq19a-_+vRdyoX4oTkP86nFdSiN97maEFE_82w@mail.gmail.com>
Message-ID: <50478D52.4070402@cheimes.de>

Am 05.09.2012 18:56, schrieb Brett Cannon:
> And a thanks to Christian and Stefan for picking this up and running
> with it. I have not been the best keeper of this stuff as of late, but
> now that Christian, Stefan, and I all have admin access to the data we
> can spread the load so that none of us become a bottleneck.

You are welcome! Sharing is caring (or so) *g*

Coverity has some new features like notification of new possible issue
and build steps. We could create a new mailing list for coverity scan
builds and results, The mailing list should be exclusive to core devs as
the issues may be security relevant.

Christian

From andrew.svetlov at gmail.com  Wed Sep  5 22:49:53 2012
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Wed, 5 Sep 2012 23:49:53 +0300
Subject: [Python-Dev] Tru64 support
In-Reply-To: <20120905001530.Horde.YtfLRElCcOxQRn2Cf-zV62A@webmail.df.eu>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
	<CAMpsgwZgQtJJxmb_-MqsXRyHby6+so5UyYT62Smy6efMy4DwCQ@mail.gmail.com>
	<CAOTb1wfciAkNLfNVyjJEBYpgcpXi_Uid3d4MFNM9mm4fX4GRJg@mail.gmail.com>
	<20120905001530.Horde.YtfLRElCcOxQRn2Cf-zV62A@webmail.df.eu>
Message-ID: <CAL3CFcWpO0XEDEsg_p8Wv+bZDF9Bynv_oRXfzTYEVa-g2Mvbyg@mail.gmail.com>

On Wed, Sep 5, 2012 at 1:15 AM,  <martin at v.loewis.de> wrote:
> 2.7 should absolutely be able to support Tru64 (even though it currently
> may not - that would be a bug). Any potential contributor to fix Tru64
> support
> should find it "easy" to add fixes.
>
> So being cautious and following existing code sounds like a good idea.
> The world wouldn't end if the test broke on Tru64, though.
>
That's ok. Chris Jerdonek is working on tuning docs for subprocess.
He wrote tests to make sure him changes are correct.
We can update documentation only for 2.7.

-- 
Thanks,
Andrew Svetlov

From tjreedy at udel.edu  Thu Sep  6 00:13:04 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 05 Sep 2012 18:13:04 -0400
Subject: [Python-Dev] Online docs: make index search available everywhere
Message-ID: <k28ipr$872$1@ger.gmane.org>

For the Windows Help version of the docs, the left side box has 4 tabs: 
Contents, Index, Search, Favorites. I now mostly use the Index tab. That 
means that I can enter an indexed keyword, topic, or object name and 
jump from place to place in the docs.

The left margin of the online docs only has a text search box equivalent 
to the Windows search tab (which I essentially never use). The index is 
only available on the front page, possibly not even visible without 
scrolling. Consequently, people, especially beginners, tend to use the 
text search box when they should be using the index. The result is often 
useless.

The issue came up on python-list about a beginner entering 'print' and 
getting nothing useful. With the Windows doc, 'print' in the index 
search immediately brings up print (builtin function) (twice, I did not 
look into why). The same thing in the search lists 153 pages, with the 
builtin functions page at 75. Not too surprising, as it is a long page 
with only 3 occurrences of 'print'. Also not useful.

So I think the online docs should also have an always available index 
search box in addition to the text search box (which should be properly 
labeled as such). 'Enter a module, class, for function name.' should be 
applied to index search, not text search.

An alternative is for the one box to give both index search results and 
text search results, but doing the latter when one only wants the former 
is a waste of both server and reader resources.

I presume this suggestion does not belong on bugs.python.org, so I hope 
one of the docs people reading pydev will do something with it.

-- 
Terry Jan Reedy


From larry at hastings.org  Thu Sep  6 00:26:03 2012
From: larry at hastings.org (Larry Hastings)
Date: Wed, 05 Sep 2012 15:26:03 -0700
Subject: [Python-Dev] Tru64 support
In-Reply-To: <20120905001530.Horde.YtfLRElCcOxQRn2Cf-zV62A@webmail.df.eu>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
	<CAMpsgwZgQtJJxmb_-MqsXRyHby6+so5UyYT62Smy6efMy4DwCQ@mail.gmail.com>
	<CAOTb1wfciAkNLfNVyjJEBYpgcpXi_Uid3d4MFNM9mm4fX4GRJg@mail.gmail.com>
	<20120905001530.Horde.YtfLRElCcOxQRn2Cf-zV62A@webmail.df.eu>
Message-ID: <5047D17B.6040608@hastings.org>


On 09/04/2012 03:15 PM, martin at v.loewis.de wrote:
> The world wouldn't end if the test broke on Tru64, though.

FWIW, HP drops support for Tru64 at the end of this year:

    http://h30097.www3.hp.com/tru64roadmap.pdf


Of course, this doesn't directly imply Python should stop supporting 
it--IIRC Python's OS/2 support outlasted IBM's support by many years.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120905/d40cb5e5/attachment.html>

From solipsis at pitrou.net  Thu Sep  6 00:55:55 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 6 Sep 2012 00:55:55 +0200
Subject: [Python-Dev] Tru64 support
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
	<CAMpsgwZgQtJJxmb_-MqsXRyHby6+so5UyYT62Smy6efMy4DwCQ@mail.gmail.com>
	<CAOTb1wfciAkNLfNVyjJEBYpgcpXi_Uid3d4MFNM9mm4fX4GRJg@mail.gmail.com>
	<20120905001530.Horde.YtfLRElCcOxQRn2Cf-zV62A@webmail.df.eu>
	<5047D17B.6040608@hastings.org>
Message-ID: <20120906005555.307584a5@pitrou.net>

On Wed, 05 Sep 2012 15:26:03 -0700
Larry Hastings <larry at hastings.org> wrote:
> 
> On 09/04/2012 03:15 PM, martin at v.loewis.de wrote:
> > The world wouldn't end if the test broke on Tru64, though.
> 
> FWIW, HP drops support for Tru64 at the end of this year:
> 
>     http://h30097.www3.hp.com/tru64roadmap.pdf
> 
> 
> Of course, this doesn't directly imply Python should stop supporting 
> it--IIRC Python's OS/2 support outlasted IBM's support by many years.

But Python for OS/2 had a dedicated maintainer.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From ncoghlan at gmail.com  Thu Sep  6 02:53:17 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 6 Sep 2012 10:53:17 +1000
Subject: [Python-Dev] Online docs: make index search available everywhere
In-Reply-To: <k28ipr$872$1@ger.gmane.org>
References: <k28ipr$872$1@ger.gmane.org>
Message-ID: <CADiSq7dqEdN3HHVd1BWPVXOBSz+nSLiBPo72dF7e0FkjGm82Qg@mail.gmail.com>

On Thu, Sep 6, 2012 at 8:13 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> I presume this suggestion does not belong on bugs.python.org, so I hope one
> of the docs people reading pydev will do something with it.

Why do you assume that? Docs infrastructure suggestions can go on the
tracker under "enhancement", the same as improvements to the docs
wording.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From martin at v.loewis.de  Thu Sep  6 08:17:24 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Thu, 06 Sep 2012 08:17:24 +0200
Subject: [Python-Dev] Tru64 support
In-Reply-To: <5047D17B.6040608@hastings.org>
References: <CAL3CFcXFVsAF6LomdX7ysuQJ0XfJgQWS8eNNVk=Q+-=8r6NFbg@mail.gmail.com>
	<CAMpsgwZgQtJJxmb_-MqsXRyHby6+so5UyYT62Smy6efMy4DwCQ@mail.gmail.com>
	<CAOTb1wfciAkNLfNVyjJEBYpgcpXi_Uid3d4MFNM9mm4fX4GRJg@mail.gmail.com>
	<20120905001530.Horde.YtfLRElCcOxQRn2Cf-zV62A@webmail.df.eu>
	<5047D17B.6040608@hastings.org>
Message-ID: <20120906081724.Horde.HAGZJaGZi1VQSD-0Os2xDfA@webmail.df.eu>


Zitat von Larry Hastings <larry at hastings.org>:

> On 09/04/2012 03:15 PM, martin at v.loewis.de wrote:
>> The world wouldn't end if the test broke on Tru64, though.
>
> FWIW, HP drops support for Tru64 at the end of this year:
>
>    http://h30097.www3.hp.com/tru64roadmap.pdf
>
>
> Of course, this doesn't directly imply Python should stop supporting  
> it--IIRC Python's OS/2 support outlasted IBM's support by many years.

It's perfectly fine (IMO) to drop support for Tru64. The main point of
PEP 11 is that this shouldn't be done ad-hoc, but by giving users advance
notice. So dropping support between bug fix releases is not ok
(but unknowingly breaking test cases for some platform where we don't
have a buildbot is ok, of course - whether the platform is little
used or not).

It's unfortunate that the presumed removal of Tru64 support in 3.0
didn't get recorded in PEP 11 (which may well be my fault as well);
it's also unfortunate that it wasn't carried out completely, so that
it's now not clear whether it was carried out at all. If 3.0 wouldn't
have worked at all on Tru64 already, removing the remains would
be a bug fix. If the removal was only documented but not implemented,
support wasn't actually removed, so users may not have received
the advance warning that configure should have spit out, in which
case removal in 3.4 may appear ad-hoc to them.

In any case, it appears that nobody ever announced removal of Tru64
support for 2.x, which means that it continues to be supported -
except that it may not actually work. But if so, we should accept
bug fixes that claim to restore support.

Regards,
Martin



From stefan at bytereef.org  Thu Sep  6 10:59:03 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Thu, 6 Sep 2012 10:59:03 +0200
Subject: [Python-Dev] Coverity scan
In-Reply-To: <50478D52.4070402@cheimes.de>
References: <5044B7DF.1040100@cheimes.de> <504748E5.9060704@cheimes.de>
	<50477F34.6050108@cheimes.de>
	<20120905165007.GA16065@sleipnir.bytereef.org>
	<CAP1=2W4M-SX2Qq19a-_+vRdyoX4oTkP86nFdSiN97maEFE_82w@mail.gmail.com>
	<50478D52.4070402@cheimes.de>
Message-ID: <20120906085903.GA23313@sleipnir.bytereef.org>

Christian Heimes <lists at cheimes.de> wrote:
> Coverity has some new features like notification of new possible issue
> and build steps. We could create a new mailing list for coverity scan
> builds and results, The mailing list should be exclusive to core devs as
> the issues may be security relevant.

The mailing list would be nice especially if we could get the results in
verbose text form, but I don't know if that's possible.


BTW, do we keep all buffer overruns secret or can we post them on the tracker
if it's an off-by-one and unlikely to be exploitable?


Stefan Krah




From tjreedy at udel.edu  Thu Sep  6 19:15:14 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 06 Sep 2012 13:15:14 -0400
Subject: [Python-Dev] Online docs: make index search available everywhere
In-Reply-To: <CADiSq7dqEdN3HHVd1BWPVXOBSz+nSLiBPo72dF7e0FkjGm82Qg@mail.gmail.com>
References: <k28ipr$872$1@ger.gmane.org>
	<CADiSq7dqEdN3HHVd1BWPVXOBSz+nSLiBPo72dF7e0FkjGm82Qg@mail.gmail.com>
Message-ID: <k2alna$7m9$1@ger.gmane.org>

On 9/5/2012 8:53 PM, Nick Coghlan wrote:
> On Thu, Sep 6, 2012 at 8:13 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>> I presume this suggestion does not belong on bugs.python.org, so I hope one
>> of the docs people reading pydev will do something with it.
>
> Why do you assume that?

My understanding is that the tracker is for the cpython repository at 
hg.python.org and my impression is that web code is elsewhere.

 > Docs infrastructure suggestions can go on the
> tracker under "enhancement", the same as improvements to the docs
> wording.

http://bugs.python.org/issue15871

-- 
Terry Jan Reedy


From rdmurray at bitdance.com  Thu Sep  6 19:26:07 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 06 Sep 2012 13:26:07 -0400
Subject: [Python-Dev] Online docs: make index search available everywhere
In-Reply-To: <k2alna$7m9$1@ger.gmane.org>
References: <k28ipr$872$1@ger.gmane.org>
	<CADiSq7dqEdN3HHVd1BWPVXOBSz+nSLiBPo72dF7e0FkjGm82Qg@mail.gmail.com>
	<k2alna$7m9$1@ger.gmane.org>
Message-ID: <20120906172607.D339D250115@webabinitio.net>

On Thu, 06 Sep 2012 13:15:14 -0400, Terry Reedy <tjreedy at udel.edu> wrote:
> On 9/5/2012 8:53 PM, Nick Coghlan wrote:
> > On Thu, Sep 6, 2012 at 8:13 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> >> I presume this suggestion does not belong on bugs.python.org, so I hope one
> >> of the docs people reading pydev will do something with it.
> >
> > Why do you assume that?
> 
> My understanding is that the tracker is for the cpython repository at 
> hg.python.org and my impression is that web code is elsewhere.

That is true.  However, the appearance and functionality of the doc web
pages is entirely controlled by Sphinx, and thus by the sources checked
in to the code repository.

--David

From lists at cheimes.de  Fri Sep  7 12:23:45 2012
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 07 Sep 2012 12:23:45 +0200
Subject: [Python-Dev] Coverity scan
In-Reply-To: <20120906085903.GA23313@sleipnir.bytereef.org>
References: <5044B7DF.1040100@cheimes.de> <504748E5.9060704@cheimes.de>
	<50477F34.6050108@cheimes.de>
	<20120905165007.GA16065@sleipnir.bytereef.org>
	<CAP1=2W4M-SX2Qq19a-_+vRdyoX4oTkP86nFdSiN97maEFE_82w@mail.gmail.com>
	<50478D52.4070402@cheimes.de>
	<20120906085903.GA23313@sleipnir.bytereef.org>
Message-ID: <5049CB31.3040402@cheimes.de>

Am 06.09.2012 10:59, schrieb Stefan Krah:
> The mailing list would be nice especially if we could get the results in
> verbose text form, but I don't know if that's possible.

I've added my account to the notification list but I've not yet received
a mail as no new issue was introduced. Coverity also sends an email for
every successful or failed build. So far the mails end up in my inbox.

> BTW, do we keep all buffer overruns secret or can we post them on the tracker
> if it's an off-by-one and unlikely to be exploitable?

I'd say use your best discretion. In the unlikely case that Coverity
finds a buffer overflow that can be abused remotely we have to go
through PSRT and publish security fix releases. At a first glance no bug
looked that severe to me.

IMHO it makes sense to define a workflow how we are going to handle
Coverity issues. Each coverity issue has an identifier and can have
information like an external reference and an action. I've seen that you
have started to create bugs in our tracker. How about we mention the
Coverity # in the bug and add a link to the bug in the "Ext. Reference"
field of the Coverity issue and set the Action to "Claimed, being worked
on".

In case you got curious about Coverity I've created a screenshot for you
http://imm.io/Duel .

Christian

From lists at cheimes.de  Fri Sep  7 13:39:47 2012
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 07 Sep 2012 13:39:47 +0200
Subject: [Python-Dev] Python 2.7: only Visual Studio 2008?
In-Reply-To: <50390517.7070300@v.loewis.de>
References: <7B27FDF2-F302-4969-A236-3D6500806007@mac.com>
	<CAD+XWwpX6PrBMTeiY04y4tyd+SaJZ5udvY_sBVZfEPs81TqA8Q@mail.gmail.com>
	<20120821170121.Horde.ekeyVruWis5QM6LBdOMjMMA@webmail.df.eu>
	<k109lh$iot$1@ger.gmane.org> <50390517.7070300@v.loewis.de>
Message-ID: <5049DD03.4010309@cheimes.de>

Hi Martin,

I'm sorry for my delayed reply, my backlog grew rather large.

Am 25.08.2012 19:02, schrieb "Martin v. L?wis":
> Interesting point. This somewhat breaks the stable ABI, which does
> include three SetFromErrno functions. So I guess we need to warn users
> of the stable ABI against using these functions.

As far as I know it's only an issue an Windows. The stable ABI
documentation should mention that mixing C runtime libraries may cause
bugs and segfaults. As a Python core developer with some experience with
Windows I'm well aware of the possible issues. Others may fall for it as
the docs lack a clear warning -- or I wasn't able to find the warning
quickly.


> A solution would then be to add an additional set of functions which
> expect errno as a parameter, although this is quite some complication.
> Another solution is to introduce a Py_errno macro (and _Py_errno
> function) which exposes Python's view of errno, so code that might
> be confronted with this issue would write
> 
>   Py_errno = errno;
> 
> before calling any of these functions.

I like the Py_errno idea. It's simple and doesn't introduce three or
more new methods. Can we still implement the feature for 3.3.0 or is it
too late?


> Anything else that you are aware of?

AFAIK each CRT has its own thread local storage. But that's not an issue
for us as we already have an API for TLS access.

Christian

From status at bugs.python.org  Fri Sep  7 18:07:17 2012
From: status at bugs.python.org (Python tracker)
Date: Fri,  7 Sep 2012 18:07:17 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20120907160717.6C7E11CA7A@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2012-08-31 - 2012-09-07)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3681 (+28)
  closed 23975 (+20)
  total  27656 (+48)

Open issues with patches: 1639 


Issues opened (40)
==================

#14223: curses addch broken on Python3.3a1
http://bugs.python.org/issue14223  reopened by Nicholas.Cole

#15833: most failures to write byte-compiled file no longer suppressed
http://bugs.python.org/issue15833  opened by sbt

#15834: 2to3 benchmark not working under Python 3
http://bugs.python.org/issue15834  opened by brett.cannon

#15835: HP-UX build needs to be tweaked to pick up PATH_MAX
http://bugs.python.org/issue15835  opened by trent

#15836: unittest assertRaises should verify excClass is actually a Bas
http://bugs.python.org/issue15836  opened by daniel.wagner-hall

#15837: Added test to test_random.py testing Random.shuffle
http://bugs.python.org/issue15837  opened by eng793

#15838: make install tries to create lib2to3 grammar pickles in source
http://bugs.python.org/issue15838  opened by ned.deily

#15840: Ambiguity with regard to the effect of accessing a closed IOBa
http://bugs.python.org/issue15840  opened by alexkon

#15842: Some SocketIO methods can succeed after close()
http://bugs.python.org/issue15842  opened by pitrou

#15844: weird import errors
http://bugs.python.org/issue15844  opened by weirdink13

#15845: Fixing some byte-to-string conversion warnings
http://bugs.python.org/issue15845  opened by eng793

#15847: parse_args stopped accepting tuples
http://bugs.python.org/issue15847  opened by zbysz

#15848: PEP 3121, 384 Refactoring applied to xxsubtype module
http://bugs.python.org/issue15848  opened by Robin.Schreiber

#15849: PEP 3121, 384 Refactoring applied to xx module
http://bugs.python.org/issue15849  opened by Robin.Schreiber

#15851: Lib/robotparser.py doesn't accept setting a user agent string,
http://bugs.python.org/issue15851  opened by dualbus

#15852: typos in curses argument error messages
http://bugs.python.org/issue15852  opened by cjerdonek

#15853: IDLE crashes selecting Preferences menu with OS X ActiveState 
http://bugs.python.org/issue15853  opened by David.Pietz

#15855: memoryview methods and data members are missing docstrings
http://bugs.python.org/issue15855  opened by belopolsky

#15856: inspect.getsource(SomeClass) doesn't show @decorators
http://bugs.python.org/issue15856  opened by takluyver

#15857: memoryview: complete support for struct packing/unpacking
http://bugs.python.org/issue15857  opened by belopolsky

#15858: tarfile missing entries due to omitted uid/gid fields
http://bugs.python.org/issue15858  opened by tlynn

#15859: PyUnicode_EncodeFSDefault win32 inconsistancy.
http://bugs.python.org/issue15859  opened by ideasman42

#15860: Use TestCase assertion methods in unittest.mock.assert* to mak
http://bugs.python.org/issue15860  opened by Julian

#15861: ttk.Treeview "unmatched open brace in list"
http://bugs.python.org/issue15861  opened by Bryan.Oakley

#15862: IDLE not working when due to wrong Hard Drive point of os.path
http://bugs.python.org/issue15862  opened by Cemal.Duman

#15863: Fine-grained info about Python versions which support changes 
http://bugs.python.org/issue15863  opened by alexkon

#15865: reflect bare star * in function signature documentation
http://bugs.python.org/issue15865  opened by cjerdonek

#15866: encode(..., 'xmlcharrefreplace') produces entities for surroga
http://bugs.python.org/issue15866  opened by wiml

#15867: make importlib documentation easier to use
http://bugs.python.org/issue15867  opened by Julian

#15868: leak in bytesio.c
http://bugs.python.org/issue15868  opened by skrah

#15869: Include .desktop file and icon
http://bugs.python.org/issue15869  opened by natureshadow

#15870: PyType_FromSpec should take metaclass as an argument
http://bugs.python.org/issue15870  opened by belopolsky

#15871: Online docs: make index search always available.
http://bugs.python.org/issue15871  opened by terry.reedy

#15872: shutil.rmtree(..., ignore_errors=True) doesn't ignore all erro
http://bugs.python.org/issue15872  opened by jwilk

#15873: "datetime" cannot parse ISO 8601 dates and times
http://bugs.python.org/issue15873  opened by nagle

#15874: argparse cannot parse shell variable arguments in file-given a
http://bugs.python.org/issue15874  opened by ZhuangZi

#15875: tarfile may not make @LongLink for non-ascii character
http://bugs.python.org/issue15875  opened by Manuke

#15877: xml.dom.minidom cannot parse ISO-2022-JP
http://bugs.python.org/issue15877  opened by dcallagh

#15879: set.__or__, __and__, etc create subclass types, but ignore __n
http://bugs.python.org/issue15879  opened by Jon.Obermark

#1528167: Tweak to make string.Templates more customizable
http://bugs.python.org/issue1528167  reopened by r.david.murray



Most recent 15 issues with no replies (15)
==========================================

#15879: set.__or__, __and__, etc create subclass types, but ignore __n
http://bugs.python.org/issue15879

#15875: tarfile may not make @LongLink for non-ascii character
http://bugs.python.org/issue15875

#15869: Include .desktop file and icon
http://bugs.python.org/issue15869

#15860: Use TestCase assertion methods in unittest.mock.assert* to mak
http://bugs.python.org/issue15860

#15856: inspect.getsource(SomeClass) doesn't show @decorators
http://bugs.python.org/issue15856

#15848: PEP 3121, 384 Refactoring applied to xxsubtype module
http://bugs.python.org/issue15848

#15835: HP-UX build needs to be tweaked to pick up PATH_MAX
http://bugs.python.org/issue15835

#15818: multiprocessing documentation of Process.exitcode
http://bugs.python.org/issue15818

#15817: Misc/gdbinit: Expose command documentation to gdb help
http://bugs.python.org/issue15817

#15812: inspect.getframeinfo() cannot show first line
http://bugs.python.org/issue15812

#15797: bdist_msi does not pass -install/remove flags to install_scrip
http://bugs.python.org/issue15797

#15796: Fix readline() docstrings
http://bugs.python.org/issue15796

#15782: Compile error for a number of Mac modules with recent Xcode
http://bugs.python.org/issue15782

#15772: Unresolved symbols in Windows 64-bit python
http://bugs.python.org/issue15772

#15767: add ModuleNotFoundError
http://bugs.python.org/issue15767



Most recent 15 issues waiting for review (15)
=============================================

#15873: "datetime" cannot parse ISO 8601 dates and times
http://bugs.python.org/issue15873

#15872: shutil.rmtree(..., ignore_errors=True) doesn't ignore all erro
http://bugs.python.org/issue15872

#15870: PyType_FromSpec should take metaclass as an argument
http://bugs.python.org/issue15870

#15868: leak in bytesio.c
http://bugs.python.org/issue15868

#15866: encode(..., 'xmlcharrefreplace') produces entities for surroga
http://bugs.python.org/issue15866

#15865: reflect bare star * in function signature documentation
http://bugs.python.org/issue15865

#15859: PyUnicode_EncodeFSDefault win32 inconsistancy.
http://bugs.python.org/issue15859

#15855: memoryview methods and data members are missing docstrings
http://bugs.python.org/issue15855

#15852: typos in curses argument error messages
http://bugs.python.org/issue15852

#15851: Lib/robotparser.py doesn't accept setting a user agent string,
http://bugs.python.org/issue15851

#15849: PEP 3121, 384 Refactoring applied to xx module
http://bugs.python.org/issue15849

#15848: PEP 3121, 384 Refactoring applied to xxsubtype module
http://bugs.python.org/issue15848

#15847: parse_args stopped accepting tuples
http://bugs.python.org/issue15847

#15845: Fixing some byte-to-string conversion warnings
http://bugs.python.org/issue15845

#15842: Some SocketIO methods can succeed after close()
http://bugs.python.org/issue15842



Top 10 most discussed issues (10)
=================================

#15814: memoryview: equality-hash invariant
http://bugs.python.org/issue15814  26 msgs

#15831: comma after leading optional argument is after bracket in docs
http://bugs.python.org/issue15831  24 msgs

#15836: unittest assertRaises should verify excClass is actually a Bas
http://bugs.python.org/issue15836  12 msgs

#15837: Added test to test_random.py testing Random.shuffle
http://bugs.python.org/issue15837  12 msgs

#15855: memoryview methods and data members are missing docstrings
http://bugs.python.org/issue15855  12 msgs

#14223: curses addch broken on Python3.3a1
http://bugs.python.org/issue14223  11 msgs

#15533: subprocess.Popen(cwd) documentation
http://bugs.python.org/issue15533  10 msgs

#15599: test_circular_imports() of test_threaded_import fails on FreeB
http://bugs.python.org/issue15599  10 msgs

#15870: PyType_FromSpec should take metaclass as an argument
http://bugs.python.org/issue15870  10 msgs

#11549: Build-out an AST optimizer, moving some functionality out of t
http://bugs.python.org/issue11549   9 msgs



Issues closed (21)
==================

#7652: Merge C version of decimal into py3k.
http://bugs.python.org/issue7652  closed by skrah

#12776: argparse: type conversion function should be called only once
http://bugs.python.org/issue12776  closed by r.david.murray

#13769: json.dump(ensure_ascii=False) return str instead of unicode
http://bugs.python.org/issue13769  closed by petri.lehtinen

#14922: mailbox.Maildir.get_message() may fail when Maildir dirname is
http://bugs.python.org/issue14922  closed by petri.lehtinen

#15447: A file is not properly closed by webbrowser._invoke
http://bugs.python.org/issue15447  closed by r.david.murray

#15509: UnixBrowser.open sometimes passes zero-length argument to the 
http://bugs.python.org/issue15509  closed by r.david.murray

#15520: Document datetime.timestamp() in 3.3 What's New
http://bugs.python.org/issue15520  closed by belopolsky

#15557: Tests for webbrowser module
http://bugs.python.org/issue15557  closed by r.david.murray

#15802: Nonsensical test for mailbox
http://bugs.python.org/issue15802  closed by petri.lehtinen

#15810: assertSequenceEqual should be fired when comparing sequence su
http://bugs.python.org/issue15810  closed by gregory.p.smith

#15830: make -s no longer silences output from setup.py
http://bugs.python.org/issue15830  closed by doko

#15832: argparse: typechecks default value too early
http://bugs.python.org/issue15832  closed by bethard

#15839: SystemError raised by super() in erronous class should be Name
http://bugs.python.org/issue15839  closed by python-dev

#15841: Some StringIO and BytesIO methods can succeed after close
http://bugs.python.org/issue15841  closed by pitrou

#15843: aifc.open expects only str or file pointer
http://bugs.python.org/issue15843  closed by petri.lehtinen

#15846: literal_eval raises SystemError instead of SyntaxError, becaus
http://bugs.python.org/issue15846  closed by python-dev

#15850: list.count() results wrong if both booleans & 1's/0's are pres
http://bugs.python.org/issue15850  closed by alex

#15854: curses test_curses.test_unget_wch crash
http://bugs.python.org/issue15854  closed by ned.deily

#15864: Package cache doesn't cache packages with overlapping sys.path
http://bugs.python.org/issue15864  closed by r.david.murray

#15876: test_curses refleak
http://bugs.python.org/issue15876  closed by rosslagerwall

#15878: struct long type size
http://bugs.python.org/issue15878  closed by mark.dickinson

From erik.m.bray at gmail.com  Fri Sep  7 20:29:42 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Fri, 7 Sep 2012 14:29:42 -0400
Subject: [Python-Dev] Edits to Metadata 1.2 to add extras (optional
	dependencies)
In-Reply-To: <CAG8k2+5BE=a6y1yJ8Beh=CmroZSJ3Jx8W+BzmkZHqoB6dU5_bw@mail.gmail.com>
References: <CAG8k2+5WyDyChg+FX6_2w0_1N5uX6DCMtM+HnTQzjQxjCTTHSQ@mail.gmail.com>
	<CAG8k2+5BE=a6y1yJ8Beh=CmroZSJ3Jx8W+BzmkZHqoB6dU5_bw@mail.gmail.com>
Message-ID: <CAOTD34akAcFyL3VoMomziHtQEzp9jcSwdndi6kkP00GHoykcew@mail.gmail.com>

On Mon, Aug 27, 2012 at 10:56 AM, Daniel Holth <dholth at gmail.com> wrote:
> On Wed, Aug 15, 2012 at 10:49 AM, Daniel Holth <dholth at gmail.com> wrote:
>> I've drafted some edits to Metadata 1.2 with valuable feedback from
> ...
>> (full changeset on https://bitbucket.org/dholth/python-peps/changeset/537e83bd4068)
>
> Metadata 1.2 is nearly 8 years old and it's Accepted but not Final. Is
> it better to continue editing it, or create a new PEP for Metadata
> 1.3?

Somehow I completely overlooked this thread until now.  Thanks Daniel
for getting the ball rolling on this.  There have already been many
bytes spilled on metadata extensions, and although I agree it would be
enormously useful to build an extension mechanism into the metadata
format, I don't have much riding on that, or much more to add that
hasn't been said.

There hasn't been much said about Setup-Requires-Dist, so I'm guessing
it's uncontroversial.  But since that's sort of my hobbyhorse I
thought I would make a comment on it.  The thing I love about the
Setup-Requires-Dist feature is that, if properly supported by
different installers, it can free those installers from a fair bit of
responsibility.

For example, in greatly simplifies the thorny issue of "compilers".
The existing compiler support in distutils, while not without its
problems, does work in most cases for building common C-extensions.
distutils2 has already made some progress on cleaning up the interface
for compilers, and making it easier to register new compiler classes
that can be imported from an arbitrary package.  This allows projects
with special needs (such as Fortran compiler support) to ship their
own compiler class with the project.  Or if there's a good enough
third-party package that provides Fortran compiler support, projects
may use it in their build process.  Support for Setup-Requires-Dist
ensures that a third-party compiler package can be made available at
build-time.

What's great about this, is that even if the stdlib still includes a
build system, it doesn't necessarily have to anticipate every possible
need for building every kind of project (it should, at a minimum, be
able to build pure-Python projects).  If someone wants to add MSVC2012
support they can do that as a third-party package. One could even
create "compilers" for other build systems like waf, or even provide
an entry-point to meta-build systems like bento.  Am I making sense?

Erik

From chris.jerdonek at gmail.com  Sat Sep  8 07:05:29 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Fri, 7 Sep 2012 22:05:29 -0700
Subject: [Python-Dev] core dev IRC nicks
In-Reply-To: <CADiSq7eJ3Y=_LHb1xJ=pJAg9t=VMKFR93YB5AEjJ+YCeWSfzBg@mail.gmail.com>
References: <CAOTb1weKPETpA5XdD8y2Oj_7Pg4BFLmtvS-F-ztAo9rdXdzwSQ@mail.gmail.com>
	<CADiSq7eJ3Y=_LHb1xJ=pJAg9t=VMKFR93YB5AEjJ+YCeWSfzBg@mail.gmail.com>
Message-ID: <CAOTb1wcoWyaa5v=tP2q=tqb+N0mgjBM7bBHacUqo8EG5aKicXQ@mail.gmail.com>

On Tue, Aug 28, 2012 at 9:31 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Wed, Aug 29, 2012 at 1:58 PM, Chris Jerdonek
> <chris.jerdonek at gmail.com> wrote:
>> Is there a list somewhere of the IRC nicks of the core developers that
>> use IRC (and who wish to be listed) alongside their real names?  If
>> there is no such list, has there ever been discussion on python-dev of
>> creating such a list?
>
> No, to all those questions. It's not a bad idea, though.
>
> One possibility (if someone was willing to work on it) would be to
> enhance our Roundup instance to handle it. (See
> http://docs.python.org/devguide/tracker.html#the-meta-tracker)
>
> Allow users to add their typical IRC handle, and perhaps add a
> "Committers List" link below the existing "Users List" link in the
> sidebar.

I created an issue for the latter here:

http://psf.upfronthosting.co.za/roundup/meta/issue479

--Chris

From roundup-admin at psf.upfronthosting.co.za  Sat Sep  8 11:26:32 2012
From: roundup-admin at psf.upfronthosting.co.za (Python tracker)
Date: Sat, 08 Sep 2012 09:26:32 +0000
Subject: [Python-Dev] Failed issue tracker submission
Message-ID: <20120908092632.99CA91CB3E@psf.upfronthosting.co.za>



The node specified by the designator in the subject of your message
("715365") does not exist.

Subject was: "[issue715365]"



Mail Gateway Help
=================
Incoming messages are examined for multiple parts:
 . In a multipart/mixed message or part, each subpart is extracted and
   examined. The text/plain subparts are assembled to form the textual
   body of the message, to be stored in the file associated with a "msg"
   class node. Any parts of other types are each stored in separate files
   and given "file" class nodes that are linked to the "msg" node.
 . In a multipart/alternative message or part, we look for a text/plain
   subpart and ignore the other parts.
 . A message/rfc822 is treated similar tomultipart/mixed (except for
   special handling of the first text part) if unpack_rfc822 is set in
   the mailgw config section.

Summary
-------
The "summary" property on message nodes is taken from the first non-quoting
section in the message body. The message body is divided into sections by
blank lines. Sections where the second and all subsequent lines begin with
a ">" or "|" character are considered "quoting sections". The first line of
the first non-quoting section becomes the summary of the message.

Addresses
---------
All of the addresses in the To: and Cc: headers of the incoming message are
looked up among the user nodes, and the corresponding users are placed in
the "recipients" property on the new "msg" node. The address in the From:
header similarly determines the "author" property of the new "msg"
node. The default handling for addresses that don't have corresponding
users is to create new users with no passwords and a username equal to the
address. (The web interface does not permit logins for users with no
passwords.) If we prefer to reject mail from outside sources, we can simply
register an auditor on the "user" class that prevents the creation of user
nodes with no passwords.

Actions
-------
The subject line of the incoming message is examined to determine whether
the message is an attempt to create a new item or to discuss an existing
item. A designator enclosed in square brackets is sought as the first thing
on the subject line (after skipping any "Fwd:" or "Re:" prefixes).

If an item designator (class name and id number) is found there, the newly
created "msg" node is added to the "messages" property for that item, and
any new "file" nodes are added to the "files" property for the item.

If just an item class name is found there, we attempt to create a new item
of that class with its "messages" property initialized to contain the new
"msg" node and its "files" property initialized to contain any new "file"
nodes.

Triggers
--------
Both cases may trigger detectors (in the first case we are calling the
set() method to add the message to the item's spool; in the second case we
are calling the create() method to create a new node). If an auditor raises
an exception, the original message is bounced back to the sender with the
explanatory message given in the exception.
-------------- next part --------------
Return-Path: <python-dev at python.org>
X-Original-To: report at bugs.python.org
Delivered-To: roundup+tracker at psf.upfronthosting.co.za
Received: from mail.python.org (mail.python.org [IPv6:2001:888:2000:d::a6])
	by psf.upfronthosting.co.za (Postfix) with ESMTPS id 537811C9D8
	for <report at bugs.python.org>; Sat,  8 Sep 2012 11:26:32 +0200 (CEST)
Received: from albatross.python.org (localhost [127.0.0.1])
	by mail.python.org (Postfix) with ESMTP id 3XDVWr0bDqzQ7Z
	for <report at bugs.python.org>; Sat,  8 Sep 2012 11:26:32 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=python.org; s=200901;
	t=1347096392; bh=6dc5mdv4dzL5yW3ZVn8qW7XJ3IHvTCmeY0s1ykUK9k4=;
	h=MIME-Version:Content-Type:Content-Transfer-Encoding:From:To:
	 Subject:Message-Id:Date;
	b=D8YO9tFWN0XXGgKQYxrCFD/5gAjlPqoCT/u8iuZcrn6ELcSY0XjT6d61kJRULvCyH
	 EHauP0hVfuZ0DyFuxmGcqROWZgVbe2kV2G2H7hUnRUgoVPp+YdWOjbLt0qVV+1Deff
	 nstshu+2MVQFJFr/+lskdt5QvARUw+WEKRFjvCA8=
Received: from localhost (HELO mail.python.org) (127.0.0.1)
  by albatross.python.org with SMTP; 08 Sep 2012 11:26:32 +0200
Received: from virt-7yvsjn.psf.osuosl.org (virt-7yvsjn.psf.osuosl.org [140.211.10.72])
	by mail.python.org (Postfix) with ESMTP
	for <report at bugs.python.org>; Sat,  8 Sep 2012 11:26:31 +0200 (CEST)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
From: python-dev at python.org
To: report at bugs.python.org
Subject: [issue715365]
Message-Id: <3XDVWr0bDqzQ7Z at mail.python.org>
Date: Sat,  8 Sep 2012 11:26:32 +0200 (CEST)

TmV3IGNoYW5nZXNldCBkZjY2MzYwM2U2N2EgYnkgU3RlZmFuIEtyYWggaW4gYnJhbmNoICczLjIn
OgpJc3N1ZSAjMTU4Njg6IEZpeCByZWZsZWFrIGluIGJ5dGVzaW8uYyAoQ292ZXJpdHkgIzcxNTM2
NSkuCmh0dHA6Ly9oZy5weXRob24ub3JnL2NweXRob24vcmV2L2RmNjYzNjAzZTY3YQo=

From stefan at bytereef.org  Sat Sep  8 11:35:16 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Sat, 8 Sep 2012 11:35:16 +0200
Subject: [Python-Dev] Coverity scan
In-Reply-To: <5049CB31.3040402@cheimes.de>
References: <5044B7DF.1040100@cheimes.de> <504748E5.9060704@cheimes.de>
	<50477F34.6050108@cheimes.de>
	<20120905165007.GA16065@sleipnir.bytereef.org>
	<CAP1=2W4M-SX2Qq19a-_+vRdyoX4oTkP86nFdSiN97maEFE_82w@mail.gmail.com>
	<50478D52.4070402@cheimes.de>
	<20120906085903.GA23313@sleipnir.bytereef.org>
	<5049CB31.3040402@cheimes.de>
Message-ID: <20120908093516.GA27512@sleipnir.bytereef.org>

Christian Heimes <lists at cheimes.de> wrote:
> IMHO it makes sense to define a workflow how we are going to handle
> Coverity issues. Each coverity issue has an identifier and can have
> information like an external reference and an action. I've seen that you
> have started to create bugs in our tracker. How about we mention the
> Coverity # in the bug and add a link to the bug in the "Ext. Reference"
> field of the Coverity issue and set the Action to "Claimed, being worked
> on".

That sounds good in principle. I'm only worried that for casual readers
of either the commit messages or the tracker issues the importance of
the Coverity tool might be overstated.

After all, 99.99% of issues are either found by developers themselves or
by gcc, Visual Studio, Valgrind, etc. It just occurred to me that for example
we don't credit other tools in commit messages.


That said, for users of the Coverity web interface it's clearly useful.


Stefan Krah



From martin at v.loewis.de  Sat Sep  8 12:41:46 2012
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Sat, 08 Sep 2012 12:41:46 +0200
Subject: [Python-Dev] Coverity scan
In-Reply-To: <20120908093516.GA27512@sleipnir.bytereef.org>
References: <5044B7DF.1040100@cheimes.de> <504748E5.9060704@cheimes.de>
	<50477F34.6050108@cheimes.de>
	<20120905165007.GA16065@sleipnir.bytereef.org>
	<CAP1=2W4M-SX2Qq19a-_+vRdyoX4oTkP86nFdSiN97maEFE_82w@mail.gmail.com>
	<50478D52.4070402@cheimes.de>
	<20120906085903.GA23313@sleipnir.bytereef.org>
	<5049CB31.3040402@cheimes.de>
	<20120908093516.GA27512@sleipnir.bytereef.org>
Message-ID: <20120908124146.Horde.wRK0cLuWis5QSyDqdM7jx-A@webmail.df.eu>


Zitat von Stefan Krah <stefan at bytereef.org>:
> After all, 99.99% of issues are either found by developers themselves or
> by gcc, Visual Studio, Valgrind, etc. It just occurred to me that for example
> we don't credit other tools in commit messages.

I agree that Coverity doesn't need to be mentioned in commit message.
We do cite tools occasionally, but in a negative way, such as "silence
gcc warning", where the commit message and/or comment explains why some
code is ugly for some non-obvious reason.

Regards,
Martin



From petri at digip.org  Sat Sep  8 13:11:11 2012
From: petri at digip.org (Petri Lehtinen)
Date: Sat, 8 Sep 2012 14:11:11 +0300
Subject: [Python-Dev] Failed issue tracker submission
In-Reply-To: <20120908092632.99CA91CB3E@psf.upfronthosting.co.za>
References: <20120908092632.99CA91CB3E@psf.upfronthosting.co.za>
Message-ID: <20120908111111.GA9778@toronto>

Python tracker wrote:
> 
> 
> The node specified by the designator in the subject of your message
> ("715365") does not exist.
> 
> Subject was: "[issue715365]"

Is this related to the Coverity ID being mentioned in
http://bugs.python.org/issue15868 ?

From stefan at bytereef.org  Sat Sep  8 13:17:41 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Sat, 8 Sep 2012 13:17:41 +0200
Subject: [Python-Dev] Failed issue tracker submission
In-Reply-To: <20120908111111.GA9778@toronto>
References: <20120908092632.99CA91CB3E@psf.upfronthosting.co.za>
	<20120908111111.GA9778@toronto>
Message-ID: <20120908111741.GA28878@sleipnir.bytereef.org>

Petri Lehtinen <petri at digip.org> wrote:
> > ("715365") does not exist.
>
> Is this related to the Coverity ID being mentioned in
> http://bugs.python.org/issue15868 ?

Must be. I used the pattern #<number> twice in the commit message:

http://hg.python.org/cpython/rev/df663603e67a


So let's avoid that in the future.


Stefan Krah



From ncoghlan at gmail.com  Sat Sep  8 16:56:46 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 9 Sep 2012 00:56:46 +1000
Subject: [Python-Dev] Failed issue tracker submission
In-Reply-To: <20120908111741.GA28878@sleipnir.bytereef.org>
References: <20120908092632.99CA91CB3E@psf.upfronthosting.co.za>
	<20120908111111.GA9778@toronto>
	<20120908111741.GA28878@sleipnir.bytereef.org>
Message-ID: <CADiSq7coM=9yQvn4Nn5xFq+ZWDVpO7_iPSO+COdcnrX85UvEvA@mail.gmail.com>

On Sat, Sep 8, 2012 at 9:17 PM, Stefan Krah <stefan at bytereef.org> wrote:
> Petri Lehtinen <petri at digip.org> wrote:
>> > ("715365") does not exist.
>>
>> Is this related to the Coverity ID being mentioned in
>> http://bugs.python.org/issue15868 ?
>
> Must be. I used the pattern #<number> twice in the commit message:
>
> http://hg.python.org/cpython/rev/df663603e67a

No, using the pattern number twice is OK now - it's using it for an
issue that *doesn't exist* that broke.

Better to just not use that notation for the Coverity ID's by writing
"CID 715365" or something similar so we don't get any false hits on
the tracker integration.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From solipsis at pitrou.net  Sat Sep  8 16:52:54 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 8 Sep 2012 16:52:54 +0200
Subject: [Python-Dev] cpython (2.7): Issue #15591 and Issue #11715:
 silence output of setup.py when make is run with
References: <3XCfcF2gx1zQLL@mail.python.org>
Message-ID: <20120908165254.4c18be51@pitrou.net>

On Fri,  7 Sep 2012 02:27:25 +0200 (CEST)
christian.heimes <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/8cd6acffbcb9
> changeset:   78871:8cd6acffbcb9
> branch:      2.7
> parent:      78867:2587aeb616b6
> user:        Christian Heimes <christian at cheimes.de>
> date:        Fri Sep 07 00:55:33 2012 +0200
> summary:
>   Issue #15591 and Issue #11715: silence output of setup.py when make is run with -s option.
> 
> files:

Looks like you committed an empty changeset, as a result the 2.7
FreeBSD buildbot is still broken:
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%202.7

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From brett at python.org  Sat Sep  8 18:24:18 2012
From: brett at python.org (Brett Cannon)
Date: Sat, 8 Sep 2012 12:24:18 -0400
Subject: [Python-Dev] Coverity scan
In-Reply-To: <20120908124146.Horde.wRK0cLuWis5QSyDqdM7jx-A@webmail.df.eu>
References: <5044B7DF.1040100@cheimes.de> <504748E5.9060704@cheimes.de>
	<50477F34.6050108@cheimes.de>
	<20120905165007.GA16065@sleipnir.bytereef.org>
	<CAP1=2W4M-SX2Qq19a-_+vRdyoX4oTkP86nFdSiN97maEFE_82w@mail.gmail.com>
	<50478D52.4070402@cheimes.de>
	<20120906085903.GA23313@sleipnir.bytereef.org>
	<5049CB31.3040402@cheimes.de>
	<20120908093516.GA27512@sleipnir.bytereef.org>
	<20120908124146.Horde.wRK0cLuWis5QSyDqdM7jx-A@webmail.df.eu>
Message-ID: <CAP1=2W4gAypxjjxdZyp0uiAOU9ksCr9W5xzRrLR6homaCwHK7w@mail.gmail.com>

On Sat, Sep 8, 2012 at 6:41 AM, <martin at v.loewis.de> wrote:

>
> Zitat von Stefan Krah <stefan at bytereef.org>:
>
>  After all, 99.99% of issues are either found by developers themselves or
>> by gcc, Visual Studio, Valgrind, etc. It just occurred to me that for
>> example
>> we don't credit other tools in commit messages.
>>
>
> I agree that Coverity doesn't need to be mentioned in commit message.
> We do cite tools occasionally, but in a negative way, such as "silence
> gcc warning", where the commit message and/or comment explains why some
> code is ugly for some non-obvious reason.
>

Well, when I fix bugs found by Clang's static analyzer (which I ran out
time to do for 3.3) I try to remember to thank the tool. IOW I don't think
thanking a tool hurts, but in no way is required required either.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120908/ceacf652/attachment.html>

From lists at cheimes.de  Sat Sep  8 18:50:13 2012
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 08 Sep 2012 18:50:13 +0200
Subject: [Python-Dev] Coverity scan
In-Reply-To: <20120908093516.GA27512@sleipnir.bytereef.org>
References: <5044B7DF.1040100@cheimes.de> <504748E5.9060704@cheimes.de>
	<50477F34.6050108@cheimes.de>
	<20120905165007.GA16065@sleipnir.bytereef.org>
	<CAP1=2W4M-SX2Qq19a-_+vRdyoX4oTkP86nFdSiN97maEFE_82w@mail.gmail.com>
	<50478D52.4070402@cheimes.de>
	<20120906085903.GA23313@sleipnir.bytereef.org>
	<5049CB31.3040402@cheimes.de>
	<20120908093516.GA27512@sleipnir.bytereef.org>
Message-ID: <504B7745.1030408@cheimes.de>

Am 08.09.2012 11:35, schrieb Stefan Krah:
> That sounds good in principle. I'm only worried that for casual readers
> of either the commit messages or the tracker issues the importance of
> the Coverity tool might be overstated.
> 
> After all, 99.99% of issues are either found by developers themselves or
> by gcc, Visual Studio, Valgrind, etc. It just occurred to me that for example
> we don't credit other tools in commit messages.

I'd like to avoid that two people create two bug tracker entries or work
on the same issue at the same time. The CID (coverity id) also makes it
easier to find the entry on the Coverity site. IMHO it's sufficient to
mention the CID in the tracker entry. As Brett has said it's also nice
to give credits, too.

By the way I've automated the build and upload process. Every six hours
the default branch is pulled from hg and a build is triggere when
changes are detected.

Christian



From lists at cheimes.de  Sat Sep  8 19:21:46 2012
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 08 Sep 2012 19:21:46 +0200
Subject: [Python-Dev] cpython (2.7): Issue #15591 and Issue #11715:
 silence output of setup.py when make is run with
In-Reply-To: <20120908165254.4c18be51@pitrou.net>
References: <3XCfcF2gx1zQLL@mail.python.org>
	<20120908165254.4c18be51@pitrou.net>
Message-ID: <504B7EAA.5040107@cheimes.de>

Am 08.09.2012 16:52, schrieb Antoine Pitrou:
> On Fri,  7 Sep 2012 02:27:25 +0200 (CEST)
> christian.heimes <python-checkins at python.org> wrote:
>> http://hg.python.org/cpython/rev/8cd6acffbcb9
>> changeset:   78871:8cd6acffbcb9
>> branch:      2.7
>> parent:      78867:2587aeb616b6
>> user:        Christian Heimes <christian at cheimes.de>
>> date:        Fri Sep 07 00:55:33 2012 +0200
>> summary:
>>   Issue #15591 and Issue #11715: silence output of setup.py when make is run with -s option.
>>
>> files:
> 
> Looks like you committed an empty changeset, as a result the 2.7
> FreeBSD buildbot is still broken:
> http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%202.7

Thanks! I took care of the problem.

The patch somehow got stuck in my outgoing queue. I'm still a bit
uncertain how to handle edge cases. #python-dev is really helpful.


From chris.jerdonek at gmail.com  Sat Sep  8 22:18:05 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sat, 8 Sep 2012 13:18:05 -0700
Subject: [Python-Dev] Online docs: make index search available everywhere
In-Reply-To: <20120906172607.D339D250115@webabinitio.net>
References: <k28ipr$872$1@ger.gmane.org>
	<CADiSq7dqEdN3HHVd1BWPVXOBSz+nSLiBPo72dF7e0FkjGm82Qg@mail.gmail.com>
	<k2alna$7m9$1@ger.gmane.org>
	<20120906172607.D339D250115@webabinitio.net>
Message-ID: <CAOTb1wesRZ2zaGs+AaCcAJEj5TrSoFffw_WZKt_Z+bPbMuEn+w@mail.gmail.com>

On Thu, Sep 6, 2012 at 10:26 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Thu, 06 Sep 2012 13:15:14 -0400, Terry Reedy <tjreedy at udel.edu> wrote:
>> My understanding is that the tracker is for the cpython repository at
>> hg.python.org and my impression is that web code is elsewhere.
>
> That is true.  However, the appearance and functionality of the doc web
> pages is entirely controlled by Sphinx, and thus by the sources checked
> in to the code repository.

This raises a related question.  Where should we file issues for
Python-controlled things that are outside the scope of the cpython
repository (or the devguide) and that are not issues with the tracker
(which has its own tracker -- the meta tracker).

For example, where should we file an issue for hg.python.org if we
suspect the issue is with how we've customized things rather than with
Mercurial's hgweb itself.  The main bugs tracker has a component for
"Devguide" but not for other non-cpython repository things like
hg.python.org.

--Chris

From ncoghlan at gmail.com  Sun Sep  9 05:16:16 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 9 Sep 2012 13:16:16 +1000
Subject: [Python-Dev] Online docs: make index search available everywhere
In-Reply-To: <CAOTb1wesRZ2zaGs+AaCcAJEj5TrSoFffw_WZKt_Z+bPbMuEn+w@mail.gmail.com>
References: <k28ipr$872$1@ger.gmane.org>
	<CADiSq7dqEdN3HHVd1BWPVXOBSz+nSLiBPo72dF7e0FkjGm82Qg@mail.gmail.com>
	<k2alna$7m9$1@ger.gmane.org>
	<20120906172607.D339D250115@webabinitio.net>
	<CAOTb1wesRZ2zaGs+AaCcAJEj5TrSoFffw_WZKt_Z+bPbMuEn+w@mail.gmail.com>
Message-ID: <CADiSq7caoG=GR_3tsbMyyW2QDsrO_m=yS_f4P2inZizctyb9Rw@mail.gmail.com>

On Sun, Sep 9, 2012 at 6:18 AM, Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
> This raises a related question.  Where should we file issues for
> Python-controlled things that are outside the scope of the cpython
> repository (or the devguide) and that are not issues with the tracker
> (which has its own tracker -- the meta tracker).
>
> For example, where should we file an issue for hg.python.org if we
> suspect the issue is with how we've customized things rather than with
> Mercurial's hgweb itself.  The main bugs tracker has a component for
> "Devguide" but not for other non-cpython repository things like
> hg.python.org.

As far as I know, we don't have a general ticketing system for
python.org infrastructure. That may change at some point in the
future, but in the meantime I just email python-committers or
python-dev if I notice something that appears broken with the
development infrastructure, or pydotorg-www for the website in
general.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From chris.jerdonek at gmail.com  Sun Sep  9 06:49:55 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sat, 8 Sep 2012 21:49:55 -0700
Subject: [Python-Dev] Online docs: make index search available everywhere
In-Reply-To: <CADiSq7caoG=GR_3tsbMyyW2QDsrO_m=yS_f4P2inZizctyb9Rw@mail.gmail.com>
References: <k28ipr$872$1@ger.gmane.org>
	<CADiSq7dqEdN3HHVd1BWPVXOBSz+nSLiBPo72dF7e0FkjGm82Qg@mail.gmail.com>
	<k2alna$7m9$1@ger.gmane.org>
	<20120906172607.D339D250115@webabinitio.net>
	<CAOTb1wesRZ2zaGs+AaCcAJEj5TrSoFffw_WZKt_Z+bPbMuEn+w@mail.gmail.com>
	<CADiSq7caoG=GR_3tsbMyyW2QDsrO_m=yS_f4P2inZizctyb9Rw@mail.gmail.com>
Message-ID: <-1317696924686962704@unknownmsgid>

On Sep 8, 2012, at 8:16 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Sun, Sep 9, 2012 at 6:18 AM, Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
>> This raises a related question.  Where should we file issues for
>> Python-controlled things that are outside the scope of the cpython
>> repository (or the devguide) and that are not issues with the tracker
>> (which has its own tracker -- the meta tracker).
>>
>> For example, where should we file an issue for hg.python.org if we
>> suspect the issue is with how we've customized things rather than with
>> Mercurial's hgweb itself.  The main bugs tracker has a component for
>> "Devguide" but not for other non-cpython repository things like
>> hg.python.org.
>
> As far as I know, we don't have a general ticketing system for
> python.org infrastructure. That may change at some point in the
> future, but in the meantime I just email python-committers or
> python-dev if I notice something that appears broken with the
> development infrastructure, or pydotorg-www for the website in
> general.

In the absence of a tracker for "other" Python development issues, why
not allow such issues to be filed with the main tracker in the
meantime?  Otherwise, there is no clear way to record or track
progress made on those issues.

In the meantime, I will post my issue to python-dev.

--Chris

Sent from my iPhone

From chris.jerdonek at gmail.com  Sun Sep  9 07:29:39 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sat, 8 Sep 2012 22:29:39 -0700
Subject: [Python-Dev] hg.python.org issue: log page entries don't always
	link to revision
Message-ID: <CAOTb1wfncycR5dimOsQHwDdupi65ioh3eDBrD=XJEkASXB4TrA@mail.gmail.com>

On hg.python.org, it seems like the entries on the "log" page don't
always link to the corresponding revision:

http://hg.python.org/cpython/shortlog

This seems to happen whenever the revision description begins with
text that results in a link to something else (e.g. an issue number),
for example:

"Issue #15822: Fix installation of lib2to3 grammar pickles to ensure."

In contrast, the "graph" page seems always to link to the revision:

http://hg.python.org/cpython/graph

Related to this, if the description field contains text that results
in a link to something else, then the UI doesn't make a distinction
between the portion of the description that links to the revision and
the portion that links to the something else.

For example, with description text "Fix for fcc629208842", "Fix for"
links to the revision for the log entry, but "fcc629208842" links to
the named revision, and there is no separation or visual indicator
that the two portions of text link to different things.

It might be better if the revision link was separate from the
description text.  That would be one way to address both of the issues
above.

--Chris

From chris.jerdonek at gmail.com  Sun Sep  9 07:33:02 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sat, 8 Sep 2012 22:33:02 -0700
Subject: [Python-Dev] hg.python.org issue: log page entries don't always
	link to revision
In-Reply-To: <CAOTb1wfncycR5dimOsQHwDdupi65ioh3eDBrD=XJEkASXB4TrA@mail.gmail.com>
References: <CAOTb1wfncycR5dimOsQHwDdupi65ioh3eDBrD=XJEkASXB4TrA@mail.gmail.com>
Message-ID: <CAOTb1wcoD071fT+t+OKBcdJ9aupzNn+yTvaJsRCtjZv-t6od8Q@mail.gmail.com>

On Sat, Sep 8, 2012 at 10:29 PM, Chris Jerdonek
<chris.jerdonek at gmail.com> wrote:
> On hg.python.org, it seems like the entries on the "log" page don't
> always link to the corresponding revision:
>
> http://hg.python.org/cpython/shortlog
>
> This seems to happen whenever the revision description begins with
> text that results in a link to something else (e.g. an issue number),
> for example:
>
> "Issue #15822: Fix installation of lib2to3 grammar pickles to ensure."

Sorry, the above is not an example of the behavior (since the leading
word "Issue" does in fact link to the revision).  Here is a better
example:

"#15865: merge with 3.2."

--Chris

> In contrast, the "graph" page seems always to link to the revision:
>
> http://hg.python.org/cpython/graph
>
> Related to this, if the description field contains text that results
> in a link to something else, then the UI doesn't make a distinction
> between the portion of the description that links to the revision and
> the portion that links to the something else.
>
> For example, with description text "Fix for fcc629208842", "Fix for"
> links to the revision for the log entry, but "fcc629208842" links to
> the named revision, and there is no separation or visual indicator
> that the two portions of text link to different things.
>
> It might be better if the revision link was separate from the
> description text.  That would be one way to address both of the issues
> above.
>
> --Chris

From solipsis at pitrou.net  Sun Sep  9 10:53:18 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 9 Sep 2012 10:53:18 +0200
Subject: [Python-Dev] devguide: Add Eric Snow.
References: <3XF3T86SBbzQQl@mail.python.org>
Message-ID: <20120909105318.5884e3bb@pitrou.net>

On Sun,  9 Sep 2012 09:11:08 +0200 (CEST)
georg.brandl <python-checkins at python.org> wrote:
> 
> diff --git a/developers.rst b/developers.rst
> --- a/developers.rst
> +++ b/developers.rst
> @@ -26,9 +26,12 @@
>  
>  - Daniel Holth was given push privileges on Sep 9 2012 by GFB, for PEP editing.
>  
> -- Peter Moody was given push privileges on May 20 2012 by Antoine Pitrou
> -  for authorship and maintenance of the ipaddress module (accepted in PEP
> -  3144 by Nick Coghlan).
> +- Eric Snow was given push privileges on Sep 5 2012 by Antoine Pitrou for
> +  general contributions, on recommendation by Nick Coghlan.

Oops, sorry, I had forgotten to update the developers list.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From georg at python.org  Sun Sep  9 11:25:39 2012
From: georg at python.org (Georg Brandl)
Date: Sun, 09 Sep 2012 11:25:39 +0200
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 2
Message-ID: <504C6093.7060304@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team, I'm delighted to announce the
second release candidate of Python 3.3.0.

This is a preview release, and its use is not recommended in
production settings.

Python 3.3 includes a range of improvements of the 3.x series, as well
as easier porting between 2.x and 3.x.  Major new features and changes
in the 3.3 release series are:

* PEP 380, syntax for delegating to a subgenerator ("yield from")
* PEP 393, flexible string representation (doing away with the
    distinction between "wide" and "narrow" Unicode builds)
* A C implementation of the "decimal" module, with up to 80x speedup
    for decimal-heavy applications
* The import system (__import__) now based on importlib by default
* The new "lzma" module with LZMA/XZ support
* PEP 397, a Python launcher for Windows
* PEP 405, virtual environment support in core
* PEP 420, namespace package support
* PEP 3151, reworking the OS and IO exception hierarchy
* PEP 3155, qualified name for classes and functions
* PEP 409, suppressing exception context
* PEP 414, explicit Unicode literals to help with porting
* PEP 418, extended platform-independent clocks in the "time" module
* PEP 412, a new key-sharing dictionary implementation that
    significantly saves memory for object-oriented code
* PEP 362, the function-signature object
* The new "faulthandler" module that helps diagnosing crashes
* The new "unittest.mock" module
* The new "ipaddress" module
* The "sys.implementation" attribute
* A policy framework for the email package, with a provisional (see
    PEP 411) policy that adds much improved unicode support for email
    header parsing
* A "collections.ChainMap" class for linking mappings to a single unit
* Wrappers for many more POSIX functions in the "os" and "signal"
    modules, as well as other useful functions such as "sendfile()"
* Hash randomization, introduced in earlier bugfix releases, is now
    switched on by default

In total, almost 500 API items are new or improved in Python 3.3.
For a more extensive list of changes in 3.3.0, see

      http://docs.python.org/3.3/whatsnew/3.3.html

To download Python 3.3.0 visit:

      http://www.python.org/download/releases/3.3.0/

Please consider trying Python 3.3.0 with your code and reporting any bugs
you may notice to:

      http://bugs.python.org/


Enjoy!

- -- 
Georg Brandl, Release Manager
georg at python.org
(on behalf of the entire python-dev team and 3.3's contributors)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iEYEARECAAYFAlBMYJMACgkQN9GcIYhpnLCc5ACfcufn57tkNBPFU7qCpZ74GzjW
msMAn3sIwWHLdqixypnnyMBOw1ijILjo
=+e0e
-----END PGP SIGNATURE-----

From solipsis at pitrou.net  Sun Sep  9 20:02:26 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 9 Sep 2012 20:02:26 +0200
Subject: [Python-Dev] cpython (3.2): Issue #15881: Fixed atexit hook in
 multiprocessing.
References: <3XFK934lnczQW8@mail.python.org>
Message-ID: <20120909200226.18f0800c@pitrou.net>

On Sun,  9 Sep 2012 19:28:07 +0200 (CEST)
alexander.belopolsky <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/27d410dd5431
> changeset:   78922:27d410dd5431
> branch:      3.2
> parent:      78895:adc5e6aca251
> user:        Alexander Belopolsky <alexander.belopolsky at gmail.com>
> date:        Sun Sep 09 13:16:15 2012 -0400
> summary:
>   Issue #15881: Fixed atexit hook in multiprocessing.
> 
> files:
>   Doc/ACKS.txt                |   1 +
>   Lib/multiprocessing/util.py |  50 ++++++++++++++++++-------
>   2 files changed, 37 insertions(+), 14 deletions(-)
> 
> 
> diff --git a/Doc/ACKS.txt b/Doc/ACKS.txt
> --- a/Doc/ACKS.txt
> +++ b/Doc/ACKS.txt
> @@ -236,3 +236,4 @@
>     * Moshe Zadka
>     * Milan Zamazal
>     * Cheng Zhang
> +   * Chris McDonough

You really want to credit code contributions in Misc/ACKS, not
Doc/ACKS.txt.  Furthermore, the entries should probably be inserted in
(rough) alphabetical order.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From alexander.belopolsky at gmail.com  Sun Sep  9 20:06:52 2012
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Sun, 9 Sep 2012 14:06:52 -0400
Subject: [Python-Dev] cpython (3.2): Issue #15881: Fixed atexit hook in
	multiprocessing.
In-Reply-To: <20120909200226.18f0800c@pitrou.net>
References: <3XFK934lnczQW8@mail.python.org>
	<20120909200226.18f0800c@pitrou.net>
Message-ID: <CAP7h-xZn4N2sR83qMTymvjLGpYU0z6gi2VYK10qEWu1x=dkF+A@mail.gmail.com>

This change was in the original patch and I did not notice that it
went into wrong file.  Will fix it now.

On Sun, Sep 9, 2012 at 2:02 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> You really want to credit code contributions in Misc/ACKS, not
> Doc/ACKS.txt.  Furthermore, the entries should probably be inserted in
> (rough) alphabetical order.

From brett at python.org  Sun Sep  9 20:56:57 2012
From: brett at python.org (Brett Cannon)
Date: Sun, 9 Sep 2012 14:56:57 -0400
Subject: [Python-Dev] [Python-checkins] devguide: Update committers list
	location.
In-Reply-To: <3XF3gB6HtHzQNv@mail.python.org>
References: <3XF3gB6HtHzQNv@mail.python.org>
Message-ID: <CAP1=2W7CiVibh85+EzxZmPzTiDO+WYg0-GFHAQvfBRre65jEyQ@mail.gmail.com>

Did you mean to use http://docs.python.org/devguide/developers.html instead
of http://hg.python.org/committers.txt as the former lists why while the
latter just lists usernames?

On Sun, Sep 9, 2012 at 3:19 AM, georg.brandl <python-checkins at python.org>wrote:

> http://hg.python.org/devguide/rev/b40bfc99c54f
> changeset:   548:b40bfc99c54f
> user:        Georg Brandl <georg at python.org>
> date:        Sun Sep 09 09:20:23 2012 +0200
> summary:
>   Update committers list location.
>
> files:
>   coredev.rst |  4 ++--
>   faq.rst     |  2 +-
>   2 files changed, 3 insertions(+), 3 deletions(-)
>
>
> diff --git a/coredev.rst b/coredev.rst
> --- a/coredev.rst
> +++ b/coredev.rst
> @@ -25,7 +25,7 @@
>  able to commit them without supervision.
>
>  A complete list of core developer usernames can be found at
> -http://www.python.org/dev/committers. :ref:`developers` lists when and
> why
> +http://hg.python.org/committers.txt. :ref:`developers` lists when and why
>  someone received commit privileges.
>
>
> @@ -68,7 +68,7 @@
>  This should match your username on the issue tracker.
>
>  You can verify your commit access by looking at
> -http://www.python.org/dev/committers which lists all core developers by
> +http://hg.python.org/committers.txt which lists all core developers by
>  username.  If you want to practice, there is a `test repository
>  <http://hg.python.org/test/>`_ where you can freely commit and push any
>  changes you like::
> diff --git a/faq.rst b/faq.rst
> --- a/faq.rst
> +++ b/faq.rst
> @@ -96,7 +96,7 @@
>  On the `issue tracker`_, most core developers will have the Python logo
>  appear next to their name.
>
> -.. _full list of developers: http://www.python.org/dev/committers
> +.. _full list of developers: http://hg.python.org/committers.txt
>
>
>  What standards of behaviour are expected in these communication channels?
>
> --
> Repository URL: http://hg.python.org/devguide
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120909/148f3b40/attachment.html>

From lists at cheimes.de  Mon Sep 10 11:22:05 2012
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 10 Sep 2012 11:22:05 +0200
Subject: [Python-Dev] cpython: Fixed out-of-bounce write to rawmode
 buffer. The fixed size buffer wasn't
In-Reply-To: <3XFXKF18FXzNsb@mail.python.org>
References: <3XFXKF18FXzNsb@mail.python.org>
Message-ID: <504DB13D.2010403@cheimes.de>

Am 10.09.2012 03:50, schrieb christian.heimes:
> http://hg.python.org/cpython/rev/85cb90f79cbf
> changeset:   78946:85cb90f79cbf
> user:        Christian Heimes <christian at cheimes.de>
> date:        Mon Sep 10 03:50:48 2012 +0200
> summary:
>   Fixed out-of-bounce write to rawmode buffer. The fixed size buffer wasn't enlarged for the new 'x' flag. The buffer may contain the 5 flags xrwa+ and the \0 byte
> 
> files:
>   Modules/_io/_iomodule.c |  2 +-
>   1 files changed, 1 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/Modules/_io/_iomodule.c b/Modules/_io/_iomodule.c
> --- a/Modules/_io/_iomodule.c
> +++ b/Modules/_io/_iomodule.c
> @@ -229,7 +229,7 @@
>      int creating = 0, reading = 0, writing = 0, appending = 0, updating = 0;
>      int text = 0, binary = 0, universal = 0;
>  
> -    char rawmode[5], *m;
> +    char rawmode[6], *m;
>      int line_buffering, isatty;
>  
>      PyObject *raw, *modeobj = NULL, *buffer = NULL, *wrapper = NULL;

Georg,

this looks rather severe and should be included in the next release, too.

Christian


From matti.picus at gmail.com  Mon Sep 10 21:11:02 2012
From: matti.picus at gmail.com (Matti Picus)
Date: Mon, 10 Sep 2012 22:11:02 +0300
Subject: [Python-Dev] creating Lib/tests/cmath_testcases.txt for 64 bit
	complex numbers
Message-ID: <504E3B46.6090304@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120910/aa722412/attachment.html>

From matti.picus at gmail.com  Mon Sep 10 22:06:26 2012
From: matti.picus at gmail.com (Matti Picus)
Date: Mon, 10 Sep 2012 23:06:26 +0300
Subject: [Python-Dev] creating Lib/tests/cmath_testcases.txt for 64 bit
 complex numbers (text)
Message-ID: <504E4842.2050902@gmail.com>

(I apologize for posting HTML mail. Retrying.)
Hi. I am implementing complex numbers for pypy's version of numpy.

Numpy has both 128 bit (based on 64 bit floats) and 64 bit (based on 32
bit floats) complex numbers, where afaict cmath uses strictly 128 bit
complex numbers. I made sure the 128 bit numpy complex numbers in pypy
pass the tests in Lib/tests/cmath_testcases.txt.
I would like to generate a similar file to cmath_testcases.txt for 64
bit complex numbers. I found the earliest commit of the file at
http://hg.python.org/cpython/file/b69f443ef480/Lib/test/cmath_testcases.txt

Can the authors of the original file help me reconstruct the scripts or
programs used to generate it, and reformulate them for 32 bit floats?
Since there are more than 2000 cases, and many need rewriting, I prefer
an automated method to error-prone hand coding.

By the way, the level of testing is most impressive.

Thanks for any help or tips,
Matti Picus

From dickinsm at gmail.com  Mon Sep 10 22:25:23 2012
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 10 Sep 2012 21:25:23 +0100
Subject: [Python-Dev] creating Lib/tests/cmath_testcases.txt for 64 bit
 complex numbers (text)
In-Reply-To: <504E4842.2050902@gmail.com>
References: <504E4842.2050902@gmail.com>
Message-ID: <CAAu3qLU7e71OfPBwnLxwfZzfz81wJg-c0Gk__-jd70FPmNHykw@mail.gmail.com>

On Mon, Sep 10, 2012 at 9:06 PM, Matti Picus <matti.picus at gmail.com> wrote:
> Can the authors of the original file help me reconstruct the scripts or
> programs used to generate it, and reformulate them for 32 bit floats?

I used a ctypes wrapper around the MPFR library for most of the
testcases, though some were generated by hand.  I later pushed a
version of the MPFR wrapper to PyPI under the name 'bigfloat', but
unfortunately haven't had time to maintain it;  it may or may not work
right now.  (IIRC, version 0.2 should work okay;  I'd started a
rewrite using Cython instead of ctypes and then got bogged down in
Python 2 / Python 3 distribution fun.  The source at
https://bitbucket.org/dickinsm/bigfloat/src is functional, but doesn't
have distribution sorted out.)

I also used Pari/GP to spot check some of the results.

And that only gets you halfway there, since MPFR only does the real
parts;  it's still necessary to implement the formulas for the real
and imaginary parts in each case, using as much additional precision
as necessary to compute intermediate results.  I used a simple
'peeling the onion' approach, computing lower and upper bounds for the
values in each case and extending precision until both the lower and
upper values gave the same result when rounded to 53 bit precision.

At the time I did this, I don't think the MPC library[1] existed in
usable form;  that may make life easier.

You should also look at Gmpy2 [2];  I believe the most recent release
supports MPC and MPFR.  That's probably the simplest way forward.

Note that MPFR doesn't support IEEE subnormals out of the box, though
it provides some convenience functions to make it relatively easy to
add subnormal support.  That was one of my motivations for writing
bigfloat.

I'm afraid I don't have the exact scripts that I used any more;
they're sitting on the hard drive of a defunct computer that's
awaiting resurrection.

Mark

[1] http://www.multiprecision.org/
[2] http://code.google.com/p/gmpy/

From lists at cheimes.de  Tue Sep 11 12:32:07 2012
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 11 Sep 2012 12:32:07 +0200
Subject: [Python-Dev] issues found by Coverity
Message-ID: <504F1327.4080800@cheimes.de>

Hello,

maybe you have noticed a bunch of commits I made the last couple of
days. They were all related to resource leaks and other issues that were
detected by Coverity. Maybe you have seen the CID in some checkin messages.

Most memory and reference leaks were found in the error branch of
functions and predates even Python 2.7. The leaks are usually in highly
unlikely branches and cause no serious problem. Coverity is really great
in following up all possible code branches.

A few issues are 3.3 regressions, for example #15900, #15895 and
85cb90f79cbf are IMHO serious issues that should be fixed in the next
rc. AFAIK Georg wants to create another rc.

I didn't create NEWS entries for my commits yet and just a few tickets
for modification I wasn't sure about. Is it sufficient to write just one
news entry that lists all fixed functions for 2.7 and 3.2?

Georg:
Shall I create a list of commits with just regressions so you can cherry
pick them? Or do you want to pull all memory leak fixes?

Christian

From victor.stinner at gmail.com  Tue Sep 11 12:41:36 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 11 Sep 2012 12:41:36 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
Message-ID: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>

Hi,

Here are some progress on my astoptimizer project. If you are interested by
the optimizer, run it on your own project or help me to implement more
optimizations.

http://pypi.python.org/pypi/astoptimizer
https://bitbucket.org/haypo/astoptimizer

---

The last version (0.3) works on Python 2.6-3.3 and implements the
following optimizations:

* Call builtin functions if arguments are constants. Examples:

  - len("abc") => 3
  - ord("A") => 65

* Call methods of builtin types if the object and arguments are constants.
  Examples:

  - u"h\\xe9ho".encode("utf-8") => b"h\\xc3\\xa9ho"
  - "python2.7".startswith("python") => True
  - (32).bit_length() => 6
  - float.fromhex("0x1.8p+0") => 1.5

* Call functions of math and string modules for functions without
  border effect. Examples:

  - math.log(32) / math.log(2) => 5.0
  - string.atoi("5") => 5

* Format strings for str%args and print(arg1, arg2, ...) if arguments
  are constants and the format string is valid.
  Examples:

  - "x=%s" % 5 => "x=5"
  - print(1.5) => print("1.5")

* Simplify expressions. Examples:

  - not(x in y) => x not in y
  - 4 and 5 and x and 6 => x and 6

* Loop: replace range() with xrange() on Python 2, and list with
  tuple.  Examples:

  - for x in range(n): ... => for x in xrange(n): ...
  - for x in [1, 2, 3]: ... => for x in (1, 2, 3): ...

* Evaluate unary and binary operators, subscript and comparaison if all
  arguments are constants. Examples:

  - 1 + 2 * 3 => 7
  - not True => False
  - "abc" * 3 => "abcabcabc"
  - abcdef[:3] => abc
  - (2, 7, 3)[1] => 7
  - frozenset("ab") | frozenset("bc") => frozenset("abc")
  - None is None => True
  - "2" in "python2.7" => True
  - "def f(): return 2 if 4 < 5 else 3" => "def f(): return 2"

* Remove dead code. Examples:

  - def f(): return 1; return 2 => def f(): return 1
  - if DEBUG: print("debug") => pass with DEBUG declared as False
  - while 0: ... => pass

---

Unsafe optimizations are disabled by default. Optimizations can be enabled
using a Config class with "features" like "builtin_funcs" (builtin functions
like len()) or "pythonbin" (optimized code will be execute by the same
Python binary executable).

astoptimizer.patch_compile() can be used to hook the optimizer in the
compile() builtin function. On Python 3.3, it is enough to use the optimizer
on imports (thanks to the importlib). On older versions, the compileall
module can be used to compile a whole project using the optimizer.

I didn't start to benchmark anything yet, I focused on fixing bugs (not
generating invalid code). I will start benchmarks when the "variables"
feature (ex: "x=1; print(x)" => "x=1; print(1)") will work. There is an
experimental support of variables, but it is still too agressive and
generate invalid code in some cases (see the TODO file).

I plan to implement other optimizations like unrolling loop or convert
a loop to a list comprehension, see the TODO file.

Don't hesitate to propose more optimizations if you have some ideas ;-)

Victor

From lists at cheimes.de  Tue Sep 11 13:13:00 2012
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 11 Sep 2012 13:13:00 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
Message-ID: <504F1CBC.5050408@cheimes.de>

Am 11.09.2012 12:41, schrieb Victor Stinner:
> Hi,
> 
> Here are some progress on my astoptimizer project. If you are interested by
> the optimizer, run it on your own project or help me to implement more
> optimizations.

Wow, that's an amazing list of optimizations. Keep up the good work!

Christian


From fijall at gmail.com  Tue Sep 11 13:27:43 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 11 Sep 2012 13:27:43 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
Message-ID: <CAK5idxTUF2t8NmuyVy1dWwp3t90eDq5HFV1bZumJ13joxVbQ_g@mail.gmail.com>

On Tue, Sep 11, 2012 at 12:41 PM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> Hi,
>
> Here are some progress on my astoptimizer project. If you are interested by
> the optimizer, run it on your own project or help me to implement more
> optimizations.
>
> http://pypi.python.org/pypi/astoptimizer
> https://bitbucket.org/haypo/astoptimizer
>
> ---
>
> The last version (0.3) works on Python 2.6-3.3 and implements the
> following optimizations:
>
> * Call builtin functions if arguments are constants. Examples:
>
>   - len("abc") => 3
>   - ord("A") => 65

Does it preserve python semantics? What if you change the len builtin?

From victor.stinner at gmail.com  Tue Sep 11 14:06:24 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 11 Sep 2012 14:06:24 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CAK5idxTUF2t8NmuyVy1dWwp3t90eDq5HFV1bZumJ13joxVbQ_g@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<CAK5idxTUF2t8NmuyVy1dWwp3t90eDq5HFV1bZumJ13joxVbQ_g@mail.gmail.com>
Message-ID: <CAMpsgwYfA9azq_bd5DPZy=bu3-L0FAjZK77QXE5Tm0UxM0WCVg@mail.gmail.com>

>> * Call builtin functions if arguments are constants. Examples:
>>
>>   - len("abc") => 3
>>   - ord("A") => 65
>
> Does it preserve python semantics? What if you change the len builtin?

This optimization is disabled by default (in the version 0.3), because
builtin functions may be shadowed. Examples: "len=ord;
print(len('A'))" or exec(compile("print(len('A'))", "test", "exec"),
{'len': ord}).

If you know that one specific builtin function is shadowed (ex:
print), you can modify the configuration to enable optimizations on
builtin functions except the specified function.

Do you projects where builtin functions are shadowed?

The idea is that the author of the application knows its application
(... and all modules used by applications) and is able to explicitly
specify what is that "constant". You can for example declare some
variables as constant using Config.add_constant("module.name", value).
In the same manner, you can declare "pure" functions (without border
effect).

Victor

From trent at snakebite.org  Tue Sep 11 14:20:01 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 11 Sep 2012 08:20:01 -0400
Subject: [Python-Dev] Snakebite v0.1: ready for beta testing.
Message-ID: <20120911122000.GA81325@snakebite.org>

    Quick start:

% cd ~ && svn co http://svn.snakebite.net/.snakebite && cd .snakebite && sh snakebite.subr

    If all goes well, you should see something like this:

A    .snakebite/snakebite.subr
A    .snakebite/ssh_config_ext
A    .snakebite/ssh_known_hosts
 U   .snakebite
Checked out revision 58.
Created link for 'sb'.
Created link for 'sbx'.
Created link for 'sby'.
Created link for 'sbctl'.
Fixed permissions for /Users/Trent/.snakebite/snakebite.subr.
The following commands can now be executed directly:
    sb
    sbx
    sby
    sbctl
Testing connection.........done.
Getting a list of your projects...done.
Looking up your username for project 'cpython'...done.
Getting project details for 'trent.nelson at cpython'...done.
Setting current project to 'cpython'...done.
Trent, you're now configured for cpython.  Enjoy!
+---------------------------------------+
|            Available Hosts            |
|  (Last Update: 2012-09-11 11:08:01Z)  |
+---------------------------------------+
| Alias |         OS         |   Arch   |
+-------|--------------------|----------+
|     a7|AIX 7.1             |  Power4  |
|     d3|DragonFlyBSD 3.0.2  |   x64    |
|    d3x|DragonFlyBSD 3.0.2  |   x86    |
|     h2|HP-UX 11iv2         | PA-RISC  |
|     h3|HP-UX 11iv3         | Itanium2 |
|     i6|IRIX 6.5.30         |   MIPS   |
|    n51|NetBSD 5.1.2        |   x64    |
|   n51x|NetBSD 5.1.2        |   x86    |
|   o51x|OpenBSD 5.1         |   x86    |
|    o51|OpenBSD 5.1         |   x64    |
|    s10|Solaris 10          |  SPARC   |
|     s9|Solaris 9           |  SPARC   |
+---------------------------------------+
Enter alias: 

    Simply enter any of the aliases in the table and it'll ssh you into
    that box as cpython@, i.e.:

Enter alias: a7
AIX arsenic 1 7 000BF95F4C00

    :::.     :::::::..    .::::::. .,::::::  :::.    :::.  :::    .,-:::::  
    ;;`;;    ;;;;``;;;;  ;;;`    ` ;;;;''''  `;;;;,  `;;;  ;;;  ,;;;'````'  
   ,[[ '[[,   [[[,/[[['  '[==/[[[[, [[cccc     [[[[[. '[[  [[[  [[[         
  c$$$cc$$$c  $$$$$$c      '''    $ $$""""     $$$ "Y$c$$  $$$  $$$         
   888   888, 888b "88bo, 88b    dP 888oo,__   888    Y88  888  `88bo,__,o, 
   YMM   ""`  MMMM   "W"   "YMmMY"  """"YUMMM  MMM     YM  MMM    "YUMMMMMP"

                                 AIX 7.1
                       IBM IntelliStation 9114-275
                          2 x 1.4GHz Power4 CPUs
                         2 x 2Gbps LP9802 FC HBAs
                            8GB RAM, 4 x 36GB

[cpython at arsenic]~% 

    General notes:

        - Almost all of the hosts have a corresponding cpython build slave,
          which always lives in ~/buildslave.

        - You're more than welcome to set up local builds on each box.
          Keep everything in ~/hg.  Some hosts already have a ~/hg dir,
          others don't.  The layout should be:

            ~/hg/3.x
            ~/hg/3.2
            ~/hg/2.7

          If they don't exist, feel free to create them.  It's going to
          be easiest to just clone the corresponding build directory
          from ~/buildslave, i.e. if you want a local 3.x area but no
          ~/hg/3.x exists:

            % cd ~/hg
            % hg clone ~/buildslave/3.x-*/build 3.x

          Once a base repo has been created, you can clone a local copy:
            hg clone 3.x 3.x.trent.issue2811

          Try follow that naming convention as it'll make it easier for
          other developers to figure out what each directory is for.

          Also, try and keep tabs on local builds and remove things you
          don't need once you're done.  I haven't finished hooking up
          the SAN yet so everything is on local disks at the moment;
          disk space is a bit light in some places.

        - If you're not used to vi shell key bindings, you're going to
          have a bad time :-)

        - Almost all of the hosts (except for the *BSDs) have been set
          up to use a common ~/.zsh and ~/.vim:
                http:/svn.snakebite.net/home/trunk/
                http:/svn.snakebite.net/skel/trunk/
          They're both based on random dotfile hacking I've done over
          the years and are far from elegant -- so, suggestions welcome.

    If I'm awake and working, I'll be on #python-dev, so that'll be the
    best place to get me if you need immediate assistance.

    So, log in and have a play around!  Oh, X11 forwarding works, too,
    just invoke `sbx` (or `sby`) instead of `sb` and it'll invoke ssh
    with -X or -Y respectively.   All the proprietary UNIX hosts have
    X11 installed, complete with glorious circa-late-nineties Motif
    eye candy.

    For those looking for tangible things to do... take a look at the
    current buildslaves with [SB] in the name -- almost all of them are
    failing in some way/shape/form, so there's plenty of stuff to get
    your teeth stuck into :-)


        Trent.

From brett at python.org  Tue Sep 11 14:23:34 2012
From: brett at python.org (Brett Cannon)
Date: Tue, 11 Sep 2012 08:23:34 -0400
Subject: [Python-Dev] Snakebite v0.1: ready for beta testing.
In-Reply-To: <20120911122000.GA81325@snakebite.org>
References: <20120911122000.GA81325@snakebite.org>
Message-ID: <CAP1=2W7MSjcBrPhmBOC-q=okGw4bP21hmQiHJHUNbuzdM5vE8Q@mail.gmail.com>

Very cool, Trent! I also love the retro use of svn as a tie-in to how long
you have been fighting to bring this project to fruition. =)

On Tue, Sep 11, 2012 at 8:20 AM, Trent Nelson <trent at snakebite.org> wrote:

>     Quick start:
>
> % cd ~ && svn co http://svn.snakebite.net/.snakebite && cd .snakebite &&
> sh snakebite.subr
>
>     If all goes well, you should see something like this:
>
> A    .snakebite/snakebite.subr
> A    .snakebite/ssh_config_ext
> A    .snakebite/ssh_known_hosts
>  U   .snakebite
> Checked out revision 58.
> Created link for 'sb'.
> Created link for 'sbx'.
> Created link for 'sby'.
> Created link for 'sbctl'.
> Fixed permissions for /Users/Trent/.snakebite/snakebite.subr.
> The following commands can now be executed directly:
>     sb
>     sbx
>     sby
>     sbctl
> Testing connection.........done.
> Getting a list of your projects...done.
> Looking up your username for project 'cpython'...done.
> Getting project details for 'trent.nelson at cpython'...done.
> Setting current project to 'cpython'...done.
> Trent, you're now configured for cpython.  Enjoy!
> +---------------------------------------+
> |            Available Hosts            |
> |  (Last Update: 2012-09-11 11:08:01Z)  |
> +---------------------------------------+
> | Alias |         OS         |   Arch   |
> +-------|--------------------|----------+
> |     a7|AIX 7.1             |  Power4  |
> |     d3|DragonFlyBSD 3.0.2  |   x64    |
> |    d3x|DragonFlyBSD 3.0.2  |   x86    |
> |     h2|HP-UX 11iv2         | PA-RISC  |
> |     h3|HP-UX 11iv3         | Itanium2 |
> |     i6|IRIX 6.5.30         |   MIPS   |
> |    n51|NetBSD 5.1.2        |   x64    |
> |   n51x|NetBSD 5.1.2        |   x86    |
> |   o51x|OpenBSD 5.1         |   x86    |
> |    o51|OpenBSD 5.1         |   x64    |
> |    s10|Solaris 10          |  SPARC   |
> |     s9|Solaris 9           |  SPARC   |
> +---------------------------------------+
> Enter alias:
>
>     Simply enter any of the aliases in the table and it'll ssh you into
>     that box as cpython@, i.e.:
>
> Enter alias: a7
> AIX arsenic 1 7 000BF95F4C00
>
>     :::.     :::::::..    .::::::. .,::::::  :::.    :::.  :::    .,-:::::
>     ;;`;;    ;;;;``;;;;  ;;;`    ` ;;;;''''  `;;;;,  `;;;  ;;;  ,;;;'````'
>    ,[[ '[[,   [[[,/[[['  '[==/[[[[, [[cccc     [[[[[. '[[  [[[  [[[
>   c$$$cc$$$c  $$$$$$c      '''    $ $$""""     $$$ "Y$c$$  $$$  $$$
>    888   888, 888b "88bo, 88b    dP 888oo,__   888    Y88  888  `88bo,__,o,
>    YMM   ""`  MMMM   "W"   "YMmMY"  """"YUMMM  MMM     YM  MMM
>  "YUMMMMMP"
>
>                                  AIX 7.1
>                        IBM IntelliStation 9114-275
>                           2 x 1.4GHz Power4 CPUs
>                          2 x 2Gbps LP9802 FC HBAs
>                             8GB RAM, 4 x 36GB
>
> [cpython at arsenic]~%
>
>     General notes:
>
>         - Almost all of the hosts have a corresponding cpython build slave,
>           which always lives in ~/buildslave.
>
>         - You're more than welcome to set up local builds on each box.
>           Keep everything in ~/hg.  Some hosts already have a ~/hg dir,
>           others don't.  The layout should be:
>
>             ~/hg/3.x
>             ~/hg/3.2
>             ~/hg/2.7
>
>           If they don't exist, feel free to create them.  It's going to
>           be easiest to just clone the corresponding build directory
>           from ~/buildslave, i.e. if you want a local 3.x area but no
>           ~/hg/3.x exists:
>
>             % cd ~/hg
>             % hg clone ~/buildslave/3.x-*/build 3.x
>
>           Once a base repo has been created, you can clone a local copy:
>             hg clone 3.x 3.x.trent.issue2811
>
>           Try follow that naming convention as it'll make it easier for
>           other developers to figure out what each directory is for.
>
>           Also, try and keep tabs on local builds and remove things you
>           don't need once you're done.  I haven't finished hooking up
>           the SAN yet so everything is on local disks at the moment;
>           disk space is a bit light in some places.
>
>         - If you're not used to vi shell key bindings, you're going to
>           have a bad time :-)
>
>         - Almost all of the hosts (except for the *BSDs) have been set
>           up to use a common ~/.zsh and ~/.vim:
>                 http:/svn.snakebite.net/home/trunk/
>                 http:/svn.snakebite.net/skel/trunk/
>           They're both based on random dotfile hacking I've done over
>           the years and are far from elegant -- so, suggestions welcome.
>
>     If I'm awake and working, I'll be on #python-dev, so that'll be the
>     best place to get me if you need immediate assistance.
>
>     So, log in and have a play around!  Oh, X11 forwarding works, too,
>     just invoke `sbx` (or `sby`) instead of `sb` and it'll invoke ssh
>     with -X or -Y respectively.   All the proprietary UNIX hosts have
>     X11 installed, complete with glorious circa-late-nineties Motif
>     eye candy.
>
>     For those looking for tangible things to do... take a look at the
>     current buildslaves with [SB] in the name -- almost all of them are
>     failing in some way/shape/form, so there's plenty of stuff to get
>     your teeth stuck into :-)
>
>
>         Trent.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120911/67723827/attachment.html>

From trent at snakebite.org  Tue Sep 11 14:28:21 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 11 Sep 2012 08:28:21 -0400
Subject: [Python-Dev] Snakebite v0.1: ready for beta testing.
In-Reply-To: <20120911122000.GA81325@snakebite.org>
References: <20120911122000.GA81325@snakebite.org>
Message-ID: <20120911122820.GB81325@snakebite.org>

On Tue, Sep 11, 2012 at 05:20:01AM -0700, Trent Nelson wrote:
>     Quick start:
> 
> % cd ~ && svn co http://svn.snakebite.net/.snakebite && cd .snakebite && sh snakebite.subr

    For those that already have ~/.snakebite, one of these will work:
        - sbctl hard-reset, or
        - svn update && sh snakebite.subr

    In general, from now on (I tweaked it quite a bit this morning),
    'sb' automatically updates itself as necessary.

    (Pro-tip though: `sh snakebite.subr` will always do its best to
    re-initialize everything, like fixing permissions when svn update
    strips them, etc.)

        Trent.

From trent at snakebite.org  Tue Sep 11 14:33:00 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 11 Sep 2012 08:33:00 -0400
Subject: [Python-Dev] Snakebite v0.1: ready for beta testing.
In-Reply-To: <CAP1=2W7MSjcBrPhmBOC-q=okGw4bP21hmQiHJHUNbuzdM5vE8Q@mail.gmail.com>
References: <20120911122000.GA81325@snakebite.org>
	<CAP1=2W7MSjcBrPhmBOC-q=okGw4bP21hmQiHJHUNbuzdM5vE8Q@mail.gmail.com>
Message-ID: <20120911123259.GC81325@snakebite.org>

On Tue, Sep 11, 2012 at 05:23:34AM -0700, Brett Cannon wrote:
>    Very cool, Trent! I also love the retro use of svn as a tie-in to how long
>    you have been fighting to bring this project to fruition. =)

    Haha.  I probably shouldn't mention that I started writing all the
    wrapper .snakebite/svn stuff from scratch last Wednesday :P

        Trent.

From ncoghlan at gmail.com  Tue Sep 11 14:57:25 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 11 Sep 2012 22:57:25 +1000
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
Message-ID: <CADiSq7dNXsXjKC12KFrfoVDRF9LjzMPUTHMyKQrgHqQShrTn9Q@mail.gmail.com>

On Tue, Sep 11, 2012 at 8:41 PM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> * Call builtin functions if arguments are constants. Examples:
>
>   - len("abc") => 3
>   - ord("A") => 65

This is fine in an external project, but should never be added to the
standard library. The barrier to semantic changes that break
monkeypatching should be high.

Yes, this is frustrating as it eliminates a great many interesting
static optimisations that are *probably* OK. That's one of the reasons
why PyPy uses tracing - it can perform these optimisations *and* still
include the appropriate dynamic checks.

However, the double barrier of third party module + off by default is
a suitable activation barrier for ensuring people know that what
they're doing is producing bytecode that doesn't behave like standard
Python any more (e.g. tests won't be able to shadow builtins or
optimised module references). Optimisations that break the language
semantics are heading towards the same territory as the byteplay and
withhacks modules (albeit not as evil internally).

> * Call methods of builtin types if the object and arguments are constants.
>   Examples:
>
>   - u"h\\xe9ho".encode("utf-8") => b"h\\xc3\\xa9ho"
>   - "python2.7".startswith("python") => True
>   - (32).bit_length() => 6
>   - float.fromhex("0x1.8p+0") => 1.5

That last one isn't constant, it's a name lookup. Very cool
optimisations for literals, though.

> * Call functions of math and string modules for functions without
>   border effect. Examples:
>
>   - math.log(32) / math.log(2) => 5.0
>   - string.atoi("5") => 5

Same comment applies here as for the builtin optimisation: fine in an
external project, not in the standard library (even if it's off by
default - merely having it there is still an official endorsement of
deliberately breaking the dynamic lookup semantics of our own
language).

> * Format strings for str%args and print(arg1, arg2, ...) if arguments
>   are constants and the format string is valid.
>   Examples:
>
>   - "x=%s" % 5 => "x=5"
>   - print(1.5) => print("1.5")

The print example runs afoul of the general rule above: not in the
standard library, because you're changing the values seen by a mocked
version of print()

> * Simplify expressions. Examples:
>
>   - not(x in y) => x not in y

This (and the "is") equivalent should be OK

>   - 4 and 5 and x and 6 => x and 6

So long as this is just constant folding, that should be fine, too.

>
> * Loop: replace range() with xrange() on Python 2, and list with
>   tuple.  Examples:
>
>   - for x in range(n): ... => for x in xrange(n): ...
>   - for x in [1, 2, 3]: ... => for x in (1, 2, 3): ...

Name lookup optimisations again: not in the standard library.

> * Evaluate unary and binary operators, subscript and comparaison if all
>   arguments are constants. Examples:
>
>   - 1 + 2 * 3 => 7
>   - not True => False
>   - "abc" * 3 => "abcabcabc"
>   - abcdef[:3] => abc
>   - (2, 7, 3)[1] => 7
>   - frozenset("ab") | frozenset("bc") => frozenset("abc")
>   - None is None => True
>   - "2" in "python2.7" => True
>   - "def f(): return 2 if 4 < 5 else 3" => "def f(): return 2"

Yep, literals are good.

> * Remove dead code. Examples:
>
>   - def f(): return 1; return 2 => def f(): return 1
>   - if DEBUG: print("debug") => pass with DEBUG declared as False
>   - while 0: ... => pass

Dangerous.

def f(): return 1; yield
if DEBUG: yield
while 0: yield

>>> def f():
...     if 0:
...         global x
...     return x
...
>>> f()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in f
NameError: global name 'x' is not defined


> Unsafe optimizations are disabled by default. Optimizations can be enabled
> using a Config class with "features" like "builtin_funcs" (builtin functions
> like len()) or "pythonbin" (optimized code will be execute by the same
> Python binary executable).
>
> astoptimizer.patch_compile() can be used to hook the optimizer in the
> compile() builtin function. On Python 3.3, it is enough to use the optimizer
> on imports (thanks to the importlib). On older versions, the compileall
> module can be used to compile a whole project using the optimizer.
>
> I didn't start to benchmark anything yet, I focused on fixing bugs (not
> generating invalid code). I will start benchmarks when the "variables"
> feature (ex: "x=1; print(x)" => "x=1; print(1)") will work. There is an
> experimental support of variables, but it is still too agressive and
> generate invalid code in some cases (see the TODO file).
>
> I plan to implement other optimizations like unrolling loop or convert
> a loop to a list comprehension, see the TODO file.
>
> Don't hesitate to propose more optimizations if you have some ideas ;-)

Mainly just a request to be *very*, *very* clear that the unsafe
optimisations will produce bytecode that *does not behave like Python*
with respect to name lookup semantics, thus mock based testing that
relies on name shadowing will not work correctly, and neither will
direct monkeypatching.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Tue Sep 11 15:16:52 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 11 Sep 2012 23:16:52 +1000
Subject: [Python-Dev] Snakebite v0.1: ready for beta testing.
In-Reply-To: <20120911122820.GB81325@snakebite.org>
References: <20120911122000.GA81325@snakebite.org>
	<20120911122820.GB81325@snakebite.org>
Message-ID: <CADiSq7d_rajfbr--Tc=ayB26xY6DqJ4swk=vDofKv+cLqW-SZg@mail.gmail.com>

On Tue, Sep 11, 2012 at 10:28 PM, Trent Nelson <trent at snakebite.org> wrote:
>     (Pro-tip though: `sh snakebite.subr` will always do its best to
>     re-initialize everything, like fixing permissions when svn update
>     strips them, etc.)

That's actually causing some problems - if SELinux security context
info or ACLs are stored for the directory, then the extra character
(either '.' or '+') in the ls -l output causes the permissions check
to fail.

Easy enough to fix though, the __get_perms command just needs to be changed to:

    ls -l $1 | cut -b 1-10

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From trent at snakebite.org  Tue Sep 11 16:00:03 2012
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 11 Sep 2012 10:00:03 -0400
Subject: [Python-Dev] Snakebite v0.1: ready for beta testing.
In-Reply-To: <CADiSq7d_rajfbr--Tc=ayB26xY6DqJ4swk=vDofKv+cLqW-SZg@mail.gmail.com>
References: <20120911122000.GA81325@snakebite.org>
	<20120911122820.GB81325@snakebite.org>
	<CADiSq7d_rajfbr--Tc=ayB26xY6DqJ4swk=vDofKv+cLqW-SZg@mail.gmail.com>
Message-ID: <20120911140002.GA86865@snakebite.org>

On Tue, Sep 11, 2012 at 06:16:52AM -0700, Nick Coghlan wrote:
> On Tue, Sep 11, 2012 at 10:28 PM, Trent Nelson <trent at snakebite.org> wrote:
> >     (Pro-tip though: `sh snakebite.subr` will always do its best to
> >     re-initialize everything, like fixing permissions when svn update
> >     strips them, etc.)
> 
> That's actually causing some problems - if SELinux security context
> info or ACLs are stored for the directory, then the extra character
> (either '.' or '+') in the ls -l output causes the permissions check
> to fail.
> 
> Easy enough to fix though, the __get_perms command just needs to be changed to:
> 
>     ls -l $1 | cut -b 1-10

    Ta, fixed.

From barry at python.org  Tue Sep 11 17:34:30 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 11 Sep 2012 11:34:30 -0400
Subject: [Python-Dev] Issue 15906;
	regression in argparse in Python 3.3, 3.2, and 2.7
Message-ID: <20120911113430.426e35eb@limelight.wooz.org>

Issue 15906 describes a problem with argparse that is breaking lots of code in
Ubuntu.  This is a recent regression caused by the fix for issue 12776, and it
affects Python 2.7, 3.2, and 3.3.

I posted a diff that should fix the problem, but at the heart of it is a
semantic ambiguity in argparse that needs clarification.  This needs to be
cleared up before a proper patch can be applied.  I have submitted a patch for
what *I* think reasonable semantics should be, but let's see what you think.

Issue 12776 tried to fix a problem illustrated by this example:

-----snip snip-----
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('--file', type=open, default='/etc/passwd')
args = parser.parse_args()

print(args.file.read())

args = parser.parse_args(['--file', '/etc/group'])
print(args.file.read())
-----snip snip-----

What this code is (IMO, sensibly) trying to do is say that args.file will
always be an open file, regardless of whether the path comes from the default
value or is given on the command line.

The problem is that this breaks some other, also sensible code:

-----snip snip-----
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--test", dest="test", type=str,
    default=[], action='append')

args = parser.parse_args(['--test', 'bar'])
args.test.append('baz')

args = parser.parse_args()
args.test.append('baz')
-----snip snip-----

This code is saying, I want to accumulate string arguments into a list,
regardless of whether any arguments are given on the command line.

The fix for issue 12776 broke the last two lines of the second example,
because in the no-command-line-arguments-given case, arg.test is the *string*
"[]" and not the actual empty list object.

It seems to me that the semantics could reasonably be implied to mean that the
type converter should only be applied to the default value when
action='store', as is the default.  Then in the second example, because
action='append', the type conversion would not be applied (it makes no sense
to do so).

I have attached a diff to issue 15906 that implements these semantics.  If you
agree, then I will apply this to all of 3.3, 3.2, and 2.7, which are all
affected by this bug (because the original fix for 12776 was applied to all
three branches).

Georg, I would like to apply this to the 3.3 branch.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120911/dea9e2e5/attachment.pgp>

From perica.zivkovic at gmail.com  Tue Sep 11 18:11:51 2012
From: perica.zivkovic at gmail.com (Perica Zivkovic)
Date: Tue, 11 Sep 2012 09:11:51 -0700 (PDT)
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 2
In-Reply-To: <mailman.421.1347205325.27098.python-announce-list@python.org>
References: <mailman.421.1347205325.27098.python-announce-list@python.org>
Message-ID: <2b80026a-2df6-4bb4-a2f6-c23ab75b9d55@googlegroups.com>

Just a small note, MD5 for RC2 file python-3.3.0rc2.msi is not correct on http://python.org/download/releases/3.3.0/

it would be nice if someone can update it

cheers,

Perica

On Sunday, September 9, 2012 4:25:39 AM UTC-5, Georg Brandl wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> 
> Hash: SHA1
> 
> 
> 
> On behalf of the Python development team, I'm delighted to announce the
> 
> second release candidate of Python 3.3.0.
> 
> 
> 
> This is a preview release, and its use is not recommended in
> 
> production settings.
> 
> 
> 
> Python 3.3 includes a range of improvements of the 3.x series, as well
> 
> as easier porting between 2.x and 3.x.  Major new features and changes
> 
> in the 3.3 release series are:
> 
> 
> 
> * PEP 380, syntax for delegating to a subgenerator ("yield from")
> 
> * PEP 393, flexible string representation (doing away with the
> 
>     distinction between "wide" and "narrow" Unicode builds)
> 
> * A C implementation of the "decimal" module, with up to 80x speedup
> 
>     for decimal-heavy applications
> 
> * The import system (__import__) now based on importlib by default
> 
> * The new "lzma" module with LZMA/XZ support
> 
> * PEP 397, a Python launcher for Windows
> 
> * PEP 405, virtual environment support in core
> 
> * PEP 420, namespace package support
> 
> * PEP 3151, reworking the OS and IO exception hierarchy
> 
> * PEP 3155, qualified name for classes and functions
> 
> * PEP 409, suppressing exception context
> 
> * PEP 414, explicit Unicode literals to help with porting
> 
> * PEP 418, extended platform-independent clocks in the "time" module
> 
> * PEP 412, a new key-sharing dictionary implementation that
> 
>     significantly saves memory for object-oriented code
> 
> * PEP 362, the function-signature object
> 
> * The new "faulthandler" module that helps diagnosing crashes
> 
> * The new "unittest.mock" module
> 
> * The new "ipaddress" module
> 
> * The "sys.implementation" attribute
> 
> * A policy framework for the email package, with a provisional (see
> 
>     PEP 411) policy that adds much improved unicode support for email
> 
>     header parsing
> 
> * A "collections.ChainMap" class for linking mappings to a single unit
> 
> * Wrappers for many more POSIX functions in the "os" and "signal"
> 
>     modules, as well as other useful functions such as "sendfile()"
> 
> * Hash randomization, introduced in earlier bugfix releases, is now
> 
>     switched on by default
> 
> 
> 
> In total, almost 500 API items are new or improved in Python 3.3.
> 
> For a more extensive list of changes in 3.3.0, see
> 
> 
> 
>       http://docs.python.org/3.3/whatsnew/3.3.html
> 
> 
> 
> To download Python 3.3.0 visit:
> 
> 
> 
>       http://www.python.org/download/releases/3.3.0/
> 
> 
> 
> Please consider trying Python 3.3.0 with your code and reporting any bugs
> 
> you may notice to:
> 
> 
> 
>       http://bugs.python.org/
> 
> 
> 
> 
> 
> Enjoy!
> 
> 
> 
> - -- 
> 
> Georg Brandl, Release Manager
> 
> georg at python.org
> 
> (on behalf of the entire python-dev team and 3.3's contributors)
> 
> -----BEGIN PGP SIGNATURE-----
> 
> Version: GnuPG v2.0.19 (GNU/Linux)
> 
> 
> 
> iEYEARECAAYFAlBMYJMACgkQN9GcIYhpnLCc5ACfcufn57tkNBPFU7qCpZ74GzjW
> 
> msMAn3sIwWHLdqixypnnyMBOw1ijILjo
> 
> =+e0e
> 
> -----END PGP SIGNATURE-----

From rdmurray at bitdance.com  Tue Sep 11 18:19:07 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 11 Sep 2012 12:19:07 -0400
Subject: [Python-Dev] Issue 15906;
	regression in argparse in Python 3.3, 3.2, and 2.7
In-Reply-To: <20120911113430.426e35eb@limelight.wooz.org>
References: <20120911113430.426e35eb@limelight.wooz.org>
Message-ID: <20120911161907.D0816250172@webabinitio.net>

On Tue, 11 Sep 2012 11:34:30 -0400, Barry Warsaw <barry at python.org> wrote:
> Issue 15906 describes a problem with argparse that is breaking lots of code in
> Ubuntu.  This is a recent regression caused by the fix for issue 12776, and it
> affects Python 2.7, 3.2, and 3.3.
> 
> I posted a diff that should fix the problem, but at the heart of it is a
> semantic ambiguity in argparse that needs clarification.  This needs to be
> cleared up before a proper patch can be applied.  I have submitted a patch for
> what *I* think reasonable semantics should be, but let's see what you think.
> 
> Issue 12776 tried to fix a problem illustrated by this example:
> 
> -----snip snip-----
> import argparse
> 
> parser = argparse.ArgumentParser()
> parser.add_argument('--file', type=open, default='/etc/passwd')
> args = parser.parse_args()
> 
> print(args.file.read())
> 
> args = parser.parse_args(['--file', '/etc/group'])
> print(args.file.read())
> -----snip snip-----
> 
> What this code is (IMO, sensibly) trying to do is say that args.file will
> always be an open file, regardless of whether the path comes from the default
> value or is given on the command line.

Actually, what 12776 was trying to fix was that the conversion function
was *always* called, and thus the file was *always* opened, even when
a file argument was passed in the arguments to be parsed.

> The problem is that this breaks some other, also sensible code:
> 
> -----snip snip-----
> import argparse
> 
> parser = argparse.ArgumentParser()
> parser.add_argument("--test", dest="test", type=str,
>     default=[], action='append')
> 
> args = parser.parse_args(['--test', 'bar'])
> args.test.append('baz')
> 
> args = parser.parse_args()
> args.test.append('baz')
> -----snip snip-----
> 
> This code is saying, I want to accumulate string arguments into a list,
> regardless of whether any arguments are given on the command line.
>
> The fix for issue 12776 broke the last two lines of the second example,
> because in the no-command-line-arguments-given case, arg.test is the *string*
> "[]" and not the actual empty list object.

But apparently in doing so we broke something else.

There is no question that the current state of the tip of each branch is
a regression.   So at a minimum the old behavior needs to be restored,
which is that the above code should do as you say:  args.test should end
up initialized to the empty list.

As interesting aside: if the above is changed to

    args = parser.parse_args(['--test], 'x'])

the append will work.  The fact that that is true is doubtless a clue
as to why the fix for 12776 broke things unexpectedly.

> It seems to me that the semantics could reasonably be implied to mean that the
> type converter should only be applied to the default value when
> action='store', as is the default.  Then in the second example, because
> action='append', the type conversion would not be applied (it makes no sense
> to do so).

There is another possible semantic, which is that when the store type is
append, the converter should be applied to each of the individual items
in the default list.

Which brings us to another issue: as things stand now, if we have, say,
'default=['abc']', then passing in '--test x' on the command line would
result in args.test being equal to

    ['abc', 'x']

which is consistent with optparse but not necessarily the desired
semantics.

> I have attached a diff to issue 15906 that implements these semantics.  If you
> agree, then I will apply this to all of 3.3, 3.2, and 2.7, which are all
> affected by this bug (because the original fix for 12776 was applied to all
> three branches).
>
> Georg, I would like to apply this to the 3.3 branch.

The 12776 fix isn't going to be in 3.3, so I don't think this is a
pressing issue.  We can take our time to make sure we have the correct
fix.  It is, however, a release blocker for 2.7.4, 3.2.4, and 3.3.1.

--David

From python at mrabarnett.plus.com  Tue Sep 11 19:06:31 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 11 Sep 2012 18:06:31 +0100
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CAMpsgwYfA9azq_bd5DPZy=bu3-L0FAjZK77QXE5Tm0UxM0WCVg@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<CAK5idxTUF2t8NmuyVy1dWwp3t90eDq5HFV1bZumJ13joxVbQ_g@mail.gmail.com>
	<CAMpsgwYfA9azq_bd5DPZy=bu3-L0FAjZK77QXE5Tm0UxM0WCVg@mail.gmail.com>
Message-ID: <504F6F97.1050906@mrabarnett.plus.com>

On 11/09/2012 13:06, Victor Stinner wrote:
>>> * Call builtin functions if arguments are constants. Examples:
>>>
>>>   - len("abc") => 3
>>>   - ord("A") => 65
>>
>> Does it preserve python semantics? What if you change the len builtin?
>
> This optimization is disabled by default (in the version 0.3), because
> builtin functions may be shadowed. Examples: "len=ord;
> print(len('A'))" or exec(compile("print(len('A'))", "test", "exec"),
> {'len': ord}).
>
> If you know that one specific builtin function is shadowed (ex:
> print), you can modify the configuration to enable optimizations on
> builtin functions except the specified function.
>
> Do you projects where builtin functions are shadowed?
>
> The idea is that the author of the application knows its application
> (... and all modules used by applications) and is able to explicitly
> specify what is that "constant". You can for example declare some
> variables as constant using Config.add_constant("module.name", value).
> In the same manner, you can declare "pure" functions (without border
> effect).
>
Do you know what the cost would be of, say, replacing:

     len("abc")

with:

     3 if len is __builtins__.len else len("abc")

if possible where the lookup __builtins__.len is been done early, such
as at compile time?

From tjreedy at udel.edu  Tue Sep 11 19:17:30 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 11 Sep 2012 13:17:30 -0400
Subject: [Python-Dev] Issue 15906;
	regression in argparse in Python 3.3, 3.2, and 2.7
In-Reply-To: <20120911113430.426e35eb@limelight.wooz.org>
References: <20120911113430.426e35eb@limelight.wooz.org>
Message-ID: <k2nrns$un2$1@ger.gmane.org>

On 9/11/2012 11:34 AM, Barry Warsaw wrote:
> Issue 15906 describes a problem with argparse that is breaking lots of code in
> Ubuntu.  This is a recent regression caused by the fix for issue 12776, and it
> affects Python 2.7, 3.2, and 3.3.
>
> I posted a diff that should fix the problem, but at the heart of it is a
> semantic ambiguity in argparse that needs clarification.  This needs to be
> cleared up before a proper patch can be applied.  I have submitted a patch for
> what *I* think reasonable semantics should be, but let's see what you think.
>
> Issue 12776 tried to fix a problem illustrated by this example:
>
> -----snip snip-----
> import argparse
>
> parser = argparse.ArgumentParser()
> parser.add_argument('--file', type=open, default='/etc/passwd')
> args = parser.parse_args()
>
> print(args.file.read())
>
> args = parser.parse_args(['--file', '/etc/group'])
> print(args.file.read())
> -----snip snip-----
>
> What this code is (IMO, sensibly) trying to do is say that args.file will
> always be an open file, regardless of whether the path comes from the default
> value or is given on the command line.
>
> The problem is that this breaks some other, also sensible code:
>
> -----snip snip-----
> import argparse
>
> parser = argparse.ArgumentParser()
> parser.add_argument("--test", dest="test", type=str,
>      default=[], action='append')
>
> args = parser.parse_args(['--test', 'bar'])
> args.test.append('baz')
>
> args = parser.parse_args()
> args.test.append('baz')
> -----snip snip-----
>
> This code is saying, I want to accumulate string arguments into a list,
> regardless of whether any arguments are given on the command line.

This second example strikes me (naively, as English speaker but not 
argparse user) as 'wrong' in that 'default' is being misused to mean 
'start value that is always used to generate the final value' [as in 
sum(iterable, start=0)], rather than 'final value that is only used if 
nothing else is given' (as in nearly all uses of 'default' in Python). 
Perhaps this is what you meant by "semantic ambiguity".

As I see it, storing is done *with* a default or explicit value, 
appending is done *to* a start value *with* whatever. Perhaps reusing 
'default' instead of using a new name such as 'start' was a bit too 
clever for our own good ;-).

> The fix for issue 12776 broke the last two lines of the second example,
> because in the no-command-line-arguments-given case, arg.test is the *string*
> "[]" and not the actual empty list object.

This seems even more wrong (as in slightly crazy) as it switches the 
meaning of 'default' within one parser example rather than between 
parser examples.

> It seems to me that the semantics could reasonably be implied to mean that the
> type converter should only be applied to the default value when
> action='store', as is the default.  Then in the second example, because
> action='append', the type conversion would not be applied (it makes no sense
> to do so).

I arrive at the same conclusion, I believe, by saying that for a given 
parser, the type converter should always or never be applied to 
'default', which should mean converting or not when the parser is 
created. Append to 'default as base or start' should mean not converting.

-- 
Terry Jan Reedy


From g.brandl at gmx.net  Tue Sep 11 19:34:23 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 11 Sep 2012 19:34:23 +0200
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 2
In-Reply-To: <2b80026a-2df6-4bb4-a2f6-c23ab75b9d55@googlegroups.com>
References: <mailman.421.1347205325.27098.python-announce-list@python.org>
	<2b80026a-2df6-4bb4-a2f6-c23ab75b9d55@googlegroups.com>
Message-ID: <k2nslm$6fd$1@ger.gmane.org>

Indeed, thanks for catching that.

The GPG signatures are good, so the downloads are the original ones
built by Martin.

Georg

On 11.09.2012 18:11, Perica Zivkovic wrote:
> Just a small note, MD5 for RC2 file python-3.3.0rc2.msi is not correct on http://python.org/download/releases/3.3.0/
> 
> it would be nice if someone can update it
> 
> cheers,
> 
> Perica
> 
> On Sunday, September 9, 2012 4:25:39 AM UTC-5, Georg Brandl wrote:
> Hash: SHA1
> 
> 
> 
> On behalf of the Python development team, I'm delighted to announce the
> 
> second release candidate of Python 3.3.0.
> 
> 
> 
> This is a preview release, and its use is not recommended in
> 
> production settings.
> 
> 
> 
> Python 3.3 includes a range of improvements of the 3.x series, as well
> 
> as easier porting between 2.x and 3.x.  Major new features and changes
> 
> in the 3.3 release series are:
> 
> 
> 
> * PEP 380, syntax for delegating to a subgenerator ("yield from")
> 
> * PEP 393, flexible string representation (doing away with the
> 
>     distinction between "wide" and "narrow" Unicode builds)
> 
> * A C implementation of the "decimal" module, with up to 80x speedup
> 
>     for decimal-heavy applications
> 
> * The import system (__import__) now based on importlib by default
> 
> * The new "lzma" module with LZMA/XZ support
> 
> * PEP 397, a Python launcher for Windows
> 
> * PEP 405, virtual environment support in core
> 
> * PEP 420, namespace package support
> 
> * PEP 3151, reworking the OS and IO exception hierarchy
> 
> * PEP 3155, qualified name for classes and functions
> 
> * PEP 409, suppressing exception context
> 
> * PEP 414, explicit Unicode literals to help with porting
> 
> * PEP 418, extended platform-independent clocks in the "time" module
> 
> * PEP 412, a new key-sharing dictionary implementation that
> 
>     significantly saves memory for object-oriented code
> 
> * PEP 362, the function-signature object
> 
> * The new "faulthandler" module that helps diagnosing crashes
> 
> * The new "unittest.mock" module
> 
> * The new "ipaddress" module
> 
> * The "sys.implementation" attribute
> 
> * A policy framework for the email package, with a provisional (see
> 
>     PEP 411) policy that adds much improved unicode support for email
> 
>     header parsing
> 
> * A "collections.ChainMap" class for linking mappings to a single unit
> 
> * Wrappers for many more POSIX functions in the "os" and "signal"
> 
>     modules, as well as other useful functions such as "sendfile()"
> 
> * Hash randomization, introduced in earlier bugfix releases, is now
> 
>     switched on by default
> 
> 
> 
> In total, almost 500 API items are new or improved in Python 3.3.
> 
> For a more extensive list of changes in 3.3.0, see
> 
> 
> 
>       http://docs.python.org/3.3/whatsnew/3.3.html
> 
> 
> 
> To download Python 3.3.0 visit:
> 
> 
> 
>       http://www.python.org/download/releases/3.3.0/
> 
> 
> 
> Please consider trying Python 3.3.0 with your code and reporting any bugs
> 
> you may notice to:
> 
> 
> 
>       http://bugs.python.org/
> 
> 
> 
> 
> 
> Enjoy!
> 
> 
> 
> 



From guido at python.org  Tue Sep 11 19:35:00 2012
From: guido at python.org (Guido van Rossum)
Date: Tue, 11 Sep 2012 10:35:00 -0700
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <504F6F97.1050906@mrabarnett.plus.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<CAK5idxTUF2t8NmuyVy1dWwp3t90eDq5HFV1bZumJ13joxVbQ_g@mail.gmail.com>
	<CAMpsgwYfA9azq_bd5DPZy=bu3-L0FAjZK77QXE5Tm0UxM0WCVg@mail.gmail.com>
	<504F6F97.1050906@mrabarnett.plus.com>
Message-ID: <CAP7+vJ+30ajtrAmr9Sr6j19qOtcyerMATMjup-J1T5Jx-9xo9Q@mail.gmail.com>

On Tue, Sep 11, 2012 at 10:06 AM, MRAB <python at mrabarnett.plus.com> wrote:
> On 11/09/2012 13:06, Victor Stinner wrote:
>>>>
>>>> * Call builtin functions if arguments are constants. Examples:
>>>>
>>>>   - len("abc") => 3
>>>>   - ord("A") => 65
>>>
>>>
>>> Does it preserve python semantics? What if you change the len builtin?
>>
>>
>> This optimization is disabled by default (in the version 0.3), because
>> builtin functions may be shadowed. Examples: "len=ord;
>> print(len('A'))" or exec(compile("print(len('A'))", "test", "exec"),
>> {'len': ord}).
>>
>> If you know that one specific builtin function is shadowed (ex:
>> print), you can modify the configuration to enable optimizations on
>> builtin functions except the specified function.
>>
>> Do you projects where builtin functions are shadowed?
>>
>> The idea is that the author of the application knows its application
>> (... and all modules used by applications) and is able to explicitly
>> specify what is that "constant". You can for example declare some
>> variables as constant using Config.add_constant("module.name", value).
>> In the same manner, you can declare "pure" functions (without border
>> effect).
>>
> Do you know what the cost would be of, say, replacing:
>
>     len("abc")
>
> with:
>
>     3 if len is __builtins__.len else len("abc")
>
> if possible where the lookup __builtins__.len is been done early, such
> as at compile time?

Doing the lookup at compile time would defeat the whole purpose (the
dynamic shadowing of len could happen at any time). Also there are two
forms of shadowing: patching the globals of the module where the call
occurs, or patching the builtins.

Doing the lookup at run time would also defeat the purpose, because
most of the cost of calling len() *is* the lookup. Actually two
lookups -- the LOAD_GLOBAL opcode is used, which first looks in the
globals (this will normally always fail) and then in the builtins.

FWIW, I expect that there are few places where len(<constant string>)
is actually used. However I expect it to be quite common for ord(),
where the same reasoning applies.

-- 
--Guido van Rossum (python.org/~guido)

From chris.jerdonek at gmail.com  Tue Sep 11 19:39:34 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Tue, 11 Sep 2012 10:39:34 -0700
Subject: [Python-Dev] Issue 15906;
 regression in argparse in Python 3.3, 3.2, and 2.7
In-Reply-To: <k2nrns$un2$1@ger.gmane.org>
References: <20120911113430.426e35eb@limelight.wooz.org>
	<k2nrns$un2$1@ger.gmane.org>
Message-ID: <CAOTb1wdm1aPAeZoVouiXz07zWkKguV_8J+275VJKMaWX+1=6ow@mail.gmail.com>

On Tue, Sep 11, 2012 at 10:17 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 9/11/2012 11:34 AM, Barry Warsaw wrote:
>> It seems to me that the semantics could reasonably be implied to mean that
>> the
>> type converter should only be applied to the default value when
>> action='store', as is the default.  Then in the second example, because
>> action='append', the type conversion would not be applied (it makes no
>> sense
>> to do so).
>
> I arrive at the same conclusion, I believe, by saying that for a given
> parser, the type converter should always or never be applied to 'default',
> which should mean converting or not when the parser is created. Append to
> 'default as base or start' should mean not converting.

I noted this in the comment to the issue, but the argparse
documentation says this about the type argument: "type= can take any
callable that takes a single string argument and returns the converted
value."

(from http://docs.python.org/dev/library/argparse.html#type )

So type is meant for converting (command-line) strings to other types.
 What is causing one of the problems here is calling type on a
non-string (an empty list in one of the cases above), which doesn't
look like it's meant to be supported..

So another way out could simply be not to call type on non-strings.
Indeed, this was done before.  One of the changes that was made in the
patch for issue 12776 was to remove the str type check prior to
calling type.

--Chris

From storchaka at gmail.com  Tue Sep 11 20:48:28 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 11 Sep 2012 21:48:28 +0300
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
Message-ID: <k2o0st$ef6$1@ger.gmane.org>

On 11.09.12 13:41, Victor Stinner wrote:
> Here are some progress on my astoptimizer project. If you are interested by
> the optimizer, run it on your own project or help me to implement more
> optimizations.
>
> http://pypi.python.org/pypi/astoptimizer
> https://bitbucket.org/haypo/astoptimizer

It's a very interesting work.

> * Call builtin functions if arguments are constants. Examples:
>
>    - len("abc") => 3
>    - ord("A") => 65

Does this mean transformation print("A") => None and output at compile time?

> * Loop: replace range() with xrange() on Python 2,

range(0, 10**10, 10**9)

> * Evaluate unary and binary operators, subscript and comparaison if all
>    arguments are constants. Examples:
>
>    - 1 + 2 * 3 => 7

Does this mean 1/0 raises exception at compile time?

> * Remove dead code. Examples:
>
>    - def f(): return 1; return 2 => def f(): return 1
>    - if DEBUG: print("debug") => pass with DEBUG declared as False
>    - while 0: ... => pass

As Nick said, it could change the semantics, if there were local 
variable assignment or yield in removed code.

> Unsafe optimizations are disabled by default. Optimizations can be enabled
> using a Config class with "features" like "builtin_funcs" (builtin functions
> like len()) or "pythonbin" (optimized code will be execute by the same
> Python binary executable).

It would be good if the optimizer checked if the built-in names 
overridden in this module or function and disabled this dangerous 
optimization in such case.

> I plan to implement other optimizations like unrolling loop or convert
> a loop to a list comprehension, see the TODO file.
>
> Don't hesitate to propose more optimizations if you have some ideas ;-)

set([1, 2, 3]) => {1, 2, 3}
set([x for ...]) => {x for ...}
dict([(k, v) for ...]) => {k: v for ...}
dict((k, v) for ...) => {k: v for ...}
''.join([s for ...]) => ''.join(s for ...)
a.extend([s for ...]) => a.extend(s for ...)
(f(x) for x in a) => map(f, a)
(x.y for x in a) => map(operator.attrgetter('y'), a)
(x[0] for x in a) => map(operator.itemgetter(0), a)
(2 * x for x in a) => map((2).__mul__, a)
(x in b for x in a) => map(b.__contains__, a)
map(lambda x: x.strip(), a) => (x.strip() for x in a)
x in ['i', 'em', 'cite'] => x in {'i', 'em', 'cite'}
x == 'i' or x == 'em' or x == 'cite'] => x in {'i', 'em', 'cite'}
a = []; for ...: a.append(x) => a = [x for ...]
a = (); for ...: a.add(x) => a = {x for ...}
a = {}; for ...: a[k] = v => a = {k: v for ...}
for ...: f.write(...) => __fwrite = f.write; for ...: __fwrite(...)
x = x + 1 => x += 1
x = x + ' ' => x += ' '
x = x + [y] => x.append(y)
x = x + [y, z] => x.extend([y, z])
'x=%s' % repr(x) => 'x=%a' % (x,)
'x=%s' % x + s => 'x=%s%s' % (x, s)
x = x + ', [%s]' % y => x = '%s, [%s]' % (x, y)
range(0, x) => range(x)
for i in range(len(a)): x = a[i] ... => for i, x in enumerate(a): ...
i = 0; for x in a: i += 1 ... => for i, x in enumerate(a, 1): ...
i = 1; for x in a: ... i += 1 => for i, x in enumerate(a, 1): ...
s = 0; for ...: if ...: s += 1 => s = sum(1 for ... if ...)
while True: s = f.readline(); if not s: break; ... => for s in f: ...
def f(x): ... len() ... => def f(x, __len = len): ... __len() ...

Not all such transformations are always safe (and I know code in stdlib 
where they are not).



From stefan_ml at behnel.de  Tue Sep 11 20:58:19 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 11 Sep 2012 20:58:19 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CADiSq7dNXsXjKC12KFrfoVDRF9LjzMPUTHMyKQrgHqQShrTn9Q@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<CADiSq7dNXsXjKC12KFrfoVDRF9LjzMPUTHMyKQrgHqQShrTn9Q@mail.gmail.com>
Message-ID: <k2o1k9$l1a$1@ger.gmane.org>

Nick Coghlan, 11.09.2012 14:57:
> On Tue, Sep 11, 2012 at 8:41 PM, Victor Stinner wrote:
>> * Loop: replace range() with xrange() on Python 2, and list with
>>   tuple.  Examples:
>>
>>   - for x in range(n): ... => for x in xrange(n): ...
>>   - for x in [1, 2, 3]: ... => for x in (1, 2, 3): ...
> 
> Name lookup optimisations again: not in the standard library.

I assume you meant the "range" part, not the second example (which will end
up caching a constant tuple).


>> * Evaluate unary and binary operators, subscript and comparaison if all
>>   arguments are constants. Examples:
>>
>>   - 1 + 2 * 3 => 7
>>   - not True => False
>>   - "abc" * 3 => "abcabcabc"
>>   - abcdef[:3] => abc
>>   - (2, 7, 3)[1] => 7
>>   - frozenset("ab") | frozenset("bc") => frozenset("abc")

That's a name lookup, too.


>>   - None is None => True
>>   - "2" in "python2.7" => True
>>   - "def f(): return 2 if 4 < 5 else 3" => "def f(): return 2"
> 
> Yep, literals are good.

Except that evaluating something like '"abc" * constant' can eat up all
memory, imagine this code:

    KILL_MEMORY = sys.argv[1] == 'NEVER PASS THIS VALUE'

    def test():
        if KILL_MEMORY:
            return 'abc' * 10000000000000000000000000000
        else:
            return 'abc'

Stefan



From stefan_ml at behnel.de  Tue Sep 11 21:03:58 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 11 Sep 2012 21:03:58 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <k2o0st$ef6$1@ger.gmane.org>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<k2o0st$ef6$1@ger.gmane.org>
Message-ID: <k2o1ur$n0u$1@ger.gmane.org>

Serhiy Storchaka, 11.09.2012 20:48:
> set([1, 2, 3]) => {1, 2, 3}
> set([x for ...]) => {x for ...}
> dict([(k, v) for ...]) => {k: v for ...}
> dict((k, v) for ...) => {k: v for ...}
> ''.join([s for ...]) => ''.join(s for ...)
> a.extend([s for ...]) => a.extend(s for ...)
> (f(x) for x in a) => map(f, a)
> (x.y for x in a) => map(operator.attrgetter('y'), a)
> (x[0] for x in a) => map(operator.itemgetter(0), a)
> (2 * x for x in a) => map((2).__mul__, a)
> (x in b for x in a) => map(b.__contains__, a)
> map(lambda x: x.strip(), a) => (x.strip() for x in a)
> x in ['i', 'em', 'cite'] => x in {'i', 'em', 'cite'}
> x == 'i' or x == 'em' or x == 'cite'] => x in {'i', 'em', 'cite'}
> a = []; for ...: a.append(x) => a = [x for ...]
> a = (); for ...: a.add(x) => a = {x for ...}
> a = {}; for ...: a[k] = v => a = {k: v for ...}
> for ...: f.write(...) => __fwrite = f.write; for ...: __fwrite(...)
> x = x + 1 => x += 1
> x = x + ' ' => x += ' '
> x = x + [y] => x.append(y)
> x = x + [y, z] => x.extend([y, z])
> 'x=%s' % repr(x) => 'x=%a' % (x,)
> 'x=%s' % x + s => 'x=%s%s' % (x, s)
> x = x + ', [%s]' % y => x = '%s, [%s]' % (x, y)
> range(0, x) => range(x)
> for i in range(len(a)): x = a[i] ... => for i, x in enumerate(a): ...
> i = 0; for x in a: i += 1 ... => for i, x in enumerate(a, 1): ...
> i = 1; for x in a: ... i += 1 => for i, x in enumerate(a, 1): ...
> s = 0; for ...: if ...: s += 1 => s = sum(1 for ... if ...)
> while True: s = f.readline(); if not s: break; ... => for s in f: ...
> def f(x): ... len() ... => def f(x, __len = len): ... __len() ...
> 
> Not all such transformations are always safe (and I know code in stdlib
> where they are not).

Actually, many of them are not, and some of them are even plain wrong or
may at least turn out not to be optimisations.

Stefan



From barry at python.org  Tue Sep 11 21:08:26 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 11 Sep 2012 15:08:26 -0400
Subject: [Python-Dev] Issue 15906;
 regression in argparse in Python 3.3, 3.2, and 2.7
In-Reply-To: <20120911161907.D0816250172@webabinitio.net>
References: <20120911113430.426e35eb@limelight.wooz.org>
	<20120911161907.D0816250172@webabinitio.net>
Message-ID: <20120911150826.556b274e@resist.wooz.org>

On Sep 11, 2012, at 12:19 PM, R. David Murray wrote:

>The 12776 fix isn't going to be in 3.3, so I don't think this is a
>pressing issue.  We can take our time to make sure we have the correct
>fix.  It is, however, a release blocker for 2.7.4, 3.2.4, and 3.3.1.

Are you sure about that?

% ./python
Python 3.3.0rc2+ (default:6fea947edead, Sep 11 2012, 15:03:16) 
[GCC 4.7.1 20120908 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import argparse
>>> p = argparse.ArgumentParser()
>>> p.add_argument('--test', dest='test', type=str, default=[], action='append')
_AppendAction(option_strings=['--test'], dest='test', nargs=None, const=None, default=[], type=<class 'str'>, choices=None, help=None, metavar=None)
>>> args = p.parse_args()
>>> args.test
'[]'


Cheers,
-Barry

From rdmurray at bitdance.com  Tue Sep 11 21:24:23 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 11 Sep 2012 15:24:23 -0400
Subject: [Python-Dev] Issue 15906;
	regression in argparse in Python 3.3, 3.2, and 2.7
In-Reply-To: <20120911150826.556b274e@resist.wooz.org>
References: <20120911113430.426e35eb@limelight.wooz.org>
	<20120911161907.D0816250172@webabinitio.net>
	<20120911150826.556b274e@resist.wooz.org>
Message-ID: <20120911192423.9D9CC250172@webabinitio.net>

On Tue, 11 Sep 2012 15:08:26 -0400, Barry Warsaw <barry at python.org> wrote:
> On Sep 11, 2012, at 12:19 PM, R. David Murray wrote:
> 
> >The 12776 fix isn't going to be in 3.3, so I don't think this is a
> >pressing issue.  We can take our time to make sure we have the correct
> >fix.  It is, however, a release blocker for 2.7.4, 3.2.4, and 3.3.1.
> 
> Are you sure about that?
> 
> % ./python
> Python 3.3.0rc2+ (default:6fea947edead, Sep 11 2012, 15:03:16) 

The actual 3.3.0 release will not be made from default, but from a
dedicated clone/branch that does not have the 12776-fix changeset 
in it.

--David

From barry at python.org  Tue Sep 11 21:25:12 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 11 Sep 2012 15:25:12 -0400
Subject: [Python-Dev] Issue 15906;
 regression in argparse in Python 3.3, 3.2, and 2.7
In-Reply-To: <20120911150826.556b274e@resist.wooz.org>
References: <20120911113430.426e35eb@limelight.wooz.org>
	<20120911161907.D0816250172@webabinitio.net>
	<20120911150826.556b274e@resist.wooz.org>
Message-ID: <20120911152512.33431012@resist.wooz.org>

On Sep 11, 2012, at 03:08 PM, Barry Warsaw wrote:

>Are you sure about that?
>
>% ./python
>Python 3.3.0rc2+ (default:6fea947edead, Sep 11 2012, 15:03:16) 

Never mind.  Georg didn't pull that patch into his release clone.

(Aside: Georg, maybe you could fiddle with the default branch's version
numbers so that it reflects being 3.3.1a1?)

(Aside 2: releasing/3.3.0 is broken:

Traceback (most recent call last):
  File "/home/barry/projects/python/3.3.0/Parser/asdl.py", line 309, in visit
    meth(object, *args)
  File "./Parser/asdl_c.py", line 1043, in visitSum
    self.simpleSum(sum, name)
  File "./Parser/asdl_c.py", line 1067, in simpleSum
    self.emit("default:" % name, 2)
TypeError: not all arguments converted during string formatting
make: *** [Python/Python-ast.c] Error 1
)

-Barry

From barry at python.org  Tue Sep 11 21:31:39 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 11 Sep 2012 15:31:39 -0400
Subject: [Python-Dev] Issue 15906;
 regression in argparse in Python 3.3, 3.2, and 2.7
In-Reply-To: <k2nrns$un2$1@ger.gmane.org>
References: <20120911113430.426e35eb@limelight.wooz.org>
	<k2nrns$un2$1@ger.gmane.org>
Message-ID: <20120911153139.18a228f7@resist.wooz.org>

On Sep 11, 2012, at 01:17 PM, Terry Reedy wrote:

>This second example strikes me (naively, as English speaker but not argparse
>user) as 'wrong' in that 'default' is being misused to mean 'start value that
>is always used to generate the final value' [as in sum(iterable, start=0)],
>rather than 'final value that is only used if nothing else is given' (as in
>nearly all uses of 'default' in Python). Perhaps this is what you meant by
>"semantic ambiguity".

Well, in a sense yes, that is an ambiguity so I won't quibble about whether
it's the same one or not. :)

>As I see it, storing is done *with* a default or explicit value, appending is
>done *to* a start value *with* whatever. Perhaps reusing 'default' instead of
>using a new name such as 'start' was a bit too clever for our own good ;-).

I suspect that it's too late to change this, by say adding a 'start' parameter
or some such, at least until Python 3.4.

>> The fix for issue 12776 broke the last two lines of the second example,
>> because in the no-command-line-arguments-given case, arg.test is the
>> *string* "[]" and not the actual empty list object.
>
>This seems even more wrong (as in slightly crazy) as it switches the meaning
>of 'default' within one parser example rather than between parser examples.

And yet, that's how it works in 2.7, 3.2, and 3.3.1.

>> It seems to me that the semantics could reasonably be implied to mean that
>> the type converter should only be applied to the default value when
>> action='store', as is the default.  Then in the second example, because
>> action='append', the type conversion would not be applied (it makes no
>> sense to do so).
>
>I arrive at the same conclusion, I believe, by saying that for a given
>parser, the type converter should always or never be applied to 'default',
>which should mean converting or not when the parser is created. Append to
>'default as base or start' should mean not converting.

Does that mean that for fixing the regression, you favor applying the type
conversion only for action='store' or by only applying it when the default is
a string?

It seems better to only apply the type conversion for action='store' but more
backward compatible for the original behavior to only apply it when default is
a string.

Cheers,
-Barry

From barry at python.org  Tue Sep 11 21:33:06 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 11 Sep 2012 15:33:06 -0400
Subject: [Python-Dev] Issue 15906;
 regression in argparse in Python 3.3, 3.2, and 2.7
In-Reply-To: <CAOTb1wdm1aPAeZoVouiXz07zWkKguV_8J+275VJKMaWX+1=6ow@mail.gmail.com>
References: <20120911113430.426e35eb@limelight.wooz.org>
	<k2nrns$un2$1@ger.gmane.org>
	<CAOTb1wdm1aPAeZoVouiXz07zWkKguV_8J+275VJKMaWX+1=6ow@mail.gmail.com>
Message-ID: <20120911153306.1d16ea52@resist.wooz.org>

On Sep 11, 2012, at 10:39 AM, Chris Jerdonek wrote:

>So another way out could simply be not to call type on non-strings.
>Indeed, this was done before.  One of the changes that was made in the
>patch for issue 12776 was to remove the str type check prior to
>calling type.

Yep, so probably that's the right fix for the regression.  I'll try to whip up
a patch.

Cheers,
-Barry


From barry at python.org  Tue Sep 11 21:35:54 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 11 Sep 2012 15:35:54 -0400
Subject: [Python-Dev] Issue 15906;
 regression in argparse in Python 3.3, 3.2, and 2.7
In-Reply-To: <20120911161907.D0816250172@webabinitio.net>
References: <20120911113430.426e35eb@limelight.wooz.org>
	<20120911161907.D0816250172@webabinitio.net>
Message-ID: <20120911153554.62ef5f8d@resist.wooz.org>

On Sep 11, 2012, at 12:19 PM, R. David Murray wrote:

>There is another possible semantic, which is that when the store type is
>append, the converter should be applied to each of the individual items in
>the default list.

Yep.  Maybe for 3.4 <wink>.

>Which brings us to another issue: as things stand now, if we have, say,
>'default=['abc']', then passing in '--test x' on the command line would
>result in args.test being equal to
>
>    ['abc', 'x']
>
>which is consistent with optparse but not necessarily the desired
>semantics.

Right.  This gets to Chris's observation that we're blurring the meaning of
'default'.  For action='append' should that be the default value when no
arguments are given, or the starting value for the append operations?

We might need to add a 'start' argument when action='append'.

Cheers,
-Barry

From g.brandl at gmx.net  Tue Sep 11 22:13:00 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 11 Sep 2012 22:13:00 +0200
Subject: [Python-Dev] Issue 15906;
	regression in argparse in Python 3.3, 3.2, and 2.7
In-Reply-To: <20120911152512.33431012@resist.wooz.org>
References: <20120911113430.426e35eb@limelight.wooz.org>
	<20120911161907.D0816250172@webabinitio.net>
	<20120911150826.556b274e@resist.wooz.org>
	<20120911152512.33431012@resist.wooz.org>
Message-ID: <k2o5v3$rnp$1@ger.gmane.org>

On 11.09.2012 21:25, Barry Warsaw wrote:
> On Sep 11, 2012, at 03:08 PM, Barry Warsaw wrote:
> 
>>Are you sure about that?
>>
>>% ./python
>>Python 3.3.0rc2+ (default:6fea947edead, Sep 11 2012, 15:03:16) 
> 
> Never mind.  Georg didn't pull that patch into his release clone.
> 
> (Aside: Georg, maybe you could fiddle with the default branch's version
> numbers so that it reflects being 3.3.1a1?)

Well, it's not yet.

> (Aside 2: releasing/3.3.0 is broken:
> 
> Traceback (most recent call last):
>   File "/home/barry/projects/python/3.3.0/Parser/asdl.py", line 309, in visit
>     meth(object, *args)
>   File "./Parser/asdl_c.py", line 1043, in visitSum
>     self.simpleSum(sum, name)
>   File "./Parser/asdl_c.py", line 1067, in simpleSum
>     self.emit("default:" % name, 2)
> TypeError: not all arguments converted during string formatting
> make: *** [Python/Python-ast.c] Error 1
> )

This bug is not specific to that repository.  Not sure how you hit the bug; a
fresh checkout doesn't do it for me here.

Georg


From barry at python.org  Tue Sep 11 23:15:11 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 11 Sep 2012 17:15:11 -0400
Subject: [Python-Dev] Issue 15906;
 regression in argparse in Python 3.3, 3.2, and 2.7
In-Reply-To: <k2o5v3$rnp$1@ger.gmane.org>
References: <20120911113430.426e35eb@limelight.wooz.org>
	<20120911161907.D0816250172@webabinitio.net>
	<20120911150826.556b274e@resist.wooz.org>
	<20120911152512.33431012@resist.wooz.org>
	<k2o5v3$rnp$1@ger.gmane.org>
Message-ID: <20120911171511.3e3789c5@resist.wooz.org>

On Sep 11, 2012, at 10:13 PM, Georg Brandl wrote:

>> Traceback (most recent call last):
>>   File "/home/barry/projects/python/3.3.0/Parser/asdl.py", line 309, in visit
>>     meth(object, *args)
>>   File "./Parser/asdl_c.py", line 1043, in visitSum
>>     self.simpleSum(sum, name)
>>   File "./Parser/asdl_c.py", line 1067, in simpleSum
>>     self.emit("default:" % name, 2)
>> TypeError: not all arguments converted during string formatting
>> make: *** [Python/Python-ast.c] Error 1
>> )
>
>This bug is not specific to that repository.  Not sure how you hit the bug; a
>fresh checkout doesn't do it for me here.

http://bugs.python.org/issue15923

-Barry

From tjreedy at udel.edu  Tue Sep 11 23:30:29 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 11 Sep 2012 17:30:29 -0400
Subject: [Python-Dev] Issue 15906;
	regression in argparse in Python 3.3, 3.2, and 2.7
In-Reply-To: <20120911153139.18a228f7@resist.wooz.org>
References: <20120911113430.426e35eb@limelight.wooz.org>
	<k2nrns$un2$1@ger.gmane.org>
	<20120911153139.18a228f7@resist.wooz.org>
Message-ID: <k2oaia$2uo$1@ger.gmane.org>

On 9/11/2012 3:31 PM, Barry Warsaw wrote:
> On Sep 11, 2012, at 01:17 PM, Terry Reedy wrote:

>> As I see it, storing is done *with* a default or explicit value, appending is
>> done *to* a start value *with* whatever. Perhaps reusing 'default' instead of
>> using a new name such as 'start' was a bit too clever for our own good ;-).

Such cleverness is something that I might have done, having been 
'raised' on IBM mainframes with statements with 8-10 position-only 
parameters (JCL DD? don't quite remember), and having had to carefully 
count commas writing things like
XX arg1, arg2,,,arg5,,,arg8,,
The cost of positions rewards multiplexing the meaning of positions.
The availability of keyword-only parameters changes the tradeoffs.


>>> The fix for issue 12776 broke the last two lines of the second example,
>>> because in the no-command-line-arguments-given case, arg.test is the
>>> *string* "[]" and not the actual empty list object.
>>
>> This seems even more wrong (as in slightly crazy) as it switches the meaning
>> of 'default' within one parser example rather than between parser examples.
>
> And yet, that's how it works in 2.7, 3.2, and 3.3.1.

In all 2.7 or 3.2?
>
>>> It seems to me that the semantics could reasonably be implied to mean that
>>> the type converter should only be applied to the default value when
>>> action='store', as is the default.  Then in the second example, because
>>> action='append', the type conversion would not be applied (it makes no
>>> sense to do so).
>>
>> I arrive at the same conclusion, I believe, by saying that for a given
>> parser, the type converter should always or never be applied to 'default',
>> which should mean converting or not when the parser is created. Append to
>> 'default as base or start' should mean not converting.
>
> Does that mean that for fixing the regression, you favor applying the type
> conversion only for action='store' or by only applying it when the default is
> a string?

How about a third rule which amounts to the second *plus* (maybe) the 
first: apply the conversion when it fulfills the purpose of having type 
conversions, which is to make sure that the caller get objects of the 
desired class. Also, only add necessary restrictions to argparse users.

In particular: parser.parse_args gets a sequence of strings. But callers 
of .parse_args often want non-string objects. argparse could return args 
as named strings and make the user convert to the types wanted, but 
there are good reasons to do the conversion at the point of parsing.

So the conversion function *must* accept strings, but it seems an 
unnecessary restriction to require it to accept anything else, including 
its output type or a non-string 'default'.

When the 'default' is a substitute for user input, as with Store, *and* 
it is a string, it should be converted just like user input. But the 
parser designer should not have to convert a non-string default object 
to a string just so it can be converted back to the real default 
non-string object (another unnecessary restriction). (If type=int, there 
is no sense in requiring default='100' instead of default=100.) Indeed, 
the default object might not even be a possible output of the conversion 
function (for instance, type=int, default=None).

If the 'default' is not a backup/replacement for a string input to 
parse_args, as with append, then it seems it should not be converted either.

> It seems better to only apply the type conversion for action='store' but more
> backward compatible for the original behavior to only apply it when default is
> a string.

Are there any other actions where the default might be a string that 
should be converted the same way as user input? If not, both rules apply.

As I said, I am not familiar with the details of argparse (or the 
previous bugfix), so I let you sort out how these thoughts apply to 
previous, current, and desired future behavior and doc claims.

-- 
Terry Jan Reedy


From victor.stinner at gmail.com  Tue Sep 11 23:46:19 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 11 Sep 2012 23:46:19 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CADiSq7dNXsXjKC12KFrfoVDRF9LjzMPUTHMyKQrgHqQShrTn9Q@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<CADiSq7dNXsXjKC12KFrfoVDRF9LjzMPUTHMyKQrgHqQShrTn9Q@mail.gmail.com>
Message-ID: <CAMpsgwbJUEk4=rL6=JABE0Pc0368FJKuOzdO_0C6EiVicTpAHw@mail.gmail.com>

2012/9/11 Nick Coghlan <ncoghlan at gmail.com>:
> This is fine in an external project, but should never be added to the
> standard library. The barrier to semantic changes that break
> monkeypatching should be high.

The version 0.3 has a known bug: "len=chr; print(len('A'))" is
optimized, whereas it should not. It is now fixed in the new version
0.3.1.

>> * Call methods of builtin types if the object and arguments are constants.
>>   Examples:
>>
>>   - u"h\\xe9ho".encode("utf-8") => b"h\\xc3\\xa9ho"
>>   - "python2.7".startswith("python") => True
>>   - (32).bit_length() => 6
>>   - float.fromhex("0x1.8p+0") => 1.5
>
> That last one isn't constant, it's a name lookup.

Well, yes, but in CPython, it is not possible to modify float.fromhex
attribute, nor unicode.encode.

>>   - print(1.5) => print("1.5")
>
> The print example runs afoul of the general rule above: not in the
> standard library, because you're changing the values seen by a mocked
> version of print()

Ah yes, you found a bug. I forgot to disable this optimization by
default (when builtin_funcs feature is disabled). Fixed in 0.3.1.

>>   - not(x in y) => x not in y
>
> This (and the "is") equivalent should be OK

Note: not(x is y) is also implemented.

>> * Loop: replace range() with xrange() on Python 2, and list with
>>   tuple.  Examples:
>>
>>   - for x in range(n): ... => for x in xrange(n): ...
>>   - for x in [1, 2, 3]: ... => for x in (1, 2, 3): ...
>
> Name lookup optimisations again: not in the standard library.

Correct, same issue than print(): forgot in version 0.3, and also
fixed in 0.3.1.

> def f(): return 1; yield
> if DEBUG: yield
> while 0: yield

I'm using the test suite of Python 2.7 and 3.3 using my optimizer. "if
0: yield" is a known captcha (there is a test) and it is handled
correctly by astoptimizer.

By the way, "return 3" is not removed in a generator because it must
raise a SyntaxError.

>>>> def f():
> ...     if 0:
> ...         global x
> ...     return x
> ...
>>>> f()
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "<stdin>", line 4, in f
> NameError: global name 'x' is not defined

Oh, nice catch. But it is possible to detect the usage of global and
nonlocal, and don't remove an expression if they are present. Fixed in
0.3.1.

Victor

From victor.stinner at gmail.com  Tue Sep 11 23:47:06 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 11 Sep 2012 23:47:06 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <k2o0st$ef6$1@ger.gmane.org>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<k2o0st$ef6$1@ger.gmane.org>
Message-ID: <CAMpsgwYRsR7XU-dxhndqRmSjAxB+p1YUMAD7rDOu8tCEO_d-sg@mail.gmail.com>

>> * Call builtin functions if arguments are constants. Examples:
>>
>>    - len("abc") => 3
>>    - ord("A") => 65
>
> Does this mean transformation print("A") => None and output at compile time?

Only a subset of builtin functions are called at compile time, and not
with any constant value. See astoptimizer/config_builtin_funcs.py.

print() has other optimizations, but it is not called at compile time.

>> * Loop: replace range() with xrange() on Python 2,
>
> range(0, 10**10, 10**9)

Oh. I didn't know this difference between range and xrange. I should
add more checks on arguments.

>> * Evaluate unary and binary operators, subscript and comparaison if all
>>    arguments are constants. Examples:
>>
>>    - 1 + 2 * 3 => 7
>
> Does this mean 1/0 raises exception at compile time?

Such trap are handled carefully. There are many checks to decide if an
operator can be optimized or not.
Example: "\U0010ffff"[0] is not optimized by default.

>> * Remove dead code. Examples:
>>
>>    - def f(): return 1; return 2 => def f(): return 1
>>    - if DEBUG: print("debug") => pass with DEBUG declared as False
>>    - while 0: ... => pass
>
> As Nick said, it could change the semantics, if there were local variable
> assignment or yield in removed code.

yield is handled correctly, but local variable is a new problem.

I should find something to detect local variables. I already have code
to handle variable, but it not fully enabled because there are known
bugs.

> It would be good if the optimizer checked if the built-in names overridden
> in this module or function and disabled this dangerous optimization in such
> case.

Simple cases like "len=chr" or "from math import floor as int" are now
handled correctly in the new version 0.3.1.

> set([1, 2, 3]) => {1, 2, 3}

I don't see how to create literal set or frozen in AST.

set([...]) may be converted to set((...)) at least.

It would be nice to have a new ast.Lit() in Python 3.4: the idea was
already proposed (with a patch, 0_ast.patch) in the issue #11549.

> set([x for ...]) => {x for ...}
> dict([(k, v) for ...]) => {k: v for ...}
> dict((k, v) for ...) => {k: v for ...}
> ''.join([s for ...]) => ''.join(s for ...)
> a.extend([s for ...]) => a.extend(s for ...)

These optimizations look correct.

> (f(x) for x in a) => map(f, a)
> (x.y for x in a) => map(operator.attrgetter('y'), a)
> (x[0] for x in a) => map(operator.itemgetter(0), a)
> (2 * x for x in a) => map((2).__mul__, a)
> (x in b for x in a) => map(b.__contains__, a)
> map(lambda x: x.strip(), a) => (x.strip() for x in a)

Is it faster? :-)

> x in ['i', 'em', 'cite'] => x in {'i', 'em', 'cite'}

A list can contain non-hashable objects, whereas a set can not.

> x == 'i' or x == 'em' or x == 'cite'] => x in {'i', 'em', 'cite'}

You need to know the type of x. Depending on the type, x.__eq__ and
x.__contains__ may be completly different.

> a = []; for ...: a.append(x) => a = [x for ...]
> a = (); for ...: a.add(x) => a = {x for ...}
> a = {}; for ...: a[k] = v => a = {k: v for ...}

Yeah, the first example is already in the TODO list :-) But it's not
trivial. I prefer to start with unrolling loops :-)

> for ...: f.write(...) => __fwrite = f.write; for ...: __fwrite(...)

f.write lookup cannot be optimized.

> x = x + 1 => x += 1
> x = x + ' ' => x += ' '

I don't know if these optimizations are safe.

> x = x + [y] => x.append(y)
> x = x + [y, z] => x.extend([y, z])

It behaves differently if x is not a list, but str for example.

> 'x=%s' % repr(x) => 'x=%a' % (x,)

I don't understand this one.

> 'x=%s' % x + s => 'x=%s%s' % (x, s)
> x = x + ', [%s]' % y => x = '%s, [%s]' % (x, y)

Doesn't work if s type is not str.

> range(0, x) => range(x)

Is it faster?

> for i in range(len(a)): x = a[i] ... => for i, x in enumerate(a): ...
> i = 0; for x in a: i += 1 ... => for i, x in enumerate(a, 1): ...
> i = 1; for x in a: ... i += 1 => for i, x in enumerate(a, 1): ...
> s = 0; for ...: if ...: s += 1 => s = sum(1 for ... if ...)

These optimizations look interesting.

> while True: s = f.readline(); if not s: break; ... => for s in f: ...

Too much assumptions on f type.

> def f(x): ... len() ... => def f(x, __len = len): ... __len() ...

Ah yes, I read somewhere than locals are faster than builtins. But
this specific example is wrong, f(1, 2) must fail. It is possible to
write something like:

def create_f():
  len = len
  def f(x):
     ... len ...
  return f
f = create_f(); del f

Victor

From victor.stinner at gmail.com  Tue Sep 11 23:47:40 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 11 Sep 2012 23:47:40 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <k2o1k9$l1a$1@ger.gmane.org>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<CADiSq7dNXsXjKC12KFrfoVDRF9LjzMPUTHMyKQrgHqQShrTn9Q@mail.gmail.com>
	<k2o1k9$l1a$1@ger.gmane.org>
Message-ID: <CAMpsgwYstiNgZGyEVYgzHo+sdxxkOo8C376xEG_xUh_RzOSS3Q@mail.gmail.com>

>>>   - frozenset("ab") | frozenset("bc") => frozenset("abc")
>
> That's a name lookup, too.

Yes, and it is only optimized if the "builtin_funcs" feature is
enabled explictly.

> Except that evaluating something like '"abc" * constant' can eat up all
> memory, imagine this code:
>
>     KILL_MEMORY = sys.argv[1] == 'NEVER PASS THIS VALUE'
>
>     def test():
>         if KILL_MEMORY:
>             return 'abc' * 10000000000000000000000000000
>         else:
>             return 'abc'

The optimizer always check the result of an optimization for
constraints. For example, an optimization is cancelled if the result
is not immutable. Integers must be in the range [-2^128; 2^128-1] by
default. Strings are limited to 4096 bytes/characters. Tuples are
limited to 20 items.

The compiler may waste some seconds to evaluate an expression and then
drop the result, but I don't really care right now. I prefer to spend
minutes to compile if the application is faster at runtime.

astoptimizer tries to avoid expensive operations when it knows that
the result will be too big. For your specific example, str * int is
*not* evulated if the result will be longer than the limit. Similar
sanity checks may be added later for pow(int, int) and
math.factorial(int) for example.

Victor

From python at mrabarnett.plus.com  Tue Sep 11 23:55:26 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 11 Sep 2012 22:55:26 +0100
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CAMpsgwbJUEk4=rL6=JABE0Pc0368FJKuOzdO_0C6EiVicTpAHw@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<CADiSq7dNXsXjKC12KFrfoVDRF9LjzMPUTHMyKQrgHqQShrTn9Q@mail.gmail.com>
	<CAMpsgwbJUEk4=rL6=JABE0Pc0368FJKuOzdO_0C6EiVicTpAHw@mail.gmail.com>
Message-ID: <504FB34E.3060600@mrabarnett.plus.com>

On 11/09/2012 22:46, Victor Stinner wrote:
> 2012/9/11 Nick Coghlan <ncoghlan at gmail.com>:
>> This is fine in an external project, but should never be added to the
>> standard library. The barrier to semantic changes that break
>> monkeypatching should be high.
>
> The version 0.3 has a known bug: "len=chr; print(len('A'))" is
> optimized, whereas it should not. It is now fixed in the new version
> 0.3.1.
>
>>> * Call methods of builtin types if the object and arguments are constants.
>>>   Examples:
>>>
>>>   - u"h\\xe9ho".encode("utf-8") => b"h\\xc3\\xa9ho"
>>>   - "python2.7".startswith("python") => True
>>>   - (32).bit_length() => 6
>>>   - float.fromhex("0x1.8p+0") => 1.5
>>
>> That last one isn't constant, it's a name lookup.
>
> Well, yes, but in CPython, it is not possible to modify float.fromhex
> attribute, nor unicode.encode.
>
It's possible to shadow 'float':

 >>> float = "foo"
 >>> float
'foo'


From larudwer at freenet.de  Wed Sep 12 01:36:51 2012
From: larudwer at freenet.de (larudwer)
Date: Wed, 12 Sep 2012 01:36:51 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
Message-ID: <07171B3102E0424C8F233F4750C2C211@ruedigerPC>

Hello,

i am a longtime Reader of this list and this is the first time i a dare to 
speak up.
Apology in advance for any noise, silly comments and not posting to 
python-ideas ;).

Am 11.09.2012 12:41, schrieb Victor Stinner:
> I plan to implement other optimizations like unrolling loop or convert
> a loop to a list comprehension, see the TODO file.
>
> Don't hesitate to propose more optimizations if you have some ideas ;-)

Well how about implementing guards like in pypy?
They are not optimizations by themselves but should enable a whole lot of 
them.

I am not good at explaining so i've written this short example (python 
2.7.3).
I hope it makes clear what i mean.

Thank you for reading
larudwer

#
# Example for implementing an guard
# by freezing global and local dictionary
# it is not an optimization by itself
# but should open the possibility for them.
#

class WorldChangedException(Exception): pass
class WorldShadowedException(Exception): pass

class frozendict(dict):
    def __setitem__(self, key, value):
        if key in globals():
            if self == locals():
                raise WorldShadowedException(key)
            else:
                raise WorldChangedException(key)

def put(string):
    print string

g = frozendict({'put': put, 'foo': 2})
l = frozendict({'bar': 'work fast'})

#
# This is just an poor example.
# astoptimizer should be able to generate
# much better code
#
f = compile("""
put( "trying fast path")
put(bar)
put = 27
""", "<astoptimizer>", "exec")

def test():
    global put # Workaround for UnboundLocalError
    put("trying slow path")
    put = 27

#
# Guard: since the world is frozen, nothing can change.
# if the fast path succeeds we take the result, else
# we use the slow path.
#
try:
    exec(f, g, l)
except WorldChangedException, e:
    print "WorldChangedException Caught", e
    test()
except WorldShadowedException, e:
    print "WorldShadowedException Caught", e
    test()



From barry at python.org  Wed Sep 12 01:55:37 2012
From: barry at python.org (Barry Warsaw)
Date: Tue, 11 Sep 2012 19:55:37 -0400
Subject: [Python-Dev] Issue 15906;
 regression in argparse in Python 3.3, 3.2, and 2.7
In-Reply-To: <k2oaia$2uo$1@ger.gmane.org>
References: <20120911113430.426e35eb@limelight.wooz.org>
	<k2nrns$un2$1@ger.gmane.org>
	<20120911153139.18a228f7@resist.wooz.org>
	<k2oaia$2uo$1@ger.gmane.org>
Message-ID: <20120911195537.1a102a94@resist.wooz.org>

On Sep 11, 2012, at 05:30 PM, Terry Reedy wrote:

>> And yet, that's how it works in 2.7, 3.2, and 3.3.1.
>
>In all 2.7 or 3.2?

It will be broken in 2.7.4 and 3.2.4, but the Ubuntu packages in 12.10 are
currently affected because they pulled in some Mercurial updates which
included these unreleased changes.  That's where it was first noticed.

>How about a third rule which amounts to the second *plus* (maybe) the first:
>apply the conversion when it fulfills the purpose of having type conversions,
>which is to make sure that the caller get objects of the desired class. Also,
>only add necessary restrictions to argparse users.

I've tried the various suggestions out, and I think from a practical point of
view, a fix for the regression in the 2.7, 3.2 and 3.3 branches should be to
apply the one line check for action being a _StoreAction instance.  This seems
to be the easiest way to preserve the current behavior (the fix for issues
#12776 and #11839) and fix issue #15906.

I'll update the issue and apply this change to the three branches.  Georg can
ignore the change for 3.3.0.

-Barry

From victor.stinner at gmail.com  Wed Sep 12 02:27:07 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 12 Sep 2012 02:27:07 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <07171B3102E0424C8F233F4750C2C211@ruedigerPC>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<07171B3102E0424C8F233F4750C2C211@ruedigerPC>
Message-ID: <CAMpsgwaxpZTZrfL1ve3fC=DNi7pePjct1aDFKuhj9unxNKYncg@mail.gmail.com>

> i am a longtime Reader of this list and this is the first time i a dare to
> speak up.
> Apology in advance for any noise, silly comments and not posting to
> python-ideas ;).

Welcome!

> Well how about implementing guards like in pypy?

Guards would allow to generate specialized functions without a JIT.

Dummy example:

def func(arg):
  arg *= 1
  arg += 0
  return arg * 16

Can be optimized to:

def func(arg):
  if type(arg) is int:
    # fast path
    return arg << 4
  else:
    # slow path
    arg *= 1
    arg += 0
    return arg * 16

I prefer a check before executing the code, rather than an exception
and reexecute the same code twice.

Specializing all functions would waste a lot of memory, so it should
only be applied on some very specific cases... It can also be slower
if the guard check is slower than the function body! But I bet that
guards would help to enable more aggressive optimizations, or at least
make some optimizations safe.

Dave Malcolm wrote a patch modifying eval.c to support specialized
functions. See the
http://bugs.python.org/issue10399

I don't know yet what is the best approach for CPython.

--

For the specific case of builtin functions and types, I made two
changes in Python 3.3:

 * the type of the builtins mapping can now be any mapping (Python <=
3.2 requires the dict type, dict subtypes are disallowed)
 * "new" types.MappingProxyType type to create a read-only proxy for a
mapping (see also the rejected PEP 416)

We may combine these two changes to use a read-only mapping for
builtins. It would at least help to ensure that an application does
not monkeypatch builtin functions/types.

Victor

From victor.stinner at gmail.com  Wed Sep 12 02:40:59 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 12 Sep 2012 02:40:59 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CAP7+vJ+30ajtrAmr9Sr6j19qOtcyerMATMjup-J1T5Jx-9xo9Q@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<CAK5idxTUF2t8NmuyVy1dWwp3t90eDq5HFV1bZumJ13joxVbQ_g@mail.gmail.com>
	<CAMpsgwYfA9azq_bd5DPZy=bu3-L0FAjZK77QXE5Tm0UxM0WCVg@mail.gmail.com>
	<504F6F97.1050906@mrabarnett.plus.com>
	<CAP7+vJ+30ajtrAmr9Sr6j19qOtcyerMATMjup-J1T5Jx-9xo9Q@mail.gmail.com>
Message-ID: <CAMpsgwa3yvAF2QtA0ghZpr4V2OpsHs+Ejer_UpqGnXX=LU5RKw@mail.gmail.com>

2012/9/11 Guido van Rossum <guido at python.org>:
> FWIW, I expect that there are few places where len(<constant string>)
> is actually used.

I found one revelant example in the stdlib.

            if path.endswith('.dylib'):
                yield path[:-len('.dylib')] + suffix + '.dylib'
            else:
                yield path + suffix

Cool, I'm not the only developer too lazy to copy/paste len('suffix')
in a Python interpreter and then copy/paste the result in my code :-)

"if text.endswith('suffix'): ... text[:-len('suffix')] ..." is a
common pattern in my code.

> However I expect it to be quite common for ord(),
> where the same reasoning applies.

Projects using the same code base for Python 2 and Python 3 contain a
lot of inefficient code. For example, using the six library, a simple
Unicode literal strings becomes a function code: u('unicode').

I expect that astoptimizer will be able to remove (or at least
reduce!) the overhead of the six library and all checks on the Python
version ("if PYTHON3: ... else: ...").

Victor

From guido at python.org  Wed Sep 12 03:54:01 2012
From: guido at python.org (Guido van Rossum)
Date: Tue, 11 Sep 2012 18:54:01 -0700
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CAMpsgwa3yvAF2QtA0ghZpr4V2OpsHs+Ejer_UpqGnXX=LU5RKw@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<CAK5idxTUF2t8NmuyVy1dWwp3t90eDq5HFV1bZumJ13joxVbQ_g@mail.gmail.com>
	<CAMpsgwYfA9azq_bd5DPZy=bu3-L0FAjZK77QXE5Tm0UxM0WCVg@mail.gmail.com>
	<504F6F97.1050906@mrabarnett.plus.com>
	<CAP7+vJ+30ajtrAmr9Sr6j19qOtcyerMATMjup-J1T5Jx-9xo9Q@mail.gmail.com>
	<CAMpsgwa3yvAF2QtA0ghZpr4V2OpsHs+Ejer_UpqGnXX=LU5RKw@mail.gmail.com>
Message-ID: <CAP7+vJ+YqEWOB_AQMurEWXO_qwS7mj-vfB6yqxfhvgjorJyfCA@mail.gmail.com>

On Tue, Sep 11, 2012 at 5:40 PM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> 2012/9/11 Guido van Rossum <guido at python.org>:
>> FWIW, I expect that there are few places where len(<constant string>)
>> is actually used.
>
> I found one revelant example in the stdlib.
>
>             if path.endswith('.dylib'):
>                 yield path[:-len('.dylib')] + suffix + '.dylib'
>             else:
>                 yield path + suffix
>
> Cool, I'm not the only developer too lazy to copy/paste len('suffix')
> in a Python interpreter and then copy/paste the result in my code :-)
>
> "if text.endswith('suffix'): ... text[:-len('suffix')] ..." is a
> common pattern in my code.

Ok, that's a good use case.

>> However I expect it to be quite common for ord(),
>> where the same reasoning applies.
>
> Projects using the same code base for Python 2 and Python 3 contain a
> lot of inefficient code. For example, using the six library, a simple
> Unicode literal strings becomes a function code: u('unicode').

But are you able to do enough static analysis to feel comfortable that
this is the right u() function? IIRC you said earlier that you're not
even capable of recognizing "len = ord; print(len('a'))" -- if that is
really true, I'm very worried about your optimizer's capacity for
breaking code. I'm not talking about "behind-their-back" changes to
__builtins__ or patching of the module globals. I'm talking about
detecting straightforward definitions that override the identifiers
you are replacing.

> I expect that astoptimizer will be able to remove (or at least
> reduce!) the overhead of the six library and all checks on the Python
> version ("if PYTHON3: ... else: ...").

Hm. Wouldn't it be just as easy to run a source-to-source translator
to remove six artefacts instead of an ast optimizer? Surely some
convention could be adopted that is easy to use, and the tool to do
the translation could be a lot simpler than an ast optimizer.

Sorry for being skeptical, but I'm not excited about advertising this
as a general optimization tool unless you can make it a lot safer.

-- 
--Guido van Rossum (python.org/~guido)

From chris.jerdonek at gmail.com  Wed Sep 12 06:30:57 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Tue, 11 Sep 2012 21:30:57 -0700
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 - Issue #15906: Fix a regression in argparse caused by the preceding change,
In-Reply-To: <3XGqP73SvPzML8@mail.python.org>
References: <3XGqP73SvPzML8@mail.python.org>
Message-ID: <CAOTb1wdj2ka71KA299a3eLGd=CnodCsazLubJXfV+X5Dj8wN+Q@mail.gmail.com>

On Tue, Sep 11, 2012 at 9:13 PM, barry.warsaw
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/25e41fdc4e60
> changeset:   79001:25e41fdc4e60
> parent:      78998:6fea947edead
> parent:      79000:bc342cd7ed96
> user:        Barry Warsaw <barry at python.org>
> date:        Wed Sep 12 00:12:29 2012 -0400
> summary:
>   - Issue #15906: Fix a regression in argparse caused by the preceding change,
>   when action='append', type='str' and default=[].

I didn't have time to respond Barry's e-mail from four hours ago
before this was committed.  I think this change may be problematic.
At the least, I think people should have an opportunity to air their
specific concerns and talk through the implications.

Also, from the discussion it seemed like the sentiment was leaning
towards a different approach for the fix.

I made a comment on the issue with some more extended remarks:

http://bugs.python.org/msg170351

--Chris


>
> files:
>   Lib/argparse.py           |   1 +
>   Lib/test/test_argparse.py |  10 ++++++++++
>   Misc/NEWS                 |   3 +++
>   3 files changed, 14 insertions(+), 0 deletions(-)
>
>
> diff --git a/Lib/argparse.py b/Lib/argparse.py
> --- a/Lib/argparse.py
> +++ b/Lib/argparse.py
> @@ -1961,6 +1961,7 @@
>                      # twice (which may fail) if the argument was given, but
>                      # only if it was defined already in the namespace
>                      if (action.default is not None and
> +                        isinstance(action, _StoreAction) and
>                          hasattr(namespace, action.dest) and
>                          action.default is getattr(namespace, action.dest)):
>                          setattr(namespace, action.dest,
> diff --git a/Lib/test/test_argparse.py b/Lib/test/test_argparse.py
> --- a/Lib/test/test_argparse.py
> +++ b/Lib/test/test_argparse.py
> @@ -4607,6 +4607,16 @@
>          args = parser.parse_args([])
>          self.assertEqual(NS(foo='foo_converted'), args)
>
> +    def test_issue_15906(self):
> +        # Issue #15906: When action='append', type=str, default=[] are
> +        # providing, the dest value was the string representation "[]" when it
> +        # should have been an empty list.
> +        parser = argparse.ArgumentParser()
> +        parser.add_argument('--test', dest='test', type=str,
> +                            default=[], action='append')
> +        args = parser.parse_args([])
> +        self.assertEqual(args.test, [])
> +
>  # ======================
>  # parse_known_args tests
>  # ======================
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -56,6 +56,9 @@
>    especially problematic for the FileType type, as a default file would always
>    be opened, even if a file argument was specified on the command line.
>
> +- Issue #15906: Fix a regression in argparse caused by the preceding change,
> +  when action='append', type='str' and default=[].
> +
>  Extension Modules
>  -----------------
>
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>

From storchaka at gmail.com  Wed Sep 12 09:36:24 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Wed, 12 Sep 2012 10:36:24 +0300
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CAMpsgwYRsR7XU-dxhndqRmSjAxB+p1YUMAD7rDOu8tCEO_d-sg@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<k2o0st$ef6$1@ger.gmane.org>
	<CAMpsgwYRsR7XU-dxhndqRmSjAxB+p1YUMAD7rDOu8tCEO_d-sg@mail.gmail.com>
Message-ID: <k2pdsi$q1n$1@ger.gmane.org>

On 12.09.12 00:47, Victor Stinner wrote:
>> set([x for ...]) => {x for ...}
>> dict([(k, v) for ...]) => {k: v for ...}
>> dict((k, v) for ...) => {k: v for ...}
>> ''.join([s for ...]) => ''.join(s for ...)
>> a.extend([s for ...]) => a.extend(s for ...)
>
> These optimizations look correct.

Actually generator can be slower list comprehension. Especially on 
Python2. I think this is an opportunity to optimize the work with 
generators.

>> (f(x) for x in a) => map(f, a)
>> (x.y for x in a) => map(operator.attrgetter('y'), a)
>> (x[0] for x in a) => map(operator.itemgetter(0), a)
>> (2 * x for x in a) => map((2).__mul__, a)
>> (x in b for x in a) => map(b.__contains__, a)
>> map(lambda x: x.strip(), a) => (x.strip() for x in a)
>
> Is it faster? :-)

Yes, significantly for large sequences. But this transformation is not 
safe in general case. For short sequences possible regression (cost of 
"map" name lookup and function call).

>> x in ['i', 'em', 'cite'] => x in {'i', 'em', 'cite'}
>
> A list can contain non-hashable objects, whereas a set can not.

Agree, it applicable if x is proven str. At least list can be replaced 
by tuple.

>> x == 'i' or x == 'em' or x == 'cite'] => x in {'i', 'em', 'cite'}
>
> You need to know the type of x. Depending on the type, x.__eq__ and
> x.__contains__ may be completly different.

Then => x in ('i', 'em', 'cite') and move forward only if x obviously is 
of the appropriate type.

>> for ...: f.write(...) => __fwrite = f.write; for ...: __fwrite(...)
>
> f.write lookup cannot be optimized.

Yes, it is a dangerous transformation and it is difficult to prove its 
safety. But name lookup is one of the main brakes of Python.

>> x = x + 1 => x += 1
>> x = x + ' ' => x += ' '
>
> I don't know if these optimizations are safe.

It is safe if x is proven number or string. If x is local variable, 
initialized by number/string and modified only by number/string. 
Counters and string accumulators are commonly used.

>> 'x=%s' % repr(x) => 'x=%a' % (x,)
>
> I don't understand this one.

Sorry, it should be => 'x=%r' % (x,). And for more arguments: 'x[' + 
repr(k) + ']=' + repr(v) + ';' => 'x[%r]=%r;' % (k, v). Same for str and 
ascii.

It is not safe (repr can be shadowed).

>> 'x=%s' % x + s => 'x=%s%s' % (x, s)
>> x = x + ', [%s]' % y => x = '%s, [%s]' % (x, y)
>
> Doesn't work if s type is not str.

Yes, this is partially applicable. In many cases, s is a literal or the 
newly formatted string.

>> range(0, x) => range(x)
>
> Is it faster?

Slightly.

>> while True: s = f.readline(); if not s: break; ... => for s in f: ...
>
> Too much assumptions on f type.

I personally would prefer a 2to3-like "modernizer" (as a separate 
utility and as plugins for the IDEs), which would have found some 
templates and offered replacing by a more modern, readable (and possibly 
effective) variant. The decision on the applicability of the 
transformation in the particular case remains for the human. For the 
automatic optimizer remain only simple transformations which deteriorate 
readability, and optimizations which cannot be expressed in the source code.


From tarek at ziade.org  Wed Sep 12 09:47:37 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Wed, 12 Sep 2012 09:47:37 +0200
Subject: [Python-Dev] packaging location ?
Message-ID: <50503E19.1030604@ziade.org>

Hello

I was wondering if anyone knows if the removed Lib/packaging directory 
landed in some other places after it was removed.

We have http://hg.python.org/distutils2 be the 'packaging' version is a 
full py3-renamed version we need to keep mirrored

Cheers
Tarek

From storchaka at gmail.com  Wed Sep 12 09:55:38 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Wed, 12 Sep 2012 10:55:38 +0300
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CAMpsgwYRsR7XU-dxhndqRmSjAxB+p1YUMAD7rDOu8tCEO_d-sg@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<k2o0st$ef6$1@ger.gmane.org>
	<CAMpsgwYRsR7XU-dxhndqRmSjAxB+p1YUMAD7rDOu8tCEO_d-sg@mail.gmail.com>
Message-ID: <k2pf0k$3vd$1@ger.gmane.org>

On 12.09.12 00:47, Victor Stinner wrote:
>> x = x + [y] => x.append(y)
>> x = x + [y, z] => x.extend([y, z])
>
> It behaves differently if x is not a list, but str for example.

Actually even worse. Transformations applicable only if x has not 
aliases. Pseudocode:

   if type(x) is list and refcount(x) == 1:
       list.append(x, y)
   else:
       x = x + [y]

This optimization can be done only in the interpreter, otherwise 
overhead costs will be too high.


From victor.stinner at gmail.com  Wed Sep 12 13:13:57 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 12 Sep 2012 13:13:57 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <k2pdsi$q1n$1@ger.gmane.org>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<k2o0st$ef6$1@ger.gmane.org>
	<CAMpsgwYRsR7XU-dxhndqRmSjAxB+p1YUMAD7rDOu8tCEO_d-sg@mail.gmail.com>
	<k2pdsi$q1n$1@ger.gmane.org>
Message-ID: <CAMpsgwa7OB6m0qy1XN7jJuf4JiVc9AHb0x6xtgV0ayNbGNU6ow@mail.gmail.com>

2012/9/12 Serhiy Storchaka <storchaka at gmail.com>:
>>> set([x for ...]) => {x for ...}
>>> dict([(k, v) for ...]) => {k: v for ...}
>>> dict((k, v) for ...) => {k: v for ...}
>>> ''.join([s for ...]) => ''.join(s for ...)
>>> a.extend([s for ...]) => a.extend(s for ...)
>>
>> These optimizations look correct.
>
> Actually generator can be slower list comprehension. Especially on Python2.
> I think this is an opportunity to optimize the work with generators.

I checked with timeit, and yes: generators are slower :-/ I will
revert this "optimization".

>>> (f(x) for x in a) => map(f, a)
>>> (x.y for x in a) => map(operator.attrgetter('y'), a)
>>> (x[0] for x in a) => map(operator.itemgetter(0), a)
>>> (2 * x for x in a) => map((2).__mul__, a)
>>> (x in b for x in a) => map(b.__contains__, a)
>>> map(lambda x: x.strip(), a) => (x.strip() for x in a)
>>
>> Is it faster? :-)

Benchmark using iterable=tuple(range(n)) and f=str (see attached script).

Python version: 2.7.3 (default, Aug 1 2012, 05:16:07) [GCC 4.6.3]
CPU model: Intel(R) Core(TM) i5 CPU 661 @ 3.33GHz
Platform: Linux-3.2.0-30-generic-pae-i686-with-Ubuntu-12.04-precise
Bits: int=32, long=32, long long=64, pointer=32

[ 3 items ]
679 ns: list comprehesion
1.08 us (+59%): itertools.imap
1.42 us (+109%): generator

[ 10 items ]
1.6 us: itertools.imap
1.64 us: list comprehesion
2.26 us (+41%): generator

[ 1000 items ]
112 us: itertools.imap
144 us (+29%): list comprehesion
156 us (+40%): generator

[ 1000000 items ]
142 ms: itertools.imap
183 ms (+29%): generator
186 ms (+31%): list comprehesion

---

Python version: 3.2.3 (default, May 3 2012, 15:54:42) [GCC 4.6.3]
CPU model: Intel(R) Core(TM) i5 CPU 661 @ 3.33GHz
Platform: Linux-3.2.0-30-generic-pae-i686-with-Ubuntu-12.04-precise
Bits: int=32, long=32, long long=64, pointer=32

[ 3 items ]
1.04 us: list comprehesion
1.21 us (+17%): map
1.51 us (+45%): generator


[ 10 items ]
2.02 us: map
2.29 us (+13%): list comprehesion
2.68 us (+33%): generator


[ 1000 items ]
132 us: map
166 us (+25%): list comprehesion
183 us (+38%): generator


[ 1000000 items ]
182 ms: map
229 ms (+26%): generator
251 ms (+38%): list comprehesion

--

> Yes, significantly for large sequences. But this transformation is not safe
> in general case. For short sequences possible regression (cost of "map" name
> lookup and function call).

So except for very small dataset, map (itertools.imap) is always
faster. List comprehension cannot be replaced with map() because map()
doesn't set the iterator variable to the last item if the iterable.

>>> x in ['i', 'em', 'cite'] => x in {'i', 'em', 'cite'}
> Agree, it applicable if x is proven str. At least list can be replaced by
> tuple.
>>> (...)
>>> x == 'i' or x == 'em' or x == 'cite'] => x in {'i', 'em', 'cite'}
>>> (...)
>>> x = x + 1 => x += 1
>>> x = x + ' ' => x += ' '

Well, type inference would permit more optimizations. It is not implemented yet.

>>> for ...: f.write(...) => __fwrite = f.write; for ...: __fwrite(...)
>>
>> f.write lookup cannot be optimized.
>
> Yes, it is a dangerous transformation and it is difficult to prove its
> safety. But name lookup is one of the main brakes of Python.

Oh sorry, I didn't read correctly the example. Yeah, such optimization
is common and it would help to have an option to enable it. Using type
inference, it may be possible to optimize safetly some cases (ex: if
you know that f is a file).

> Sorry, it should be 'x=%s' % repr(x) => 'x=%r' % (x,)

Ah ok, why not.

>>> range(0, x) => range(x)
>>
>> Is it faster?
>
> Slightly.

timeit gives me exactly the same timing.

> I personally would prefer a 2to3-like "modernizer" (as a separate utility
> and as plugins for the IDEs), which would have found some templates and
> offered replacing by a more modern, readable (and possibly effective)
> variant. The decision on the applicability of the transformation in the
> particular case remains for the human. For the automatic optimizer remain
> only simple transformations which deteriorate readability, and optimizations
> which cannot be expressed in the source code.

If the optimizer sees an interesting optimization but cannot decide if
one option is better than another, it may write an advice for the
developer. The developer can review the advice and decide which option
is the best.

Some patterns are faster or slower depending on the Python versions
:-/ The optimizer may be able to decide which pattern is the best for
the running Python version.

Victor
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bench_gen.py
Type: application/octet-stream
Size: 875 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120912/74724c76/attachment.obj>

From victor.stinner at gmail.com  Wed Sep 12 13:20:55 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 12 Sep 2012 13:20:55 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CAP7+vJ+YqEWOB_AQMurEWXO_qwS7mj-vfB6yqxfhvgjorJyfCA@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<CAK5idxTUF2t8NmuyVy1dWwp3t90eDq5HFV1bZumJ13joxVbQ_g@mail.gmail.com>
	<CAMpsgwYfA9azq_bd5DPZy=bu3-L0FAjZK77QXE5Tm0UxM0WCVg@mail.gmail.com>
	<504F6F97.1050906@mrabarnett.plus.com>
	<CAP7+vJ+30ajtrAmr9Sr6j19qOtcyerMATMjup-J1T5Jx-9xo9Q@mail.gmail.com>
	<CAMpsgwa3yvAF2QtA0ghZpr4V2OpsHs+Ejer_UpqGnXX=LU5RKw@mail.gmail.com>
	<CAP7+vJ+YqEWOB_AQMurEWXO_qwS7mj-vfB6yqxfhvgjorJyfCA@mail.gmail.com>
Message-ID: <CAMpsgwZNHRNyjM46xWPwNN1PvueZ8ZCkoMzqGBvXirA+Mu3qQw@mail.gmail.com>

>> Projects using the same code base for Python 2 and Python 3 contain a
>> lot of inefficient code. For example, using the six library, a simple
>> Unicode literal strings becomes a function code: u('unicode').
>
> But are you able to do enough static analysis to feel comfortable that
> this is the right u() function? IIRC you said earlier that you're not
> even capable of recognizing "len = ord; print(len('a'))" -- if that is
> really true, I'm very worried about your optimizer's capacity for
> breaking code. I'm not talking about "behind-their-back" changes to
> __builtins__ or patching of the module globals. I'm talking about
> detecting straightforward definitions that override the identifiers
> you are replacing.

astoptimizer is still experimental, but I prefer to release early and
release often, because I already get interesting feedback.

"from math import pow as len" is already supported in the version 0.3,
and "len=ord" was fixed in the version 0.3.1.

astoptimizer is supposed to handle any instruction setting variables
(if I forgot to handle a specific instruction, it's a bug). My initial
goal was to optimize "x=1; return x", but then I realized that I must
handle take care of all variables, because they may shadow builtin
functions or constants.

The following AST statements creates a "new" namespace (inherit some
properties from the parent):
 - Module
 - FunctionDef
 - ClassDef

The following AST statements set variables or have an impact on scope:
 - Assign, AugAssign, Del
 - Global, Nonlocal
 - Import, ImportFrom
 - With
 - arguments (of FunctionDef)
 - comprehension (of ListComp or GeneratorExp)

There is an experimental support for assignments. If an unsupported
assignment is found (ex: ((x, y), z) = x_y_z), all optimizations on
names (ex: len("abc") or math.e) are disabled. For example, "from re
import *" disables optimizations (but "from math import *" is
supported).

>> I expect that astoptimizer will be able to remove (or at least
>> reduce!) the overhead of the six library and all checks on the Python
>> version ("if PYTHON3: ... else: ...").
>
> Hm. Wouldn't it be just as easy to run a source-to-source translator
> to remove six artefacts instead of an ast optimizer?

You mean something like 2to3? I understood that the six module is
written for developers who prefer to use the same code base for Python
2 and Python 3.

With Python 3.3, if astoptimizer hooks the compile() builtin, the
optimization is enabled transparently when importing a module (during
.py => .pyc compiling). There is no need to explicitly "compile" an
application.

> Surely some convention could be adopted that is easy to use,
> and the tool to do the translation could be a lot simpler
> than an ast optimizer.

I like AST because I don't need to write my own parser.

> Sorry for being skeptical, but I'm not excited about advertising this
> as a general optimization tool unless you can make it a lot safer.

It is safer at every release. I will use the version number 1.0 when
the optimizer will be completly safe :-) (So it's only 31% safe yet!)

Victor

From raymond.hettinger at gmail.com  Wed Sep 12 14:00:04 2012
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Wed, 12 Sep 2012 08:00:04 -0400
Subject: [Python-Dev] issues found by Coverity
In-Reply-To: <504F1327.4080800@cheimes.de>
References: <504F1327.4080800@cheimes.de>
Message-ID: <9D06C5C8-70AE-401D-A1C5-179AB9737C08@gmail.com>


On Sep 11, 2012, at 6:32 AM, Christian Heimes <lists at cheimes.de> wrote:
> 
> maybe you have noticed a bunch of commits I made the last couple of
> days. 

I noticed!  Thank you for all the work to improve quality.


Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120912/016e32b8/attachment.html>

From fijall at gmail.com  Wed Sep 12 16:04:03 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 12 Sep 2012 16:04:03 +0200
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <CADiSq7dNXsXjKC12KFrfoVDRF9LjzMPUTHMyKQrgHqQShrTn9Q@mail.gmail.com>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<CADiSq7dNXsXjKC12KFrfoVDRF9LjzMPUTHMyKQrgHqQShrTn9Q@mail.gmail.com>
Message-ID: <CAK5idxRSO+RyQyHkZ2ra+dq+SXXd=Hjk+bPph+kxZfb5SNsMew@mail.gmail.com>

On Tue, Sep 11, 2012 at 2:57 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Tue, Sep 11, 2012 at 8:41 PM, Victor Stinner
> <victor.stinner at gmail.com> wrote:
>> * Call builtin functions if arguments are constants. Examples:
>>
>>   - len("abc") => 3
>>   - ord("A") => 65
>
> This is fine in an external project, but should never be added to the
> standard library. The barrier to semantic changes that break
> monkeypatching should be high.
>
> Yes, this is frustrating as it eliminates a great many interesting
> static optimisations that are *probably* OK. That's one of the reasons
> why PyPy uses tracing - it can perform these optimisations *and* still
> include the appropriate dynamic checks.
>
> However, the double barrier of third party module + off by default is
> a suitable activation barrier for ensuring people know that what
> they're doing is producing bytecode that doesn't behave like standard
> Python any more (e.g. tests won't be able to shadow builtins or
> optimised module references). Optimisations that break the language
> semantics are heading towards the same territory as the byteplay and
> withhacks modules (albeit not as evil internally).

The third (and the most important) barrier is that constant folding
len("abc") is essentially useless. You can do some optimizations that
are sound, at probably a great deal of complexity, if you maintain all
the places that constant folded stuff and change them if you shadow a
builtin. This approach is done in PyPy for example in a more
systematic way (by invalidating the assembler).

Anyway, since this is, in it's current shape clearly not designed to
preserve the semantics of python at all, is the discussion of this
package on-topic for python-dev? More so than say discussing Cython or
Numba or any other kind-of-python-but-not-quite project?

Cheers,
fijal

From barry at python.org  Wed Sep 12 16:06:35 2012
From: barry at python.org (Barry Warsaw)
Date: Wed, 12 Sep 2012 10:06:35 -0400
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 - Issue #15906: Fix a regression in argparse caused by the preceding change,
In-Reply-To: <CAOTb1wdj2ka71KA299a3eLGd=CnodCsazLubJXfV+X5Dj8wN+Q@mail.gmail.com>
References: <3XGqP73SvPzML8@mail.python.org>
	<CAOTb1wdj2ka71KA299a3eLGd=CnodCsazLubJXfV+X5Dj8wN+Q@mail.gmail.com>
Message-ID: <20120912100635.5ad0fe45@limelight.wooz.org>

On Sep 11, 2012, at 09:30 PM, Chris Jerdonek wrote:

>I didn't have time to respond Barry's e-mail from four hours ago
>before this was committed.  I think this change may be problematic.
>At the least, I think people should have an opportunity to air their
>specific concerns and talk through the implications.
>
>Also, from the discussion it seemed like the sentiment was leaning
>towards a different approach for the fix.
>
>I made a comment on the issue with some more extended remarks:
>
>http://bugs.python.org/msg170351

The alternative suggested fix breaks the test suite (yes I tried it).  So
maybe we have to go back and decide whether the original fix for #12776 and
#11839 is correct.

More detail in the tracker issue.

-Barry

From stefan at bytereef.org  Wed Sep 12 16:22:59 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Wed, 12 Sep 2012 16:22:59 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Fix out of bounds read in long_new() for empty bytes with an explicit base.
In-Reply-To: <3XH3pq1Xy6zQbQ@mail.python.org>
References: <3XH3pq1Xy6zQbQ@mail.python.org>
Message-ID: <20120912142259.GA30437@sleipnir.bytereef.org>

christian.heimes <python-checkins at python.org> wrote:
>   Fix out of bounds read in long_new() for empty bytes with an explicit base. int(b'', somebase) calls PyLong_FromString() with char* of length 1 but the function accesses the first argument at offset 1. CID 715359
> 
> files:
>   Objects/longobject.c |  4 ++--
>   1 files changed, 2 insertions(+), 2 deletions(-)
> 
> 
> diff --git a/Objects/longobject.c b/Objects/longobject.c
> --- a/Objects/longobject.c
> +++ b/Objects/longobject.c
> @@ -4285,8 +4285,8 @@
>              string = PyByteArray_AS_STRING(x);
>          else
>              string = PyBytes_AS_STRING(x);
> -        if (strlen(string) != (size_t)size) {
> -            /* We only see this if there's a null byte in x,
> +        if (strlen(string) != (size_t)size || !size) {
> +            /* We only see this if there's a null byte in x or x is empty,
>                 x is a bytes or buffer, *and* a base is given. */
>              PyErr_Format(PyExc_ValueError,
>                           "invalid literal for int() with base %d: %R",


This is a false positive:                                                                                    
                                                                                                             
    Assumption: string == ""                                                                                 
                                                                                                             
    Call:  PyLong_FromString("", NULL, (int)base);                                                           
                                                                                                             
        Now: str == ""                                                                                       
                                                                                                             
        Coverity claims an invalid access at str[1]:                                                         
                                                                                                             
            if (str[0] == '0' &&                                                                             
                ((base == 16 && (str[1] == 'x' || str[1] == 'X')) ||                                         
                (base == 8  && (str[1] == 'o' || str[1] == 'O')) ||                                          
                (base == 2  && (str[1] == 'b' || str[1] == 'B'))))                                           
                                                                                                             
        But str[1] is never accessed due to shortcut evaluation.                                             
                                                                                                             
                                                                                                             
Coverity appears to have serious problems with shortcut evaluations in many                                  
places.                                                                                                      
                                                                                                             
                                                                                                             
Stefan Krah                                                                                                  
                                     



From stefan at bytereef.org  Wed Sep 12 16:58:04 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Wed, 12 Sep 2012 16:58:04 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Make sure that *really*
 no more than sizeof(ifr.ifr_name) chars are strcpy-ed
In-Reply-To: <3XFT5z6kCCzQZJ@mail.python.org>
References: <3XFT5z6kCCzQZJ@mail.python.org>
Message-ID: <20120912145804.GA6593@sleipnir.bytereef.org>

christian.heimes <python-checkins at python.org> wrote:
>   Make sure that *really* no more than sizeof(ifr.ifr_name) chars are strcpy-ed to ifr.ifr_name and that the string is *always* NUL terminated. New code shouldn't use strcpy(), too. CID 719692
> 
> files:
>   Modules/socketmodule.c |  3 ++-
>   1 files changed, 2 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c
> --- a/Modules/socketmodule.c
> +++ b/Modules/socketmodule.c
> @@ -1674,7 +1674,8 @@
>              if (len == 0) {
>                  ifr.ifr_ifindex = 0;
>              } else if (len < sizeof(ifr.ifr_name)) {
> -                strcpy(ifr.ifr_name, PyBytes_AS_STRING(interfaceName));
> +                strncpy(ifr.ifr_name, PyBytes_AS_STRING(interfaceName), sizeof(ifr.ifr_name));
> +                ifr.ifr_name[(sizeof(ifr.ifr_name))-1] = '\0';
>                  if (ioctl(s->sock_fd, SIOCGIFINDEX, &ifr) < 0) {
>                      s->errorhandler();
>                      Py_DECREF(interfaceName);


I've trouble finding the overrun in the existing code. Previously:

We have:

    1) len = PyBytes_GET_SIZE(interfaceName); (without NUL terminator)


At the point of strcpy() we have:

    2) len < sizeof(ifr.ifr_name)

PyBytes_AS_STRING(interfaceName) always adds a NUL terminator, so:

    3) len+1 <= sizeof(ifr.ifr_name)

    4) strcpy(ifr.ifr_name, PyBytes_AS_STRING(interfaceName)) will not overrun
       ifr.ifr_name and ifr.ifr_name is always NUL terminated.


So IMO the strcpy() was safe and the report is a false positive.


Stefan Krah




From lists at cheimes.de  Wed Sep 12 17:40:55 2012
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 12 Sep 2012 17:40:55 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Fix out of bounds read in long_new() for empty bytes with an explicit base.
In-Reply-To: <20120912142259.GA30437@sleipnir.bytereef.org>
References: <3XH3pq1Xy6zQbQ@mail.python.org>
	<20120912142259.GA30437@sleipnir.bytereef.org>
Message-ID: <5050AD07.1020906@cheimes.de>

Am 12.09.2012 16:22, schrieb Stefan Krah:
> This is a false positive:                                                                                    
>                                                                                                              
>     Assumption: string == ""                                                                                 
>                                                                                                              
>     Call:  PyLong_FromString("", NULL, (int)base);                                                           
>                                                                                                              
>         Now: str == ""                                                                                       
>                                                                                                              
>         Coverity claims an invalid access at str[1]:                                                         
>                                                                                                              
>             if (str[0] == '0' &&                                                                             
>                 ((base == 16 && (str[1] == 'x' || str[1] == 'X')) ||                                         
>                 (base == 8  && (str[1] == 'o' || str[1] == 'O')) ||                                          
>                 (base == 2  && (str[1] == 'b' || str[1] == 'B'))))                                           
>                                                                                                              
>         But str[1] is never accessed due to shortcut evaluation.                                             
>                                                                                                              
>                                                                                                              
> Coverity appears to have serious problems with shortcut evaluations in many                                  
> places.                                                                                                      

You might be right. But did you notice that there is much more code
beyond the large comment block in PyLong_FromString()? There might be
other code paths that push str beyond its limit.

My change adds an early opt out in an error case and doesn't cause a
performance degradation. I'd have no hard feeling if you'd prefer a
revert but I'd keep the modification as it causes no harm.


From stefan at bytereef.org  Wed Sep 12 17:42:11 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Wed, 12 Sep 2012 17:42:11 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Closed reference leak of variable 'k' in function ste_new which wasn't
 decrefed
In-Reply-To: <3XFWPx4L7ZzQbd@mail.python.org>
References: <3XFWPx4L7ZzQbd@mail.python.org>
Message-ID: <20120912154211.GA9605@sleipnir.bytereef.org>

christian.heimes <python-checkins at python.org> wrote:
> summary:
>   Closed reference leak of variable 'k' in function ste_new which wasn't decrefed in error cases
> 
> files:
>   Python/symtable.c |  3 ++-
>   1 files changed, 2 insertions(+), 1 deletions(-)
> 
> 
> diff --git a/Python/symtable.c b/Python/symtable.c
> --- a/Python/symtable.c
> +++ b/Python/symtable.c
> @@ -24,7 +24,7 @@
>          void *key, int lineno, int col_offset)
>  {
>      PySTEntryObject *ste = NULL;
> -    PyObject *k;
> +    PyObject *k = NULL;
>  
>      k = PyLong_FromVoidPtr(key);
>      if (k == NULL)
> @@ -79,6 +79,7 @@
>  
>      return ste;
>   fail:
> +    Py_XDECREF(k);
>      Py_XDECREF(ste);

I think 'k' is owned by the PySTEntryObject after it is assigned here:

ste->ste_id = k;


So ste_dealloc() will call Py_XDECREF(k) a second time.


Stefan Krah



From lists at cheimes.de  Wed Sep 12 18:00:52 2012
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 12 Sep 2012 18:00:52 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Closed reference leak of variable 'k' in function ste_new which wasn't
 decrefed
In-Reply-To: <20120912154211.GA9605@sleipnir.bytereef.org>
References: <3XFWPx4L7ZzQbd@mail.python.org>
	<20120912154211.GA9605@sleipnir.bytereef.org>
Message-ID: <5050B1B4.20503@cheimes.de>

Am 12.09.2012 17:42, schrieb Stefan Krah:
> I think 'k' is owned by the PySTEntryObject after it is assigned here:
> 
> ste->ste_id = k;
> 
> 
> So ste_dealloc() will call Py_XDECREF(k) a second time.

You are right. I missed that ste steals the reference to k and does its
own cleanup. I've fixed the issue and moved Py_DECREF(k) into ste ==
NULL block.

http://hg.python.org/cpython/rev/2888356cdd4e
http://hg.python.org/cpython/rev/99ab7006e466

Thanks!

From tjreedy at udel.edu  Wed Sep 12 18:14:32 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 12 Sep 2012 12:14:32 -0400
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Fix out of bounds read in long_new() for empty bytes with an explicit base.
In-Reply-To: <20120912142259.GA30437@sleipnir.bytereef.org>
References: <3XH3pq1Xy6zQbQ@mail.python.org>
	<20120912142259.GA30437@sleipnir.bytereef.org>
Message-ID: <k2qcdq$ho5$1@ger.gmane.org>

On 9/12/2012 10:22 AM, Stefan Krah wrote:
> christian.heimes <python-checkins at python.org> wrote:
>> Fix out of bounds read in long_new() for empty bytes with an explicit base.
 >> int(b'', somebase) calls PyLong_FromString() with char* of length 1

I don't know what happens internally, but such calls raise
ValueError: invalid literal for int() with base 16: ''
Of course, even if int() traps such calls before calling
PyLong_FromString, an extension writer could goof.

Does the length 1 come from added \0?

By the way, excessively long lines in checkin messages are a nuisance 
from reading and responding ;-).


-- 
Terry Jan Reedy


From lists at cheimes.de  Wed Sep 12 18:33:35 2012
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 12 Sep 2012 18:33:35 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Fix out of bounds read in long_new() for empty bytes with an explicit base.
In-Reply-To: <k2qcdq$ho5$1@ger.gmane.org>
References: <3XH3pq1Xy6zQbQ@mail.python.org>
	<20120912142259.GA30437@sleipnir.bytereef.org>
	<k2qcdq$ho5$1@ger.gmane.org>
Message-ID: <5050B95F.4060907@cheimes.de>

Am 12.09.2012 18:14, schrieb Terry Reedy:
> On 9/12/2012 10:22 AM, Stefan Krah wrote:
>> christian.heimes <python-checkins at python.org> wrote:
>>> Fix out of bounds read in long_new() for empty bytes with an explicit
>>> base.
>>> int(b'', somebase) calls PyLong_FromString() with char* of length 1
> 
> I don't know what happens internally, but such calls raise
> ValueError: invalid literal for int() with base 16: ''
> Of course, even if int() traps such calls before calling
> PyLong_FromString, an extension writer could goof.
> 
> Does the length 1 come from added \0?

Coverity (a static code analyzing tool) claims that the some code paths
may read beyond the end of data when an empty byte string and any base
is given. Internally b'' is converted to a null terminated char array
(PyBytes_AS_STRING() returns a null terminated char*).

My change shortcuts the execution path and immediately raises an
exception for the combination of b'' and some base.

> By the way, excessively long lines in checkin messages are a nuisance
> from reading and responding ;-).

Sorry! In the future I'll add more line breaks. :)


From tjreedy at udel.edu  Wed Sep 12 18:35:54 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 12 Sep 2012 12:35:54 -0400
Subject: [Python-Dev] Release of astoptimizer 0.3
In-Reply-To: <k2pdsi$q1n$1@ger.gmane.org>
References: <CAMpsgwaRzhMhRXVjyKCMefaVotVmTFV=Fks1pLXDBxC81FvmSg@mail.gmail.com>
	<k2o0st$ef6$1@ger.gmane.org>
	<CAMpsgwYRsR7XU-dxhndqRmSjAxB+p1YUMAD7rDOu8tCEO_d-sg@mail.gmail.com>
	<k2pdsi$q1n$1@ger.gmane.org>
Message-ID: <k2qdlt$tb7$1@ger.gmane.org>

On 9/12/2012 3:36 AM, Serhiy Storchaka wrote:

> I personally would prefer a 2to3-like "modernizer" (as a separate
> utility and as plugins for the IDEs), which would have found some
> templates and offered replacing by a more modern, readable (and possibly
> effective) variant. The decision on the applicability of the
> transformation in the particular case remains for the human.

IDLE has a plug-in mechanism, though I am not familiar with it yet. It 
also has a built-in parser of some sort. It is used, for instance, to 
determine the function expression that preceeds '(' in order to get the 
function object for a tool tip.

 > For the
> automatic optimizer remain only simple transformations which deteriorate
> readability, and optimizations which cannot be expressed in the source
> code.

I had just made the same observation that some of the proposed 
optimization are really source transformations and others are only ast 
or even lower level changes. We also need to differentiate changes which 
are pretty much guaranteed to be faster (at least with current releases) 
and those those might be with particular hardware, os, and python version.

-- 
Terry Jan Reedy


From stefan at bytereef.org  Wed Sep 12 18:37:27 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Wed, 12 Sep 2012 18:37:27 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default):
 Fix out of bounds read in long_new() for empty bytes with an explicit base.
In-Reply-To: <5050AD07.1020906@cheimes.de>
References: <3XH3pq1Xy6zQbQ@mail.python.org>
	<20120912142259.GA30437@sleipnir.bytereef.org>
	<5050AD07.1020906@cheimes.de>
Message-ID: <20120912163727.GA20852@sleipnir.bytereef.org>

Christian Heimes <lists at cheimes.de> wrote:
> Am 12.09.2012 16:22, schrieb Stefan Krah:
> > This is a false positive:
> 
> You might be right. But did you notice that there is much more code
> beyond the large comment block in PyLong_FromString()? There might be
> other code paths that push str beyond its limit.

Yes, I understand. My reasoning was different: The str[1] location Coverity
pointed out is a false positive. I checked other locations and they seem to
be okay, too.

Now, because there's so much code my first instinct would be not to touch
it unless there's a proven invalid access. This is to avoid subtle behavior
changes.


> My change adds an early opt out in an error case and doesn't cause a
> performance degradation. I'd have no hard feeling if you'd prefer a
> revert but I'd keep the modification as it causes no harm.

As far as I can see, only the error message is affected. Previously:

>>> int(b'', 0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: ''


Now the fact that base=0 is converted to base=10 is lost:

>>> int(b'', 0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 0: b''


No big deal of course, but still a change.



Stefan Krah



From eric at netwok.org  Wed Sep 12 21:02:42 2012
From: eric at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Wed, 12 Sep 2012 15:02:42 -0400
Subject: [Python-Dev] packaging location ?
In-Reply-To: <50503E19.1030604@ziade.org>
References: <50503E19.1030604@ziade.org>
Message-ID: <5050DC52.508@netwok.org>

Hi,

Lib/packaging is in the repository history, and in my backup clones, but
it?s not visible in any branch head as we have no branch for 3.4 yet.  I
can bring the directory back with a simple Mercurial command.

However, it?s not clear to me that we want to do that.  At the inception
of the project, we wanted a new distutils with support for the latest
PEPs and improved extensibility.  Then we found a number of problems in
the PEPs; the last time I pointed the problems out I got no reply but
?find a PEP dictator and propose changes?.  And when I started the
thread about removing packaging in 3.3, hundreds of replies discussed
changing the whole distutils architecture, splitting the project,
exploring new systems, etc., which is why I?m not sure that we can just
bring back packaging in 3.4 as it was and continue with our previous
roadmap.

Cheers

From solipsis at pitrou.net  Wed Sep 12 21:21:45 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 12 Sep 2012 21:21:45 +0200
Subject: [Python-Dev] packaging location ?
References: <50503E19.1030604@ziade.org>
	<5050DC52.508@netwok.org>
Message-ID: <20120912212145.03d46e53@pitrou.net>

On Wed, 12 Sep 2012 15:02:42 -0400
?ric Araujo <eric at netwok.org> wrote:
> Hi,
> 
> Lib/packaging is in the repository history, and in my backup clones, but
> it?s not visible in any branch head as we have no branch for 3.4 yet.  I
> can bring the directory back with a simple Mercurial command.
> 
> However, it?s not clear to me that we want to do that.  At the inception
> of the project, we wanted a new distutils with support for the latest
> PEPs and improved extensibility.  Then we found a number of problems in
> the PEPs; the last time I pointed the problems out I got no reply but
> ?find a PEP dictator and propose changes?.  And when I started the
> thread about removing packaging in 3.3, hundreds of replies discussed
> changing the whole distutils architecture, splitting the project,
> exploring new systems, etc., which is why I?m not sure that we can just
> bring back packaging in 3.4 as it was and continue with our previous
> roadmap.

People who want a whole new distutils architecture can start distutils3
(or repackaging) if they want. If I have to give my advice, I would
favour re-integrating packaging in the stdlib or, better, integrating
all changes, one by one, into distutils itself.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From robertc at robertcollins.net  Wed Sep 12 21:27:32 2012
From: robertc at robertcollins.net (Robert Collins)
Date: Thu, 13 Sep 2012 07:27:32 +1200
Subject: [Python-Dev] packaging location ?
In-Reply-To: <5050DC52.508@netwok.org>
References: <50503E19.1030604@ziade.org>
	<5050DC52.508@netwok.org>
Message-ID: <CAJ3HoZ19so3BMFDuGHuKvnQpXd+y+Qv5_tu7TH98a1gayqw+1w@mail.gmail.com>

On Thu, Sep 13, 2012 at 7:02 AM, ?ric Araujo <eric at netwok.org> wrote:
> Hi,
>
> Lib/packaging is in the repository history, and in my backup clones, but
> it?s not visible in any branch head as we have no branch for 3.4 yet.  I
> can bring the directory back with a simple Mercurial command.
>
> However, it?s not clear to me that we want to do that.  At the inception
> of the project, we wanted a new distutils with support for the latest
> PEPs and improved extensibility.  Then we found a number of problems in
> the PEPs; the last time I pointed the problems out I got no reply but
> ?find a PEP dictator and propose changes?.  And when I started the
> thread about removing packaging in 3.3, hundreds of replies discussed
> changing the whole distutils architecture, splitting the project,
> exploring new systems, etc., which is why I?m not sure that we can just
> bring back packaging in 3.4 as it was and continue with our previous
> roadmap.
>
> Cheers

+1 - FWIW, I'd like to see the previous project drive for
consolidation done without landing in the stdlib, /until/ you've got
something that folk are willingly migrating to en masse - at that
point we'll know that the bulk of use cases are well satisfied, and
that we won't need to be fighting an uphill battle for adoption.

If folk are saying 'I would adopt but its not in the stdlib', well - I
think thats ignorable TBH: the market of adopters that matter are
those using setuptools/distribute/$other_thing today.

I rather suspect we'll face things like 'I still support Python2.7'
more than 'its not in the stdlib' for now.

-Rob (who was just looking into what the state of the art to choose
was yesterday)

From regebro at gmail.com  Wed Sep 12 21:28:35 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 12 Sep 2012 21:28:35 +0200
Subject: [Python-Dev] packaging location ?
In-Reply-To: <5050DC52.508@netwok.org>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
Message-ID: <CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>

On Wed, Sep 12, 2012 at 9:02 PM, ?ric Araujo <eric at netwok.org> wrote:
> ?find a PEP dictator and propose changes?.  And when I started the
> thread about removing packaging in 3.3, hundreds of replies discussed
> changing the whole distutils architecture, splitting the project,
> exploring new systems, etc.,

Yes, yes, but that's just the same old drama that pops up every time
this is discussed with the same old arguments all over again. We'll
never get anywhere if we care about *that*.

The way to go forward is via PEPs, fix them if needed, implement in a
separate package, stick into stdlib once it works.

//Lennart

From dholth at gmail.com  Wed Sep 12 21:56:36 2012
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 12 Sep 2012 15:56:36 -0400
Subject: [Python-Dev] packaging location ?
In-Reply-To: <CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
Message-ID: <CAG8k2+4-Mah9EwbTMQwn3cRSVLiRN3VT4H7jZcbGW6rBRf8U2Q@mail.gmail.com>

On Wed, Sep 12, 2012 at 3:28 PM, Lennart Regebro <regebro at gmail.com> wrote:
> On Wed, Sep 12, 2012 at 9:02 PM, ?ric Araujo <eric at netwok.org> wrote:
>> ?find a PEP dictator and propose changes?.  And when I started the
>> thread about removing packaging in 3.3, hundreds of replies discussed
>> changing the whole distutils architecture, splitting the project,
>> exploring new systems, etc.,
>
> Yes, yes, but that's just the same old drama that pops up every time
> this is discussed with the same old arguments all over again. We'll
> never get anywhere if we care about *that*.
>
> The way to go forward is via PEPs, fix them if needed, implement in a
> separate package, stick into stdlib once it works.
>
> //Lennart

I'm happy to note that as of version 0.6.28 distutils (the setuptools
fork) can now consume PEP 345 / 376 "Database of Installed Python
Distributions" installations. Entry points could probably go in as an
extension to the metadata, but at the moment they work as
entry_points.txt with no changes and would be harmlessly ignored by
"import packaging".

From eric at netwok.org  Wed Sep 12 21:57:33 2012
From: eric at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Wed, 12 Sep 2012 15:57:33 -0400
Subject: [Python-Dev] packaging location ?
In-Reply-To: <CAG8k2+4-Mah9EwbTMQwn3cRSVLiRN3VT4H7jZcbGW6rBRf8U2Q@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAG8k2+4-Mah9EwbTMQwn3cRSVLiRN3VT4H7jZcbGW6rBRf8U2Q@mail.gmail.com>
Message-ID: <5050E92D.9020002@netwok.org>

> I'm happy to note that as of version 0.6.28 distutils (the setuptools
> fork)

You certainly mean distribute. :-)

From brett at python.org  Thu Sep 13 00:07:42 2012
From: brett at python.org (Brett Cannon)
Date: Wed, 12 Sep 2012 18:07:42 -0400
Subject: [Python-Dev] packaging location ?
In-Reply-To: <CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
Message-ID: <CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>

On Wed, Sep 12, 2012 at 3:28 PM, Lennart Regebro <regebro at gmail.com> wrote:

> On Wed, Sep 12, 2012 at 9:02 PM, ?ric Araujo <eric at netwok.org> wrote:
> > ?find a PEP dictator and propose changes?.  And when I started the
> > thread about removing packaging in 3.3, hundreds of replies discussed
> > changing the whole distutils architecture, splitting the project,
> > exploring new systems, etc.,
>
> Yes, yes, but that's just the same old drama that pops up every time
> this is discussed with the same old arguments all over again. We'll
> never get anywhere if we care about *that*.
>
> The way to go forward is via PEPs, fix them if needed, implement in a
> separate package, stick into stdlib once it works.
>

I agree with Lennart's and Antoine's advice of just move forward with what
we have. If some PEPs need fixing then let's fix them, but we don't need to
rock the horse even more by going overboard. Getting the sane, core bits
into the stdlib as packaging is meant to is plenty to take on. If people
want to reinvent stuff they can do it elsewhere. I personally don't care if
it is done inside or outside the stdlib initially or if it stays in
packaging or goes directly into distutils, but forward movement with what
we have is the most important thing.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120912/fbc0e864/attachment.html>

From rdmurray at bitdance.com  Thu Sep 13 00:43:07 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 12 Sep 2012 18:43:07 -0400
Subject: [Python-Dev] packaging location ?
In-Reply-To: <CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
Message-ID: <20120912224308.56B48250180@webabinitio.net>

On Wed, 12 Sep 2012 18:07:42 -0400, Brett Cannon <brett at python.org> wrote:
> On Wed, Sep 12, 2012 at 3:28 PM, Lennart Regebro <regebro at gmail.com> wrote:
> 
> > On Wed, Sep 12, 2012 at 9:02 PM, ??ric Araujo <eric at netwok.org> wrote:
> > > ???find a PEP dictator and propose changes???.  And when I started the
> > > thread about removing packaging in 3.3, hundreds of replies discussed
> > > changing the whole distutils architecture, splitting the project,
> > > exploring new systems, etc.,
> >
> > Yes, yes, but that's just the same old drama that pops up every time
> > this is discussed with the same old arguments all over again. We'll
> > never get anywhere if we care about *that*.
> >
> > The way to go forward is via PEPs, fix them if needed, implement in a
> > separate package, stick into stdlib once it works.
> >
> 
> I agree with Lennart's and Antoine's advice of just move forward with what
> we have. If some PEPs need fixing then let's fix them, but we don't need to
> rock the horse even more by going overboard. Getting the sane, core bits
> into the stdlib as packaging is meant to is plenty to take on. If people
> want to reinvent stuff they can do it elsewhere. I personally don't care if
> it is done inside or outside the stdlib initially or if it stays in
> packaging or goes directly into distutils, but forward movement with what
> we have is the most important thing.

When the removal was being pondered, the possibility of keeping certain
bits that were more ready than others was discussed.  Perhaps the best
way forward is to put it back in bits, with the most finished (and PEP
relevant) stuff going in first.  That might also give non-packaging
people bite-sized-enough chunks to actually digest and help with.

--David

From ncoghlan at gmail.com  Thu Sep 13 03:14:17 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 13 Sep 2012 11:14:17 +1000
Subject: [Python-Dev] packaging location ?
In-Reply-To: <20120912224308.56B48250180@webabinitio.net>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
Message-ID: <CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>

On Thu, Sep 13, 2012 at 8:43 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> When the removal was being pondered, the possibility of keeping certain
> bits that were more ready than others was discussed.  Perhaps the best
> way forward is to put it back in bits, with the most finished (and PEP
> relevant) stuff going in first.  That might also give non-packaging
> people bite-sized-enough chunks to actually digest and help with.

This is the plan I'm going to propose. The previous approach was to
just throw the entirety of distutils2 in there, but there are some
hard questions that doesn't address, and some use cases it doesn't
handle. So, rather than importing it wholesale and making the stdlib
the upstream for distutils2, I believe it makes more sense for
distutils2 to remain an independent project, and we cherry pick bits
and pieces for the standard library's new packaging module as they
stabilise.

In particular, Tarek was focused on being able to create *binary* RPMs
automatically. That isn't enough for my purposes, I need to be able to
create *source* RPMs, which can then be fed to the koji build service
for conversion to binary RPMs in accordance with the (ideally)
autogenerated spec file. A binary RPM that isn't built from a source
RPM is no good to me, and the distutils2 support for this approach is
awful, because setup.cfg inherits all the command model cruft from
distutils which is stupidly hard to integrate with other build
systems. I also want to be able to automate most dependency
management, so people can write "Requires: python(pypi-dist-name)" in
their RPM spec files and have it work, just as they can already write
things like "Requires: perl(File::Rules)"

I'm currently working on creating a coherent map of the status quo,
that describes the overall process of software distribution across
various phases (from development -> source archive -> building ->
binary archive -> installation -> import) and looks at the tools and
formats which exist at each step, both legacy
(distutils/setuptools/distribute) and proposed (e.g. distutils2,
bento, wheel), and the kinds of tasks which need to be automated.

Part of the problem with distutils is that the phases of software
distribution are not clearly documented, instead being implicit in the
behaviour of setuptools. The distutils2 project, to date, has not
remedied that deficiency, instead retaining the implicit overall
workflow and just hacking on various pieces in order to "fix Python
packaging". If we're going to get support from the scientific
community (which has some of the more exotic build requirements going
around) and the existing community that wants the full setuptools
feature set rather than the subset currently standardised (primarily
non-Django web developers in my experience), then we need to address
*their* concerns as well, not just the concerns of those of us that
don't like the opaque nature of setuptools and its preference for
guessing in the presence of ambiguity.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From solipsis at pitrou.net  Thu Sep 13 11:38:57 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 13 Sep 2012 11:38:57 +0200
Subject: [Python-Dev] packaging location ?
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
Message-ID: <20120913113857.69dafcb1@pitrou.net>

On Thu, 13 Sep 2012 11:14:17 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Thu, Sep 13, 2012 at 8:43 AM, R. David Murray <rdmurray at bitdance.com> wrote:
> > When the removal was being pondered, the possibility of keeping certain
> > bits that were more ready than others was discussed.  Perhaps the best
> > way forward is to put it back in bits, with the most finished (and PEP
> > relevant) stuff going in first.  That might also give non-packaging
> > people bite-sized-enough chunks to actually digest and help with.
> 
> This is the plan I'm going to propose. The previous approach was to
> just throw the entirety of distutils2 in there, but there are some
> hard questions that doesn't address, and some use cases it doesn't
> handle. So, rather than importing it wholesale and making the stdlib
> the upstream for distutils2, I believe it makes more sense for
> distutils2 to remain an independent project, and we cherry pick bits
> and pieces for the standard library's new packaging module as they
> stabilise.

How is that going to be useful? Most people use distutils / packaging as
an application, not a library. If you provide only a subset of
the necessary features, people won't use packaging.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From donald.stufft at gmail.com  Thu Sep 13 13:27:12 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Thu, 13 Sep 2012 07:27:12 -0400
Subject: [Python-Dev] packaging location ?
In-Reply-To: <20120913113857.69dafcb1@pitrou.net>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net>
Message-ID: <E74B7F9F8ED147B8B01DE99D898A21E7@gmail.com>

On Thursday, September 13, 2012 at 5:38 AM, Antoine Pitrou wrote:
> Most people use distutils / packaging as
> an application, not a library. If you provide only a subset of
> the necessary features, people won't use packaging.

Not that I think current usage patterns matter since moving from setup.py 
to a static file, but that's not really true. The wide proliferation of setuptools
shows pretty clearly that people are fine using distutils as a library. Even
beyond that the popularity of pip shows that as well since very few people
even directly interact with setup.py at all except to create the distributions.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120913/403d0ae6/attachment.html>

From tarek at ziade.org  Thu Sep 13 13:32:35 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 13 Sep 2012 13:32:35 +0200
Subject: [Python-Dev] packaging location ?
In-Reply-To: <20120913113857.69dafcb1@pitrou.net>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net>
Message-ID: <5051C453.3080003@ziade.org>

On 9/13/12 11:38 AM, Antoine Pitrou wrote:
> On Thu, 13 Sep 2012 11:14:17 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Thu, Sep 13, 2012 at 8:43 AM, R. David Murray <rdmurray at bitdance.com> wrote:
>>> When the removal was being pondered, the possibility of keeping certain
>>> bits that were more ready than others was discussed.  Perhaps the best
>>> way forward is to put it back in bits, with the most finished (and PEP
>>> relevant) stuff going in first.  That might also give non-packaging
>>> people bite-sized-enough chunks to actually digest and help with.
>> This is the plan I'm going to propose. The previous approach was to
>> just throw the entirety of distutils2 in there, but there are some
>> hard questions that doesn't address, and some use cases it doesn't
>> handle. So, rather than importing it wholesale and making the stdlib
>> the upstream for distutils2, I believe it makes more sense for
>> distutils2 to remain an independent project, and we cherry pick bits
>> and pieces for the standard library's new packaging module as they
>> stabilise.
> How is that going to be useful? Most people use distutils / packaging as
> an application, not a library. If you provide only a subset of
> the necessary features, people won't use packaging.
>
> Regards
>
> Antoine.

Yeah but we've been too ambitious.

Here's my proposal - actually it's Nick's proposal but I want to make 
sure we're on the same
page wrt steps, and I think that addresses Antoine concerns

1. create a new package, called pkglib (or whatever), located at hg 
.python.org as a new project that just strictly contains :

  - the PEP implementations
  - non controversial features like files parser, pypi index browser etc

   it's doable - since that's what we have done in distutils2. the 
modules that implements those PEPs are standalone

   Let's avoid by all means to put the old distutils command logic there.

   Let's have a strict process on every new thing we're adding there.


2. make pkglib python 2 *and* python 3 compatible - natively, not w/ 2to3


3. make distutils2, distribute, pip, bento, etc.  use that and try to 
share as many bits as possible


4. ask each project to pour in pkglib anything that can be reused by others


when 3.4 comes around, I guess we can decide if pkglib can go in or not.

That way, we won't have the usual controversy about distutils' command 
machinery. People will use whatever tool
and hopefully this tool will be based on pkgutil


>


From chris.jerdonek at gmail.com  Thu Sep 13 13:47:15 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Thu, 13 Sep 2012 04:47:15 -0700
Subject: [Python-Dev] packaging location ?
In-Reply-To: <5051C453.3080003@ziade.org>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
Message-ID: <CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>

On Thu, Sep 13, 2012 at 4:32 AM, Tarek Ziad? <tarek at ziade.org> wrote:
> Here's my proposal - actually it's Nick's proposal but I want to make sure
> we're on the same
> page wrt steps, and I think that addresses Antoine concerns
>
> 1. create a new package, called pkglib (or whatever), located at hg
> .python.org as a new project that just strictly contains :
>
> ...
>
> That way, we won't have the usual controversy about distutils' command
> machinery. People will use whatever tool
> and hopefully this tool will be based on pkgutil

Hopefully it will be based on pkglib (or whatever) rather than pkgutil. ;)

--Chris

From donald.stufft at gmail.com  Thu Sep 13 13:48:22 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Thu, 13 Sep 2012 07:48:22 -0400
Subject: [Python-Dev] packaging location ?
In-Reply-To: <5051C453.3080003@ziade.org>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
Message-ID: <70636C49B0954536ADDF566BA3F6D488@gmail.com>

On Thursday, September 13, 2012 at 7:32 AM, Tarek Ziad? wrote:
>  
> Yeah but we've been too ambitious.
>  
> Here's my proposal - actually it's Nick's proposal but I want to make  
> sure we're on the same
> page wrt steps, and I think that addresses Antoine concerns
>  
> 1. create a new package, called pkglib (or whatever), located at hg  
> .python.org (http://python.org) as a new project that just strictly contains :
>  
> - the PEP implementations
> - non controversial features like files parser, pypi index browser etc
>  
> it's doable - since that's what we have done in distutils2. the  
> modules that implements those PEPs are standalone
>  
> Let's avoid by all means to put the old distutils command logic there.
>  
> Let's have a strict process on every new thing we're adding there.
>  
>  
> 2. make pkglib python 2 *and* python 3 compatible - natively, not w/ 2to3
>  
>  
> 3. make distutils2, distribute, pip, bento, etc. use that and try to  
> share as many bits as possible
>  
>  
> 4. ask each project to pour in pkglib anything that can be reused by others
I started messing around with yanking some of the parts of distutils2 (things I've been
calling packaging primitives for lack of a better word). Don't have anything particularly
usable yet, but the approach you're talking about is similar to what I started to do.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120913/aa3103fb/attachment.html>

From p.f.moore at gmail.com  Thu Sep 13 14:45:50 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 13 Sep 2012 13:45:50 +0100
Subject: [Python-Dev] packaging location ?
In-Reply-To: <5051C453.3080003@ziade.org>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
Message-ID: <CACac1F8oCseX6g1Y563rgW3aMB8immSVW5QFC-JnFuSXtz1JCw@mail.gmail.com>

On 13 September 2012 12:32, Tarek Ziad? <tarek at ziade.org> wrote:
> Here's my proposal - actually it's Nick's proposal but I want to make sure
> we're on the same
> page wrt steps, and I think that addresses Antoine concerns
>
> 1. create a new package, called pkglib (or whatever), located at hg
> .python.org as a new project that just strictly contains :
>
>  - the PEP implementations
>  - non controversial features like files parser, pypi index browser etc
>
>   it's doable - since that's what we have done in distutils2. the modules
> that implements those PEPs are standalone

+1. I've seen far too many implementation of code that reads/writes
RECORD files for instance. They are all subtly different and manage to
have incompatible obscure bugs :-(

A reference implementation would be an excellent thing. In the stdlib
by preference, external for older versions is as good as you can get.
The key here is to avoid having packaging tools with any more
dependencies than necessary (because there's a bootstrapping issue
with installing those dependencies...)

>   Let's avoid by all means to put the old distutils command logic there.

+1

>   Let's have a strict process on every new thing we're adding there.

Hmm. Agreed up to a point, but please let's not make it so hard to
change things that are present that people go off and do their own
thing again[1]. OTOH, I agree let's be cautious about adding new
things. Once pkglib goes into the stdlib, that's when things get
strict. Let's leave a bit of flexibility while the details are
thrashed out.

> 2. make pkglib python 2 *and* python 3 compatible - natively, not w/ 2to3

+0

> 3. make distutils2, distribute, pip, bento, etc.  use that and try to share
> as many bits as possible

We can't "make" anyone use pkglib, but if it clearly makes it easier
to support the standards than implementing them yourself, it should
persuade people. And we should certainly advocate supporting PEPs by
using the reference implementation rather than reimplementing it
yourself.

> 4. ask each project to pour in pkglib anything that can be reused by others

+1, although again it'll be down to the projects whether they do
actually contribute. Also this somewhat contradicts the "be strict"
point above, which is why I'm lukewarm on "be strict". Practicality vs
purity - getting contributors/users is more important than insisting
that everything be standardised before it can be implemented.

> when 3.4 comes around, I guess we can decide if pkglib can go in or not.

I'd have an explicit goal to be included in 3.4 (so that people can
imagine an end to the need for pkgutil being a dependency of their
installation tool). But agreed it should not be assumed that this will
happen if it's not ready (that was the mistake with packaging).

> That way, we won't have the usual controversy about distutils' command
> machinery. People will use whatever tool
> and hopefully this tool will be based on pkgutil

+1. It also means that people will have a much greater incentive to
support the new standards, because it should simply be a case of
calling the pkglib implementation.

Paul.

From tarek at ziade.org  Thu Sep 13 15:15:19 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Thu, 13 Sep 2012 15:15:19 +0200
Subject: [Python-Dev] packaging location ?
In-Reply-To: <CACac1F8oCseX6g1Y563rgW3aMB8immSVW5QFC-JnFuSXtz1JCw@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CACac1F8oCseX6g1Y563rgW3aMB8immSVW5QFC-JnFuSXtz1JCw@mail.gmail.com>
Message-ID: <5051DC67.4070802@ziade.org>

On 9/13/12 2:45 PM, Paul Moore wrote:
>> 4. ask each project to pour in pkglib anything that can be reused by others
> +1, although again it'll be down to the projects whether they do
> actually contribute. Also this somewhat contradicts the "be strict"
> point above, which is why I'm lukewarm on "be strict". Practicality vs
> purity - getting contributors/users is more important than insisting
> that everything be standardised before it can be implemented.

Let me take back 'strict process' and replace it with:

Everything added in pkglib should be a basic feature/implementation that 
does not force the tools to change the
way *they* see their build/release/UI processes.

The most sophisticated feature I am thinking about is the set of apis in 
distutils2.pypi
They are APIs to browse and interact with PyPI, and they are useful to 
many projects,  so I think it fits my definition.

otho, anything relating to compilation should not be added there, unless 
it's APIs to get some aggregated info,
like on the top of platform/sys/etc


Cheers
Tarek

From chris.jerdonek at gmail.com  Thu Sep 13 15:25:52 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Thu, 13 Sep 2012 06:25:52 -0700
Subject: [Python-Dev] unsupported doctest directives in
	Doc/library/ctypes.rst
Message-ID: <CAOTb1wd+t72qrhRvpFc=oktsHVg8v2p5H6334WBMj3PdUdMKuQ@mail.gmail.com>

In the process of studying the global state of our doctests, I noticed
that the doctests in Doc/library/ctypes.rst use the following doctest
directives:

# doctest: +LINUX
# doctest: +WINDOWS

However, I'm not able to find evidence that these directives are
implemented anywhere.  Thus I get an error like the following when
trying to parse the file using doctest:

"ValueError: line 55 of the doctest for ctypes.rst has an invalid
option: '+WINDOWS'"

The reST file says this towards the top:

"Note: The code samples in this tutorial use :mod:`doctest` to make sure that
they actually work.  Since some code samples behave differently under Linux,
Windows, or Mac OS X, they contain doctest directives in comments.""

The farthest back I was able to trace this paragraph was here from April 2006:

http://hg.python.org/cpython/rev/0e0e315b4202

in which the docs seem to have been added from reST sources outside
the repository (from ctypes 0.9.9.6 upstream -- see the parent
changeset).

I would like for our docs to be parseable by doctest without error.
Possible options seem to include at least (1) changing these
directives to normal code comments until we can support these
directives in a meaningful way, and (2) calling
doctest.register_optionflag() for these directives as no-ops as
appropriate to make the errors go away.

Does anyone have any thoughts on this?

Thanks,
--Chris

From ncoghlan at gmail.com  Thu Sep 13 16:34:29 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 14 Sep 2012 00:34:29 +1000
Subject: [Python-Dev] packaging location ?
In-Reply-To: <CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
Message-ID: <CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>

On Thu, Sep 13, 2012 at 9:47 PM, Chris Jerdonek
<chris.jerdonek at gmail.com> wrote:
> On Thu, Sep 13, 2012 at 4:32 AM, Tarek Ziad? <tarek at ziade.org> wrote:
>> Here's my proposal - actually it's Nick's proposal but I want to make sure
>> we're on the same
>> page wrt steps, and I think that addresses Antoine concerns
>>
>> 1. create a new package, called pkglib (or whatever), located at hg
>> .python.org as a new project that just strictly contains :
>>
>> ...
>>
>> That way, we won't have the usual controversy about distutils' command
>> machinery. People will use whatever tool
>> and hopefully this tool will be based on pkgutil
>
> Hopefully it will be based on pkglib (or whatever) rather than pkgutil. ;)

Actually, I'd be happy to do the rearrangement needed to turn pkgutil
into a package rather than the current single module. I think we'd
have people breaking out the torches and pitchforks if we ever ended
up with modules or packages called packaging, pkglib *and* pkgutil all
in the standard library. "distcore" might work, since they're core
functionality for distribution more so than they are for packages.

Anyway, the essential idea of getting those 4 modules (and support
libraries) that almost made it into 3.3 into sufficiently good shape
that they can be distributed independently of distutils2 and adopted
as a dependency by multiple projects is a good one.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Thu Sep 13 16:36:29 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 14 Sep 2012 00:36:29 +1000
Subject: [Python-Dev] packaging location ?
In-Reply-To: <CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
Message-ID: <CADiSq7c=qrMa+=ThUkxvRq7iY9oxsvh03KWmBgbWMdAT7NrJGw@mail.gmail.com>

On Fri, Sep 14, 2012 at 12:34 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Anyway, the essential idea of getting those 4 modules (and support
> libraries) that almost made it into 3.3 into sufficiently good shape
> that they can be distributed independently of distutils2 and adopted
> as a dependency by multiple projects is a good one.

I also like Tarek's suggestion of including the pypi client APIs.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From merwok at netwok.org  Thu Sep 13 16:39:09 2012
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 13 Sep 2012 10:39:09 -0400
Subject: [Python-Dev] packaging location ?
In-Reply-To: <CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
Message-ID: <5051F00D.3040807@netwok.org>

Hi,

Le 13/09/2012 10:34, Nick Coghlan a ?crit :
> Actually, I'd be happy to do the rearrangement needed to turn pkgutil
> into a package rather than the current single module.

I very much prefer not mixing pkgutil (dealing with packages that you
import) and build/distribution/installation support (dealing with
packages/bundles/distributions/parcels/stuff that your distribute).

Regards

From ncoghlan at gmail.com  Thu Sep 13 17:04:00 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 14 Sep 2012 01:04:00 +1000
Subject: [Python-Dev] packaging location ?
In-Reply-To: <5051F00D.3040807@netwok.org>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
Message-ID: <CADiSq7cDMKmiSW+nE1oLhqL3zgGa1zs2Sh14xmK9JbedQgY=bw@mail.gmail.com>

On Fri, Sep 14, 2012 at 12:39 AM, ?ric Araujo <merwok at netwok.org> wrote:
> Hi,
>
> Le 13/09/2012 10:34, Nick Coghlan a ?crit :
>> Actually, I'd be happy to do the rearrangement needed to turn pkgutil
>> into a package rather than the current single module.
>
> I very much prefer not mixing pkgutil (dealing with packages that you
> import) and build/distribution/installation support (dealing with
> packages/bundles/distributions/parcels/stuff that your distribute).

Tarek started it when he suggested "pkglib" :)

I like "distcore" or "distlib", though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Thu Sep 13 17:04:02 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 14 Sep 2012 01:04:02 +1000
Subject: [Python-Dev] packaging location ?
In-Reply-To: <5051F00D.3040807@netwok.org>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
Message-ID: <CADiSq7fXkwETzDzy=pNY-HWyp2=g8=B-amS0kvHR22NcsevH5A@mail.gmail.com>

On Fri, Sep 14, 2012 at 12:39 AM, ?ric Araujo <merwok at netwok.org> wrote:
> Hi,
>
> Le 13/09/2012 10:34, Nick Coghlan a ?crit :
>> Actually, I'd be happy to do the rearrangement needed to turn pkgutil
>> into a package rather than the current single module.
>
> I very much prefer not mixing pkgutil (dealing with packages that you
> import) and build/distribution/installation support (dealing with
> packages/bundles/distributions/parcels/stuff that your distribute).

Tarek started it when he suggested "pkglib" :)

I like "distcore" or "distlib", though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From erik.m.bray at gmail.com  Thu Sep 13 17:13:19 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Thu, 13 Sep 2012 11:13:19 -0400
Subject: [Python-Dev] packaging location ?
In-Reply-To: <20120913113857.69dafcb1@pitrou.net>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net>
Message-ID: <CAOTD34ZONfYrSBni6Z5=uMKK0--0mu4p-0DtAV80kux_48KBPg@mail.gmail.com>

On Thu, Sep 13, 2012 at 5:38 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Thu, 13 Sep 2012 11:14:17 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>> On Thu, Sep 13, 2012 at 8:43 AM, R. David Murray <rdmurray at bitdance.com> wrote:
>> > When the removal was being pondered, the possibility of keeping certain
>> > bits that were more ready than others was discussed.  Perhaps the best
>> > way forward is to put it back in bits, with the most finished (and PEP
>> > relevant) stuff going in first.  That might also give non-packaging
>> > people bite-sized-enough chunks to actually digest and help with.
>>
>> This is the plan I'm going to propose. The previous approach was to
>> just throw the entirety of distutils2 in there, but there are some
>> hard questions that doesn't address, and some use cases it doesn't
>> handle. So, rather than importing it wholesale and making the stdlib
>> the upstream for distutils2, I believe it makes more sense for
>> distutils2 to remain an independent project, and we cherry pick bits
>> and pieces for the standard library's new packaging module as they
>> stabilise.
>
> How is that going to be useful? Most people use distutils / packaging as
> an application, not a library. If you provide only a subset of
> the necessary features, people won't use packaging.

Third-party install/packing software (pip, bento, even distribute) can
still gradually absorb any standard pieces added to the stdlib for
better interoperability and PEP compliance.  I'm still strongly in
favor of a `pysetup` like command making it into Python too, but in
the meantime the top priority should be anything that supports better
consistency across existing projects.

From chris.jerdonek at gmail.com  Fri Sep 14 01:22:09 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Thu, 13 Sep 2012 16:22:09 -0700
Subject: [Python-Dev] [Python-checkins] cpython (2.7): #15437,
 #15439: merge Doc/ACKS.txt with Misc/ACKS and modify Doc/about.rst
In-Reply-To: <3XHwLr547zzQjv@mail.python.org>
References: <3XHwLr547zzQjv@mail.python.org>
Message-ID: <CAOTb1wdkqmvHdYx_LbX=qz5tCKQKYg+WJrOKydtCELkE14-HfA@mail.gmail.com>

On Thu, Sep 13, 2012 at 4:00 PM, ezio.melotti
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/76dd082d332e
> changeset:   79022:76dd082d332e
> branch:      2.7
> parent:      79014:8f847f66a49f
> user:        Ezio Melotti <ezio.melotti at gmail.com>
> date:        Fri Sep 14 01:58:33 2012 +0300
> summary:
>   #15437, #15439: merge Doc/ACKS.txt with Misc/ACKS and modify Doc/about.rst accordingly.

I also contributed to this. :)

--Chris



> files:
>   Doc/ACKS.txt  |  231 --------------------------------------
>   Doc/about.rst |    9 +-
>   Misc/ACKS     |  119 ++++++++++++++++++-
>   3 files changed, 122 insertions(+), 237 deletions(-)
>
>
> diff --git a/Doc/ACKS.txt b/Doc/ACKS.txt
> deleted file mode 100644
> --- a/Doc/ACKS.txt
> +++ /dev/null
> @@ -1,231 +0,0 @@
> -Contributors to the Python Documentation
> -----------------------------------------
> -
> -This section lists people who have contributed in some way to the Python
> -documentation.  It is probably not complete -- if you feel that you or
> -anyone else should be on this list, please let us know (send email to
> -docs at python.org), and we'll be glad to correct the problem.
> -
> -.. acks::
> -
> -   * Aahz
> -   * Michael Abbott
> -   * Steve Alexander
> -   * Jim Ahlstrom
> -   * Fred Allen
> -   * A. Amoroso
> -   * Pehr Anderson
> -   * Oliver Andrich
> -   * Heidi Annexstad
> -   * Jes?s Cea Avi?n
> -   * Manuel Balsera
> -   * Daniel Barclay
> -   * Chris Barker
> -   * Don Bashford
> -   * Anthony Baxter
> -   * Alexander Belopolsky
> -   * Bennett Benson
> -   * Jonathan Black
> -   * Robin Boerdijk
> -   * Michal Bozon
> -   * Aaron Brancotti
> -   * Georg Brandl
> -   * Keith Briggs
> -   * Ian Bruntlett
> -   * Lee Busby
> -   * Arnaud Calmettes
> -   * Lorenzo M. Catucci
> -   * Carl Cerecke
> -   * Mauro Cicognini
> -   * Gilles Civario
> -   * Mike Clarkson
> -   * Steve Clift
> -   * Dave Cole
> -   * Matthew Cowles
> -   * Jeremy Craven
> -   * Andrew Dalke
> -   * Ben Darnell
> -   * L. Peter Deutsch
> -   * Robert Donohue
> -   * Fred L. Drake, Jr.
> -   * Josip Dzolonga
> -   * Jeff Epler
> -   * Michael Ernst
> -   * Blame Andy Eskilsson
> -   * Carey Evans
> -   * Martijn Faassen
> -   * Carl Feynman
> -   * Dan Finnie
> -   * Hern?n Mart?nez Foffani
> -   * Stefan Franke
> -   * Jim Fulton
> -   * Peter Funk
> -   * Lele Gaifax
> -   * Matthew Gallagher
> -   * Gabriel Genellina
> -   * Ben Gertzfield
> -   * Nadim Ghaznavi
> -   * Jonathan Giddy
> -   * Shelley Gooch
> -   * Nathaniel Gray
> -   * Grant Griffin
> -   * Thomas Guettler
> -   * Anders Hammarquist
> -   * Mark Hammond
> -   * Harald Hanche-Olsen
> -   * Manus Hand
> -   * Gerhard H?ring
> -   * Travis B. Hartwell
> -   * Tim Hatch
> -   * Janko Hauser
> -   * Ben Hayden
> -   * Thomas Heller
> -   * Bernhard Herzog
> -   * Magnus L. Hetland
> -   * Konrad Hinsen
> -   * Stefan Hoffmeister
> -   * Albert Hofkamp
> -   * Gregor Hoffleit
> -   * Steve Holden
> -   * Thomas Holenstein
> -   * Gerrit Holl
> -   * Rob Hooft
> -   * Brian Hooper
> -   * Randall Hopper
> -   * Mike Hoy
> -   * Michael Hudson
> -   * Eric Huss
> -   * Jeremy Hylton
> -   * Roger Irwin
> -   * Jack Jansen
> -   * Philip H. Jensen
> -   * Pedro Diaz Jimenez
> -   * Kent Johnson
> -   * Lucas de Jonge
> -   * Andreas Jung
> -   * Robert Kern
> -   * Jim Kerr
> -   * Jan Kim
> -   * Kamil Kisiel
> -   * Greg Kochanski
> -   * Guido Kollerie
> -   * Peter A. Koren
> -   * Daniel Kozan
> -   * Andrew M. Kuchling
> -   * Dave Kuhlman
> -   * Erno Kuusela
> -   * Ross Lagerwall
> -   * Thomas Lamb
> -   * Detlef Lannert
> -   * Piers Lauder
> -   * Glyph Lefkowitz
> -   * Robert Lehmann
> -   * Marc-Andr? Lemburg
> -   * Ross Light
> -   * Ulf A. Lindgren
> -   * Everett Lipman
> -   * Mirko Liss
> -   * Martin von L?wis
> -   * Fredrik Lundh
> -   * Jeff MacDonald
> -   * John Machin
> -   * Andrew MacIntyre
> -   * Vladimir Marangozov
> -   * Vincent Marchetti
> -   * Westley Mart?nez
> -   * Laura Matson
> -   * Daniel May
> -   * Rebecca McCreary
> -   * Doug Mennella
> -   * Paolo Milani
> -   * Skip Montanaro
> -   * Paul Moore
> -   * Ross Moore
> -   * Sjoerd Mullender
> -   * Dale Nagata
> -   * Michal Nowikowski
> -   * Steffen Daode Nurpmeso
> -   * Ng Pheng Siong
> -   * Koray Oner
> -   * Tomas Oppelstrup
> -   * Denis S. Otkidach
> -   * Zooko O'Whielacronx
> -   * Shriphani Palakodety
> -   * William Park
> -   * Joonas Paalasmaa
> -   * Harri Pasanen
> -   * Bo Peng
> -   * Tim Peters
> -   * Benjamin Peterson
> -   * Christopher Petrilli
> -   * Justin D. Pettit
> -   * Chris Phoenix
> -   * Fran?ois Pinard
> -   * Paul Prescod
> -   * Eric S. Raymond
> -   * Edward K. Ream
> -   * Terry J. Reedy
> -   * Sean Reifschneider
> -   * Bernhard Reiter
> -   * Armin Rigo
> -   * Wes Rishel
> -   * Armin Ronacher
> -   * Jim Roskind
> -   * Guido van Rossum
> -   * Donald Wallace Rouse II
> -   * Mark Russell
> -   * Nick Russo
> -   * Chris Ryland
> -   * Constantina S.
> -   * Hugh Sasse
> -   * Bob Savage
> -   * Scott Schram
> -   * Neil Schemenauer
> -   * Barry Scott
> -   * Joakim Sernbrant
> -   * Justin Sheehy
> -   * Charlie Shepherd
> -   * Yue Shuaijie
> -   * Michael Simcich
> -   * Ionel Simionescu
> -   * Michael Sloan
> -   * Gregory P. Smith
> -   * Roy Smith
> -   * Clay Spence
> -   * Nicholas Spies
> -   * Tage Stabell-Kulo
> -   * Frank Stajano
> -   * Anthony Starks
> -   * Greg Stein
> -   * Peter Stoehr
> -   * Mark Summerfield
> -   * Reuben Sumner
> -   * Kalle Svensson
> -   * Jim Tittsler
> -   * David Turner
> -   * Sandro Tosi
> -   * Ville Vainio
> -   * Martijn Vries
> -   * Charles G. Waldman
> -   * Greg Ward
> -   * Barry Warsaw
> -   * Corran Webster
> -   * Glyn Webster
> -   * Bob Weiner
> -   * Eddy Welbourne
> -   * Jeff Wheeler
> -   * Mats Wichmann
> -   * Gerry Wiener
> -   * Timothy Wild
> -   * Paul Winkler
> -   * Collin Winter
> -   * Blake Winton
> -   * Dan Wolfe
> -   * Adam Woodbeck
> -   * Steven Work
> -   * Thomas Wouters
> -   * Ka-Ping Yee
> -   * Rory Yorke
> -   * Moshe Zadka
> -   * Milan Zamazal
> -   * Cheng Zhang
> diff --git a/Doc/about.rst b/Doc/about.rst
> --- a/Doc/about.rst
> +++ b/Doc/about.rst
> @@ -29,8 +29,13 @@
>  See :ref:`reporting-bugs` for information how to report bugs in this
>  documentation, or Python itself.
>
> -.. including the ACKS file here so that it can be maintained separately
> -.. include:: ACKS.txt
> +
> +Contributors to the Python Documentation
> +----------------------------------------
> +
> +Many people have contributed to the Python language, the Python standard
> +library, and the Python documentation.  See :source:`Misc/ACKS` in the Python
> +source distribution for a partial list of contributors.
>
>  It is only with the input and contributions of the Python community
>  that Python has such wonderful documentation -- Thank You!
> diff --git a/Misc/ACKS b/Misc/ACKS
> --- a/Misc/ACKS
> +++ b/Misc/ACKS
> @@ -11,25 +11,33 @@
>  PS: In the standard Python distribution, this file is encoded in UTF-8
>  and the list is in rough alphabetical order by last names.
>
> +Aahz
> +Michael Abbott
>  David Abrahams
>  Jim Ahlstrom
>  Farhan Ahmad
>  Nir Aides
>  Yaniv Aknin
>  Jyrki Alakuijala
> +Steve Alexander
> +Fred Allen
>  Billy G. Allie
>  Kevin Altis
>  Joe Amenta
> +A. Amoroso
>  Mark Anacker
>  Anders Andersen
>  John Anderson
> +Pehr Anderson
>  Erik Anders?n
>  Oliver Andrich
>  Ross Andrus
> +Heidi Annexstad
>  ?ric Araujo
>  Jason Asbahr
>  David Ascher
>  Chris AtLee
> +Jes?s Cea Avi?n
>  John Aycock
>  Jan-Hein B?hrman
>  Donovan Baarda
> @@ -41,8 +49,10 @@
>  Greg Ball
>  Luigi Ballabio
>  Jeff Balogh
> +Manuel Balsera
>  Matt Bandy
>  Michael J. Barber
> +Daniel Barclay
>  Chris Barker
>  Anton Barkovsky
>  Nick Barnes
> @@ -51,6 +61,7 @@
>  Cesar Eduardo Barros
>  Des Barry
>  Ulf Bartelt
> +Don Bashford
>  Nick Bastin
>  Jeff Bauer
>  Mike Bayer
> @@ -69,6 +80,7 @@
>  Alexander ?????? Belopolsky
>  Andrew Bennetts
>  Andy Bensky
> +Bennett Benson
>  Michel Van den Bergh
>  Eric Beser
>  Steven Bethard
> @@ -80,6 +92,7 @@
>  Philippe Biondi
>  Stuart Bishop
>  Roy Bixler
> +Jonathan Black
>  Mike Bland
>  Martin Bless
>  Pablo Bleyer
> @@ -88,6 +101,7 @@
>  Finn Bock
>  Paul Boddie
>  Matthew Boedicker
> +Robin Boerdijk
>  David Bolen
>  Gawain Bolton
>  Gregory Bond
> @@ -96,13 +110,16 @@
>  Eric Bouck
>  Thierry Bousch
>  Sebastian Boving
> +Michal Bozon
>  Jeff Bradberry
> +Aaron Brancotti
>  Monty Brandenberg
>  Georg Brandl
>  Christopher Brannon
>  Terrence Brannon
>  Dave Brennan
>  Tom Bridgman
> +Keith Briggs
>  Richard Brodie
>  Michael Broghton
>  Daniel Brotsky
> @@ -111,6 +128,7 @@
>  Oleg Broytmann
>  Dave Brueck
>  Francisco Mart?n Brugu?
> +Ian Bruntlett
>  Stan Bubrouski
>  Erik de Bueger
>  Dick Bulterman
> @@ -134,6 +152,7 @@
>  Donn Cave
>  Charles Cazabon
>  Per Cederqvist
> +Carl Cerecke
>  Octavian Cerna
>  Pascal Chambon
>  John Chandler
> @@ -151,8 +170,10 @@
>  Anders Chrigstr?m
>  Tom Christiansen
>  Vadim Chugunov
> +Mauro Cicognini
>  David Cinege
>  Craig Citro
> +Gilles Civario
>  Mike Clarkson
>  Andrew Clegg
>  Brad Clements
> @@ -180,6 +201,7 @@
>  Matthew Dixon Cowles
>  Ryan Coyner
>  Christopher A. Craig
> +Jeremy Craven
>  Laura Creighton
>  Simon Cross
>  Drew Csillag
> @@ -202,6 +224,7 @@
>  Arnaud Delobelle
>  Erik Demaine
>  John Dennis
> +L. Peter Deutsch
>  Roger Dev
>  Philippe Devalkeneer
>  Raghuram Devarakonda
> @@ -215,6 +238,7 @@
>  Daniel Dittmar
>  Jaromir Dolecek
>  Ismail Donmez
> +Robert Donohue
>  Marcos Donolo
>  Dima Dorfman
>  Cesar Douady
> @@ -263,10 +287,12 @@
>  Mark Favas
>  Niels Ferguson
>  Sebastian Fernandez
> +Carl Feynman
>  Vincent Fiack
>  Tomer Filiba
>  Jeffrey Finkelstein
>  Russell Finn
> +Dan Finnie
>  Nils Fischbeck
>  Frederik Fix
>  Matt Fleming
> @@ -276,6 +302,7 @@
>  Amaury Forgeot d'Arc
>  Doug Fort
>  John Fouhy
> +Stefan Franke
>  Martin Franklin
>  Robin Friedrich
>  Bradley Froehle
> @@ -293,6 +320,7 @@
>  Lele Gaifax
>  Santiago Gala
>  Yitzchak Gale
> +Matthew Gallagher
>  Quentin Gallet-Gilles
>  Raymund Galvin
>  Nitin Ganatra
> @@ -308,6 +336,7 @@
>  Gabriel Genellina
>  Christos Georgiou
>  Ben Gertzfield
> +Nadim Ghaznavi
>  Dinu Gherman
>  Jonathan Giddy
>  Johannes Gijsbers
> @@ -315,14 +344,18 @@
>  Christoph Gohlke
>  Tim Golden
>  Chris Gonnerman
> +Shelley Gooch
>  David Goodger
>  Hans de Graaff
> +Nathaniel Gray
>  Eddy De Greef
> +Grant Griffin
>  Duncan Grisby
>  Fabian Groffen
>  John S. Gruber
>  Dag Gruneau
>  Filip Gruszczy?ski
> +Thomas Guettler
>  Michael Guravage
>  Lars Gust?bel
>  Thomas G?ttler
> @@ -335,7 +368,9 @@
>  Bob Halley
>  Jesse Hallio
>  Jun Hamano
> +Anders Hammarquist
>  Mark Hammond
> +Harald Hanche-Olsen
>  Manus Hand
>  Milton L. Hankins
>  Stephen Hansen
> @@ -343,9 +378,13 @@
>  Lynda Hardman
>  Derek Harland
>  Jason Harper
> +Travis B. Hartwell
>  Larry Hastings
> +Tim Hatch
>  Shane Hathaway
> +Janko Hauser
>  Rycharde Hawkes
> +Ben Hayden
>  Jochen Hayek
>  Christian Heimes
>  Thomas Heller
> @@ -371,10 +410,13 @@
>  Joerg-Cyril Hoehle
>  Gregor Hoffleit
>  Chris Hoffman
> +Stefan Hoffmeister
>  Albert Hofkamp
>  Tomas Hoger
>  Jonathan Hogg
> +Steve Holden
>  Akintayo Holder
> +Thomas Holenstein
>  Gerrit Holl
>  Shane Holloway
>  Rune Holm
> @@ -389,6 +431,7 @@
>  Jan Hosang
>  Ken Howard
>  Brad Howes
> +Mike Hoy
>  Chih-Hao Huang
>  Lawrence Hudson
>  Michael Hudson
> @@ -407,6 +450,7 @@
>  Tony Ingraldi
>  John Interrante
>  Bob Ippolito
> +Roger Irwin
>  Atsuo Ishimoto
>  Paul Jackson
>  Ben Jackson
> @@ -419,12 +463,15 @@
>  Thomas Jarosch
>  Drew Jenkins
>  Flemming Kj?r Jensen
> +Philip H. Jensen
>  Philip Jenvey
>  Chris Jerdonek
>  Jiba
> +Pedro Diaz Jimenez
>  Orjan Johansen
>  Fredrik Johansson
>  Gregory K. Johnson
> +Kent Johnson
>  Simon Johnston
>  Nicolas Joly
>  Evan Jones
> @@ -450,17 +497,20 @@
>  Ryan Kelly
>  Robert Kern
>  Randall Kern
> +Jim Kerr
>  Magnus Kessler
>  Lawrence Kesteloot
>  Rafe Kettler
>  Vivek Khera
> -Akira Kitada
>  Mads Kiilerich
> +Jan Kim
>  Taek Joo Kim
>  W. Trevor King
>  Paul Kippes
>  Steve Kirsch
>  Sebastian Kirsche
> +Kamil Kisiel
> +Akira Kitada
>  Ron Klatchko
>  Bastian Kleineidam
>  Bob Kline
> @@ -474,7 +524,10 @@
>  Greg Kochanski
>  Damon Kohler
>  Marko Kohtala
> +Guido Kollerie
> +Peter A. Koren
>  Joseph Koshy
> +Daniel Kozan
>  Jerzy Kozera
>  Maksim Kozyarchuk
>  Stefan Krah
> @@ -488,10 +541,13 @@
>  Ivan Krsti?
>  Andrew Kuchling
>  Ralf W. Grosse-Kunstleve
> +Dave Kuhlman
>  Vladimir Kushnir
> +Erno Kuusela
>  Kirill Kuzminykh (?????? ?????????)
>  Ross Lagerwall
>  Cameron Laird
> +Thomas Lamb
>  Torsten Landschoff
>  ?ukasz Langa
>  Tino Lange
> @@ -508,6 +564,7 @@
>  Thomas Lee
>  Christopher Lee
>  Luc Lefebvre
> +Glyph Lefkowitz
>  Vincent Legoll
>  Kip Lehman
>  Joerg Lehmann
> @@ -515,7 +572,7 @@
>  Petri Lehtinen
>  Luke Kenneth Casson Leighton
>  Tshepang Lekhonkhobe
> -Marc-Andre Lemburg
> +Marc-Andr? Lemburg
>  John Lenton
>  Christopher Tur Lesniewski-Laas
>  Mark Levinson
> @@ -526,10 +583,13 @@
>  Shawn Ligocki
>  Martin Ligr
>  Christopher Lindblad
> +Ulf A. Lindgren
>  Bj?rn Lindqvist
>  Per Lindqvist
>  Eric Lindvall
>  Gregor Lingl
> +Everett Lipman
> +Mirko Liss
>  Nick Lockwood
>  Stephanie Lockwood
>  Anne Lord
> @@ -545,6 +605,8 @@
>  Jim Lynch
>  Mikael Lyngvig
>  Martin von L?wis
> +Jeff MacDonald
> +John Machin
>  Andrew I MacIntyre
>  Tim MacKenzie
>  Nick Maclaren
> @@ -553,17 +615,22 @@
>  David Malcolm
>  Ken Manheimer
>  Vladimir Marangozov
> +Vincent Marchetti
>  David Marek
>  Doug Marien
>  Sven Marnach
>  Alex Martelli
>  Anthony Martin
> +Westley Mart?nez
>  S?bastien Martini
>  Roger Masse
>  Nick Mathewson
> +Laura Matson
>  Graham Matthews
>  Dieter Maurer
> +Daniel May
>  Arnaud Mazin
> +Rebecca McCreary
>  Kirk McDonald
>  Chris McDonough
>  Greg McFarlane
> @@ -578,6 +645,7 @@
>  Bill van Melle
>  Lucas Prado Melo
>  Ezio Melotti
> +Doug Mennella
>  Brian Merrell
>  Luke Mewburn
>  Carl Meyer
> @@ -587,6 +655,7 @@
>  Tom Middleton
>  Stan Mihai
>  Aristotelis Mikropoulos
> +Paolo Milani
>  Damien Miller
>  Chad Miller
>  Jason V. Miller
> @@ -600,6 +669,7 @@
>  The Dragon De Monsyne
>  Skip Montanaro
>  Paul Moore
> +Ross Moore
>  Derek Morr
>  James A Morrison
>  Alessandro Moura
> @@ -611,6 +681,7 @@
>  Neil Muller
>  R. David Murray
>  Piotr Meyer
> +Dale Nagata
>  John Nagle
>  Takahiro Nakayama
>  Travers Naran
> @@ -639,11 +710,14 @@
>  Nigel O'Brian
>  Kevin O'Connor
>  Tim O'Malley
> +Zooko O'Whielacronx
>  Pascal Oberndoerfer
>  Jeffrey Ollie
>  Adam Olsen
>  Grant Olson
> +Koray Oner
>  Piet van Oostrum
> +Tomas Oppelstrup
>  Jason Orendorff
>  Douglas Orr
>  Michele Orr?
> @@ -652,6 +726,8 @@
>  Michael Otteneder
>  R. M. Oudkerk
>  Russel Owen
> +Joonas Paalasmaa
> +Shriphani Palakodety
>  Ondrej Palkovsky
>  Mike Pall
>  Todd R. Palmer
> @@ -661,11 +737,13 @@
>  Peter Parente
>  Alexandre Parenteau
>  Dan Parisien
> +William Park
>  Harri Pasanen
>  Randy Pausch
>  Samuele Pedroni
>  Marcel van der Peijl
>  Steven Pemberton
> +Bo Peng
>  Santiago Peres?n
>  Mark Perrego
>  Trevor Perrin
> @@ -674,6 +752,7 @@
>  Benjamin Peterson
>  Chris Petrilli
>  Bjorn Pettersen
> +Justin D. Pettit
>  Geoff Philbrick
>  Gavrie Philipson
>  Adrian Phillips
> @@ -705,12 +784,12 @@
>  Brodie Rao
>  Antti Rasinen
>  Sridhar Ratnakumar
> -Eric Raymond
> +Eric S. Raymond
>  Edward K. Ream
>  Chris Rebert
>  Marc Recht
>  John Redford
> -Terry Reedy
> +Terry J. Reedy
>  Gareth Rees
>  Steve Reeves
>  Lennart Regebro
> @@ -727,6 +806,7 @@
>  Nicholas Riley
>  Jean-Claude Rimbault
>  Vlad Riscutia
> +Wes Rishel
>  Juan M. Bello Rivas
>  Davide Rizzo
>  Anthony Roach
> @@ -742,6 +822,7 @@
>  Case Roole
>  Timothy Roscoe
>  Jim Roskind
> +Guido van Rossum
>  Just van Rossum
>  Hugo van Rossum
>  Saskia van Rossum
> @@ -757,6 +838,8 @@
>  Sam Rushing
>  Mark Russell
>  Nick Russo
> +Chris Ryland
> +Constantina S.
>  S?bastien Sabl?
>  Suman Saha
>  Hajime Saitou
> @@ -766,6 +849,8 @@
>  Ilya Sandler
>  Mark Sapiro
>  Ty Sarna
> +Hugh Sasse
> +Bob Savage
>  Ben Sayer
>  sbt
>  Marco Scataglini
> @@ -778,6 +863,7 @@
>  Michael Schneider
>  Peter Schneider-Kamp
>  Arvin Schnell
> +Scott Schram
>  Chad J. Schroeder
>  Sam Schulenburg
>  Stefan Schwarzer
> @@ -789,6 +875,7 @@
>  ?iga Seilnach
>  Fred Sells
>  Jiwon Seo
> +Joakim Sernbrant
>  Roger Serwy
>  Jerry Seutter
>  Denis Severson
> @@ -796,6 +883,8 @@
>  Ha Shao
>  Mark Shannon
>  Richard Shapiro
> +Justin Sheehy
> +Charlie Shepherd
>  Bruce Sherwood
>  Alexander Shigin
>  Pete Shinners
> @@ -803,31 +892,40 @@
>  John W. Shipman
>  Joel Shprentz
>  Itamar Shtull-Trauring
> +Yue Shuaijie
>  Eric Siegerman
>  Paul Sijben
>  Tim Silk
> +Michael Simcich
> +Ionel Simionescu
>  Kirill Simonov
>  Nathan Paul Simons
>  Janne Sinkkonen
> +Ng Pheng Siong
>  George Sipe
>  J. Sipprell
>  Kragen Sitaker
> +Michael Sloan
>  Eric V. Smith
>  Christopher Smith
>  Gregory P. Smith
> +Roy Smith
>  Rafal Smotrzyk
>  Dirk Soede
>  Paul Sokolovsky
>  Cody Somerville
>  Clay Spence
>  Stefan Sperling
> +Nicholas Spies
>  Per Spilling
>  Joshua Spoerri
>  Noah Spurrier
>  Nathan Srebro
>  RajGopal Srinivasan
> +Tage Stabell-Kulo
>  Quentin Stafford-Fraser
>  Frank Stajano
> +Anthony Starks
>  Oliver Steele
>  Greg Stein
>  Chris Stern
> @@ -844,6 +942,7 @@
>  Andreas St?hrk
>  Nathan Sullivan
>  Mark Summerfield
> +Reuben Sumner
>  Hisao Suzuki
>  Andrew Svetlov
>  Kalle Svensson
> @@ -871,6 +970,7 @@
>  Oren Tirosh
>  Jason Tishler
>  Christian Tismer
> +Jim Tittsler
>  Frank J. Tobin
>  R Lindsay Todd
>  Bennett Todd
> @@ -881,6 +981,7 @@
>  John Tromp
>  Jason Trowbridge
>  Anthony Tuininga
> +David Turner
>  Stephen Turner
>  Theodore Turocy
>  Bill Tutt
> @@ -890,6 +991,7 @@
>  Roger Upole
>  Michael Urman
>  Hector Urtubia
> +Ville Vainio
>  Andi Vajda
>  Case Van Horsen
>  Kyle VanderBeek
> @@ -907,6 +1009,7 @@
>  Norman Vine
>  Frank Visser
>  Johannes Vogel
> +Martijn Vries
>  Niki W. Waibel
>  Wojtek Walczak
>  Charles Waldman
> @@ -921,8 +1024,10 @@
>  Aaron Watters
>  Henrik Weber
>  Corran Webster
> +Glyn Webster
>  Stefan Wehr
>  Zack Weinberg
> +Bob Weiner
>  Edward Welbourne
>  Cliff Wells
>  Rickard Westman
> @@ -935,6 +1040,7 @@
>  Gerry Wiener
>  Frank Wierzbicki
>  Bryce "Zooko" Wilcox-O'Hearn
> +Timothy Wild
>  Jason Williams
>  John Williams
>  Sue Williams
> @@ -943,6 +1049,7 @@
>  Frank Willison
>  Greg V. Wilson
>  J Derek Wilson
> +Paul Winkler
>  Jody Winston
>  Collin Winter
>  Dik Winter
> @@ -956,6 +1063,8 @@
>  Klaus-Juergen Wolf
>  Dan Wolfe
>  Richard Wolff
> +Adam Woodbeck
> +Steven Work
>  Darren Worrall
>  Gordon Worley
>  Thomas Wouters
> @@ -966,6 +1075,7 @@
>  Ka-Ping Yee
>  Bob Yodlowski
>  Danny Yoo
> +Rory Yorke
>  George Yoshida
>  Masazumi Yoshikawa
>  Arnaud Ysmal
> @@ -976,5 +1086,6 @@
>  Mike Zarnstorff
>  Siebren van der Zee
>  Uwe Zessin
> +Cheng Zhang
>  Tarek Ziad?
>  Peter ?strand
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>

From ezio.melotti at gmail.com  Fri Sep 14 01:50:47 2012
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Fri, 14 Sep 2012 02:50:47 +0300
Subject: [Python-Dev] [Python-checkins] cpython (2.7): #15437,
 #15439: merge Doc/ACKS.txt with Misc/ACKS and modify Doc/about.rst
In-Reply-To: <CAOTb1wdkqmvHdYx_LbX=qz5tCKQKYg+WJrOKydtCELkE14-HfA@mail.gmail.com>
References: <3XHwLr547zzQjv@mail.python.org>
	<CAOTb1wdkqmvHdYx_LbX=qz5tCKQKYg+WJrOKydtCELkE14-HfA@mail.gmail.com>
Message-ID: <CACBhJdGDAO97Ub=8X9DX8mWDkNrtJby4aV0Zm_M77fMne+cOLg@mail.gmail.com>

On Fri, Sep 14, 2012 at 2:22 AM, Chris Jerdonek
<chris.jerdonek at gmail.com> wrote:
> On Thu, Sep 13, 2012 at 4:00 PM, ezio.melotti
> <python-checkins at python.org> wrote:
>> http://hg.python.org/cpython/rev/76dd082d332e
>> changeset:   79022:76dd082d332e
>> branch:      2.7
>> parent:      79014:8f847f66a49f
>> user:        Ezio Melotti <ezio.melotti at gmail.com>
>> date:        Fri Sep 14 01:58:33 2012 +0300
>> summary:
>>   #15437, #15439: merge Doc/ACKS.txt with Misc/ACKS and modify Doc/about.rst accordingly.
>
> I also contributed to this. :)
>

Yes, with all these ACKS and names I forgot to mention yours in the
commit message :)
Thanks again for writing the script that automatically merged all the names!

Best Regards,
Ezio Melotti

> --Chris
>

From peck at us.ibm.com  Fri Sep 14 12:03:34 2012
From: peck at us.ibm.com (Jon K Peck)
Date: Fri, 14 Sep 2012 04:03:34 -0600
Subject: [Python-Dev] AUTO: Jon K Peck is out of the office
Message-ID: <OF6C9BFC84.D934A7B0-ON87257A79.00374231-87257A79.00374231@us.ibm.com>



I am out of the office until 09/17/2012.

I will be out of the office from Sept 14 through Sept 16.  I will not have
email access during this time.


Note: This is an automated response to your message  "Python-Dev Digest,
Vol 110, Issue 24" sent on 09/14/2012 4:00:03.

This is the only notification you will receive while this person is away.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120914/a4a15699/attachment.html>

From vinay_sajip at yahoo.co.uk  Fri Sep 14 17:12:48 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Fri, 14 Sep 2012 15:12:48 +0000 (UTC)
Subject: [Python-Dev] packaging location ?
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
	<CADiSq7cDMKmiSW+nE1oLhqL3zgGa1zs2Sh14xmK9JbedQgY=bw@mail.gmail.com>
Message-ID: <loom.20120914T165543-69@post.gmane.org>

Nick Coghlan <ncoghlan <at> gmail.com> writes:

> 
> I like "distcore" or "distlib", though.
> 

I have set up a BitBucket repo called distlib, at

https://bitbucket.org/vinay.sajip/distlib/

This has the following bits of distutils2 / packaging, updated to run on 2.x and
3.x with a single codebase, and including tests (though not docs, yet):

version.py - version specifiers as per PEP 386
metadata.py - metadata as per PEPs 345, 314 and 241
markers.py - environment markers as per PEP 345
database.py - installed distributions as per PEP 376
depgraph.py - distribution dependency graph logic
glob.py - globbing functionality

The code was taken at around the time packaging was removed, and may not have
more recent changes.

Regards,

Vinay Sajip


From status at bugs.python.org  Fri Sep 14 18:07:04 2012
From: status at bugs.python.org (Python tracker)
Date: Fri, 14 Sep 2012 18:07:04 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20120914160704.2F35A1CE3D@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2012-09-07 - 2012-09-14)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3701 (+20)
  closed 24020 (+45)
  total  27721 (+65)

Open issues with patches: 1637 


Issues opened (46)
==================

#15880: os.path.split() and long UNC names
http://bugs.python.org/issue15880  opened by kaba2

#15881: multiprocessing 'NoneType' object is not callable
http://bugs.python.org/issue15881  opened by mcdonc

#15882: _decimal.Decimal constructed from tuple
http://bugs.python.org/issue15882  opened by hac.man

#15883: Add Py_errno to work around multiple CRT issue
http://bugs.python.org/issue15883  opened by christian.heimes

#15884: PEP 3121, 384 Refactoring applied to ctypes module
http://bugs.python.org/issue15884  opened by belopolsky

#15887: urlencode should accept generators or two elements tuples
http://bugs.python.org/issue15887  opened by nagisa

#15888: ipaddress doctest examples have some errors
http://bugs.python.org/issue15888  opened by cjerdonek

#15889: regrtest --start option raises AttributeError in many scenario
http://bugs.python.org/issue15889  opened by cjerdonek

#15890: lib2to3 pickles created with wrong permissions
http://bugs.python.org/issue15890  opened by tpievila

#15892: _PyImport_GetDynLoadFunc() doesn't check return value of fstat
http://bugs.python.org/issue15892  opened by christian.heimes

#15893: Py_FrozenMain() resource leak and missing malloc checks
http://bugs.python.org/issue15893  opened by christian.heimes

#15894: _PyImport_ReInitLock() doesn't check return value of PyThread_
http://bugs.python.org/issue15894  opened by christian.heimes

#15895: PyRun_SimpleFileExFlags() can leak a FILE pointer
http://bugs.python.org/issue15895  opened by christian.heimes

#15896: Sporadic EINVAL in nonblocking pipe os.read when forked child 
http://bugs.python.org/issue15896  opened by vitaly

#15897: zipimport.c doesn't check return value of fseek()
http://bugs.python.org/issue15897  opened by christian.heimes

#15898: OSX TTY bug
http://bugs.python.org/issue15898  opened by amoffat

#15900: Memory leak in PyUnicode_TranslateCharmap()
http://bugs.python.org/issue15900  opened by christian.heimes

#15902: imp.load_module won't accept None for the file argument for a 
http://bugs.python.org/issue15902  opened by pmoore

#15903: Make rawiobase_read() read directly to bytes object
http://bugs.python.org/issue15903  opened by sbt

#15904: file,close() can fail assert on Windows in 2.7
http://bugs.python.org/issue15904  opened by sbt

#15905: Copy to fixed size buffer w/o check in sys_update_path
http://bugs.python.org/issue15905  opened by christian.heimes

#15907: move doctest test-data files into a subdirectory of Lib/test
http://bugs.python.org/issue15907  opened by cjerdonek

#15911: can't step through _frozen_importlib/importlib._bootstrap usin
http://bugs.python.org/issue15911  opened by exarkun

#15913: PyBuffer_SizeFromFormat is missing
http://bugs.python.org/issue15913  opened by ellery.newcomer

#15914: multiprocessing.SyncManager connection hang
http://bugs.python.org/issue15914  opened by palmer

#15916: change doctest DocTestSuite not to raise ValueError if no docs
http://bugs.python.org/issue15916  opened by cjerdonek

#15917: hg hook to detect unmerged changesets
http://bugs.python.org/issue15917  opened by ezio.melotti

#15918: subprocess.Popen reads errpipe_read incorrectly, can result in
http://bugs.python.org/issue15918  opened by vitaly

#15919: hg.python.org: log page entries don't always link to revision
http://bugs.python.org/issue15919  opened by cjerdonek

#15920: make howto/regex.rst doctests pass
http://bugs.python.org/issue15920  opened by cjerdonek

#15922: make howto/urllib2.rst doctests pass
http://bugs.python.org/issue15922  opened by cjerdonek

#15923: Building from a fresh clone breaks on Parser/asdl_c.py
http://bugs.python.org/issue15923  opened by barry

#15925: email.utils.parsedate(), email.utils.parsedate_tz() and email.
http://bugs.python.org/issue15925  opened by Arfrever

#15926: Segmentation fault after multiple reinitializations
http://bugs.python.org/issue15926  opened by Arfrever

#15927: csv.reader() does not support escaped newline when quoting=csv
http://bugs.python.org/issue15927  opened by kalaxy

#15931: inspect.findsource fails after directory change
http://bugs.python.org/issue15931  opened by kgabor79

#15932: files in the csv documentation's examples are not closed
http://bugs.python.org/issue15932  opened by berdario

#15933: flaky test in test_datetime
http://bugs.python.org/issue15933  opened by pitrou

#15934: flaky test in test_ftplib
http://bugs.python.org/issue15934  opened by pitrou

#15935: clarify argparse docs re: add_argument() type and default argu
http://bugs.python.org/issue15935  opened by cjerdonek

#15936: Add link from os.urandom to random.SystemRandom
http://bugs.python.org/issue15936  opened by Jacek.Bzdak

#15938: expose way to count examples in doctest.DocFileSuite()
http://bugs.python.org/issue15938  opened by cjerdonek

#15939: make *.rst files in Doc/ parseable by doctest
http://bugs.python.org/issue15939  opened by cjerdonek

#15940: Time module: effect of locale
http://bugs.python.org/issue15940  opened by terry.reedy

#15941: Time module: effect of time.timezone change
http://bugs.python.org/issue15941  opened by terry.reedy

#15944: memoryviews and ctypes
http://bugs.python.org/issue15944  opened by dabeaz



Most recent 15 issues with no replies (15)
==========================================

#15936: Add link from os.urandom to random.SystemRandom
http://bugs.python.org/issue15936

#15931: inspect.findsource fails after directory change
http://bugs.python.org/issue15931

#15927: csv.reader() does not support escaped newline when quoting=csv
http://bugs.python.org/issue15927

#15925: email.utils.parsedate(), email.utils.parsedate_tz() and email.
http://bugs.python.org/issue15925

#15922: make howto/urllib2.rst doctests pass
http://bugs.python.org/issue15922

#15920: make howto/regex.rst doctests pass
http://bugs.python.org/issue15920

#15919: hg.python.org: log page entries don't always link to revision
http://bugs.python.org/issue15919

#15916: change doctest DocTestSuite not to raise ValueError if no docs
http://bugs.python.org/issue15916

#15907: move doctest test-data files into a subdirectory of Lib/test
http://bugs.python.org/issue15907

#15897: zipimport.c doesn't check return value of fseek()
http://bugs.python.org/issue15897

#15894: _PyImport_ReInitLock() doesn't check return value of PyThread_
http://bugs.python.org/issue15894

#15893: Py_FrozenMain() resource leak and missing malloc checks
http://bugs.python.org/issue15893

#15892: _PyImport_GetDynLoadFunc() doesn't check return value of fstat
http://bugs.python.org/issue15892

#15884: PEP 3121, 384 Refactoring applied to ctypes module
http://bugs.python.org/issue15884

#15869: Include .desktop file and icon
http://bugs.python.org/issue15869



Most recent 15 issues waiting for review (15)
=============================================

#15939: make *.rst files in Doc/ parseable by doctest
http://bugs.python.org/issue15939

#15935: clarify argparse docs re: add_argument() type and default argu
http://bugs.python.org/issue15935

#15932: files in the csv documentation's examples are not closed
http://bugs.python.org/issue15932

#15922: make howto/urllib2.rst doctests pass
http://bugs.python.org/issue15922

#15920: make howto/regex.rst doctests pass
http://bugs.python.org/issue15920

#15917: hg hook to detect unmerged changesets
http://bugs.python.org/issue15917

#15903: Make rawiobase_read() read directly to bytes object
http://bugs.python.org/issue15903

#15900: Memory leak in PyUnicode_TranslateCharmap()
http://bugs.python.org/issue15900

#15895: PyRun_SimpleFileExFlags() can leak a FILE pointer
http://bugs.python.org/issue15895

#15892: _PyImport_GetDynLoadFunc() doesn't check return value of fstat
http://bugs.python.org/issue15892

#15889: regrtest --start option raises AttributeError in many scenario
http://bugs.python.org/issue15889

#15888: ipaddress doctest examples have some errors
http://bugs.python.org/issue15888

#15883: Add Py_errno to work around multiple CRT issue
http://bugs.python.org/issue15883

#15882: _decimal.Decimal constructed from tuple
http://bugs.python.org/issue15882

#15881: multiprocessing 'NoneType' object is not callable
http://bugs.python.org/issue15881



Top 10 most discussed issues (10)
=================================

#15896: Sporadic EINVAL in nonblocking pipe os.read when forked child 
http://bugs.python.org/issue15896  16 msgs

#15881: multiprocessing 'NoneType' object is not callable
http://bugs.python.org/issue15881  13 msgs

#15882: _decimal.Decimal constructed from tuple
http://bugs.python.org/issue15882  12 msgs

#15340: OSError with "import random" when /dev/urandom doesn't exist (
http://bugs.python.org/issue15340  11 msgs

#15898: OSX TTY bug
http://bugs.python.org/issue15898   9 msgs

#15629: Add to regrtest the ability to run Lib and Doc doctests
http://bugs.python.org/issue15629   8 msgs

#15873: "datetime" cannot parse ISO 8601 dates and times
http://bugs.python.org/issue15873   8 msgs

#15918: subprocess.Popen reads errpipe_read incorrectly, can result in
http://bugs.python.org/issue15918   8 msgs

#15851: Lib/robotparser.py doesn't accept setting a user agent string,
http://bugs.python.org/issue15851   7 msgs

#15895: PyRun_SimpleFileExFlags() can leak a FILE pointer
http://bugs.python.org/issue15895   7 msgs



Issues closed (43)
==================

#5766: Mac/scripts/BuildApplet.py reset of sys.executable during inst
http://bugs.python.org/issue5766  closed by ned.deily

#10167: ESET Trojan Alert [python-3.1.2.amd64 ON Win7-64]
http://bugs.python.org/issue10167  closed by christian.heimes

#13990: Benchmarks: 2to3 failures on the py3 side
http://bugs.python.org/issue13990  closed by ezio.melotti

#13992: Segfault in PyTrash_destroy_chain
http://bugs.python.org/issue13992  closed by georg.brandl

#14617: confusing docs with regard to __hash__
http://bugs.python.org/issue14617  closed by r.david.murray

#14649: doctest.DocTestSuite error misleading when module has no docst
http://bugs.python.org/issue14649  closed by r.david.murray

#15437: Merge Doc/ACKS.txt names into Misc/ACKS
http://bugs.python.org/issue15437  closed by ezio.melotti

#15439: Include Misc/ACKS names into the documentation
http://bugs.python.org/issue15439  closed by ezio.melotti

#15510: textwrap.wrap('') returns empty list
http://bugs.python.org/issue15510  closed by r.david.murray

#15676: mmap: add empty file check prior to offset check
http://bugs.python.org/issue15676  closed by python-dev

#15784: OSError.__str__() should distinguish between errno and winerro
http://bugs.python.org/issue15784  closed by georg.brandl

#15785: curses.get_wch() returns keypad codes incorrectly
http://bugs.python.org/issue15785  closed by georg.brandl

#15793: Stack corruption in ssl.RAND_egd()
http://bugs.python.org/issue15793  closed by python-dev

#15820: Add additional info to Resources area on Dev Guide
http://bugs.python.org/issue15820  closed by r.david.murray

#15822: installed python may fail incorrectly trying to rebuild lib2to
http://bugs.python.org/issue15822  closed by ned.deily

#15828: imp.load_module doesn't support C_EXTENSION type
http://bugs.python.org/issue15828  closed by georg.brandl

#15831: comma after leading optional argument is after bracket in docs
http://bugs.python.org/issue15831  closed by ezio.melotti

#15838: make install tries to create lib2to3 grammar pickles in source
http://bugs.python.org/issue15838  closed by ned.deily

#15842: Some SocketIO methods can succeed after close()
http://bugs.python.org/issue15842  closed by pitrou

#15847: parse_args stopped accepting tuples
http://bugs.python.org/issue15847  closed by r.david.murray

#15865: reflect bare star * in function signature documentation
http://bugs.python.org/issue15865  closed by ezio.melotti

#15868: leak in bytesio.c
http://bugs.python.org/issue15868  closed by skrah

#15874: argparse cannot parse shell variable arguments in file-given a
http://bugs.python.org/issue15874  closed by ZhuangZi

#15879: set.__or__, __and__, etc create subclass types, but ignore __n
http://bugs.python.org/issue15879  closed by amaury.forgeotdarc

#15885: @staticmethod __getattr__ doesn't work
http://bugs.python.org/issue15885  closed by eric.snow

#15886: os.stat() docs repeat "specifying a file descriptor" support
http://bugs.python.org/issue15886  closed by r.david.murray

#15891: A public facing API for __unittest = True
http://bugs.python.org/issue15891  closed by r.david.murray

#15899: howto/unicode.rst doctest corrections
http://bugs.python.org/issue15899  closed by orsenthil

#15901: multiprocessing sharedctypes Array don't accept strings
http://bugs.python.org/issue15901  closed by sbt

#15906: argparse add_argument() confusing behavior when type=str, defa
http://bugs.python.org/issue15906  closed by barry

#15908: SHA1 crashes in 64 bits when the string to hash is bigger than
http://bugs.python.org/issue15908  closed by python-dev

#15909: test_mmap failure on Windows buildbots
http://bugs.python.org/issue15909  closed by skrah

#15910: MD5 and SHA1 crash when "updated" with strings bigger than 2**
http://bugs.python.org/issue15910  closed by python-dev

#15912: Intermittent import failure
http://bugs.python.org/issue15912  closed by eric.snow

#15915: array.array does not support the buffer interface
http://bugs.python.org/issue15915  closed by benjamin.peterson

#15921: select module uses uninitialized value "tv.tv_usec"
http://bugs.python.org/issue15921  closed by python-dev

#15924: 404 link on Python about page
http://bugs.python.org/issue15924  closed by cjerdonek

#15928: Open url with proxy causes TypeError
http://bugs.python.org/issue15928  closed by orsenthil

#15929: argparse non alphanum characters replacement
http://bugs.python.org/issue15929  closed by bethard

#15930: buffer overrun in wcstombs_errorpos()
http://bugs.python.org/issue15930  closed by christian.heimes

#15937: Assertion failure (C:\Delphi\progs32\Python\PythonIDE\cPyRemot
http://bugs.python.org/issue15937  closed by r.david.murray

#15942: Time representation for Windows incorrect when BIOS clock set 
http://bugs.python.org/issue15942  closed by r.david.murray

#15943: urllib2 always sends header connection: close
http://bugs.python.org/issue15943  closed by r.david.murray

From dholth at gmail.com  Fri Sep 14 18:30:20 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 14 Sep 2012 12:30:20 -0400
Subject: [Python-Dev] Edits to Metadata 1.2 to add extras (optional
	dependencies)
In-Reply-To: <CAOTD34akAcFyL3VoMomziHtQEzp9jcSwdndi6kkP00GHoykcew@mail.gmail.com>
References: <CAG8k2+5WyDyChg+FX6_2w0_1N5uX6DCMtM+HnTQzjQxjCTTHSQ@mail.gmail.com>
	<CAG8k2+5BE=a6y1yJ8Beh=CmroZSJ3Jx8W+BzmkZHqoB6dU5_bw@mail.gmail.com>
	<CAOTD34akAcFyL3VoMomziHtQEzp9jcSwdndi6kkP00GHoykcew@mail.gmail.com>
Message-ID: <CAG8k2+4+k9dQhXf7pGZ=wkdHLadvHK40aCq6j-=TCPcyMy2nWw@mail.gmail.com>

Add to metadata 1.3:

Description-File: README(\..+)?

Meaning the description should be read from a file in the same
directory as PKG-INFO or METADATA (including in the .dist-info
directories) and we strongly recommend it be named as README.* and be
utf-8 encoded text.

Description: is the only multi-line field in the metadata. It is
almost never needed at runtime. It would be great for performance and
simplify the parser to just put it in another file.

Mutually exclusive with Description.

May beg for a Summary: tag with a one-line description.

From donald.stufft at gmail.com  Fri Sep 14 19:42:22 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Fri, 14 Sep 2012 13:42:22 -0400
Subject: [Python-Dev] Edits to Metadata 1.2 to add extras (optional
 dependencies)
In-Reply-To: <CAG8k2+4+k9dQhXf7pGZ=wkdHLadvHK40aCq6j-=TCPcyMy2nWw@mail.gmail.com>
References: <CAG8k2+5WyDyChg+FX6_2w0_1N5uX6DCMtM+HnTQzjQxjCTTHSQ@mail.gmail.com>
	<CAG8k2+5BE=a6y1yJ8Beh=CmroZSJ3Jx8W+BzmkZHqoB6dU5_bw@mail.gmail.com>
	<CAOTD34akAcFyL3VoMomziHtQEzp9jcSwdndi6kkP00GHoykcew@mail.gmail.com>
	<CAG8k2+4+k9dQhXf7pGZ=wkdHLadvHK40aCq6j-=TCPcyMy2nWw@mail.gmail.com>
Message-ID: <7DB2CF6C44D2471DB7D5DFBB74484042@gmail.com>

On Friday, September 14, 2012 at 12:30 PM, Daniel Holth wrote:
> Add to metadata 1.3:
> 
> Description-File: README(\..+)?
> 
> Meaning the description should be read from a file in the same
> directory as PKG-INFO or METADATA (including in the .dist-info
> directories) and we strongly recommend it be named as README.* and be
> utf-8 encoded text.
> 
> Description: is the only multi-line field in the metadata. It is
> almost never needed at runtime. It would be great for performance and
> simplify the parser to just put it in another file.
> 
> Mutually exclusive with Description.
> 
> May beg for a Summary: tag with a one-line description.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org (mailto:Python-Dev at python.org)
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/donald.stufft%40gmail.com
> 
> 

-1 on this personally 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120914/c57f6db9/attachment.html>

From erik.m.bray at gmail.com  Fri Sep 14 19:43:10 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Fri, 14 Sep 2012 13:43:10 -0400
Subject: [Python-Dev] Edits to Metadata 1.2 to add extras (optional
	dependencies)
In-Reply-To: <CAG8k2+4+k9dQhXf7pGZ=wkdHLadvHK40aCq6j-=TCPcyMy2nWw@mail.gmail.com>
References: <CAG8k2+5WyDyChg+FX6_2w0_1N5uX6DCMtM+HnTQzjQxjCTTHSQ@mail.gmail.com>
	<CAG8k2+5BE=a6y1yJ8Beh=CmroZSJ3Jx8W+BzmkZHqoB6dU5_bw@mail.gmail.com>
	<CAOTD34akAcFyL3VoMomziHtQEzp9jcSwdndi6kkP00GHoykcew@mail.gmail.com>
	<CAG8k2+4+k9dQhXf7pGZ=wkdHLadvHK40aCq6j-=TCPcyMy2nWw@mail.gmail.com>
Message-ID: <CAOTD34YTP8g11QVxkq18dQvDrExCaSkNf+aoaBL5WKB8WXrkEg@mail.gmail.com>

On Fri, Sep 14, 2012 at 12:30 PM, Daniel Holth <dholth at gmail.com> wrote:
> Add to metadata 1.3:
>
> Description-File: README(\..+)?
>
> Meaning the description should be read from a file in the same
> directory as PKG-INFO or METADATA (including in the .dist-info
> directories) and we strongly recommend it be named as README.* and be
> utf-8 encoded text.
>
> Description: is the only multi-line field in the metadata. It is
> almost never needed at runtime. It would be great for performance and
> simplify the parser to just put it in another file.
>
> Mutually exclusive with Description.
>
> May beg for a Summary: tag with a one-line description.

Can we make Description-File multiple-use?  The meaning of this would
be that the Description is formed from concatenating each
Description-File in order.  That raises the question: Is ordering
guaranteed for multiple-use fields?

I ask, because distutils2 supports exactly such a feature, and I've
found it useful.  For example, if I have a README.rst and a
CHANGELOG.rst I can specify:

description-file =
    README.rst
    CHANGELOG.rst

Then the full description, contains my readme and my changelog, which
look nice together on PyPI, but I prefer to keep as separate files in
the source.

My only other concern is that if the value of this field can
theoretically be arbitrary, it could conflict with other .dist-info
files.  Does the .dist-info format allow subdirectories?  Placing
description-files in a subdirectory of .dist-info could be a
reasonable workaround.

Erik

From dholth at gmail.com  Fri Sep 14 19:57:55 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 14 Sep 2012 13:57:55 -0400
Subject: [Python-Dev] Edits to Metadata 1.2 to add extras (optional
	dependencies)
In-Reply-To: <CAOTD34YTP8g11QVxkq18dQvDrExCaSkNf+aoaBL5WKB8WXrkEg@mail.gmail.com>
References: <CAG8k2+5WyDyChg+FX6_2w0_1N5uX6DCMtM+HnTQzjQxjCTTHSQ@mail.gmail.com>
	<CAG8k2+5BE=a6y1yJ8Beh=CmroZSJ3Jx8W+BzmkZHqoB6dU5_bw@mail.gmail.com>
	<CAOTD34akAcFyL3VoMomziHtQEzp9jcSwdndi6kkP00GHoykcew@mail.gmail.com>
	<CAG8k2+4+k9dQhXf7pGZ=wkdHLadvHK40aCq6j-=TCPcyMy2nWw@mail.gmail.com>
	<CAOTD34YTP8g11QVxkq18dQvDrExCaSkNf+aoaBL5WKB8WXrkEg@mail.gmail.com>
Message-ID: <CAG8k2+6TVDVB5zcrnGMNOS+=ioDhpcqdkFmjT6eGvqFn00qbRQ@mail.gmail.com>

On Fri, Sep 14, 2012 at 1:43 PM, Erik Bray <erik.m.bray at gmail.com> wrote:
> On Fri, Sep 14, 2012 at 12:30 PM, Daniel Holth <dholth at gmail.com> wrote:
>> Add to metadata 1.3:
>>
>> Description-File: README(\..+)?
>>
>> Meaning the description should be read from a file in the same
>> directory as PKG-INFO or METADATA (including in the .dist-info
>> directories) and we strongly recommend it be named as README.* and be
>> utf-8 encoded text.
>>
>> Description: is the only multi-line field in the metadata. It is
>> almost never needed at runtime. It would be great for performance and
>> simplify the parser to just put it in another file.
>>
>> Mutually exclusive with Description.
>>
>> May beg for a Summary: tag with a one-line description.
>
> Can we make Description-File multiple-use?  The meaning of this would
> be that the Description is formed from concatenating each
> Description-File in order.  That raises the question: Is ordering
> guaranteed for multiple-use fields?
>
> I ask, because distutils2 supports exactly such a feature, and I've
> found it useful.  For example, if I have a README.rst and a
> CHANGELOG.rst I can specify:
>
> description-file =
>     README.rst
>     CHANGELOG.rst
>
> Then the full description, contains my readme and my changelog, which
> look nice together on PyPI, but I prefer to keep as separate files in
> the source.
>
> My only other concern is that if the value of this field can
> theoretically be arbitrary, it could conflict with other .dist-info
> files.  Does the .dist-info format allow subdirectories?  Placing
> description-files in a subdirectory of .dist-info could be a
> reasonable workaround.
>
> Erik

The .dist-info design asks for every metadata file (the one in all
caps, not any of the other metadata in .dist-info) to be parsed for
many packaging operations that do not require the description, such as
resolving the dependency graph of a package. Description-File would
give an installer the option to pull Description: out into
Description-File:. I would expect the concatenation to happen before
this point.

I would like to forbid subdirectories in .dist-info but I think they
are allowed.

The order of multi-use fields is probably preserved. I don't think it
is required to be by any spec.

From donald.stufft at gmail.com  Fri Sep 14 20:03:22 2012
From: donald.stufft at gmail.com (Donald Stufft)
Date: Fri, 14 Sep 2012 14:03:22 -0400
Subject: [Python-Dev] Edits to Metadata 1.2 to add extras (optional
 dependencies)
In-Reply-To: <CAG8k2+4+k9dQhXf7pGZ=wkdHLadvHK40aCq6j-=TCPcyMy2nWw@mail.gmail.com>
References: <CAG8k2+5WyDyChg+FX6_2w0_1N5uX6DCMtM+HnTQzjQxjCTTHSQ@mail.gmail.com>
	<CAG8k2+5BE=a6y1yJ8Beh=CmroZSJ3Jx8W+BzmkZHqoB6dU5_bw@mail.gmail.com>
	<CAOTD34akAcFyL3VoMomziHtQEzp9jcSwdndi6kkP00GHoykcew@mail.gmail.com>
	<CAG8k2+4+k9dQhXf7pGZ=wkdHLadvHK40aCq6j-=TCPcyMy2nWw@mail.gmail.com>
Message-ID: <636D126DC0FA4514BBC3CE048EF45623@gmail.com>

On Friday, September 14, 2012 at 12:30 PM, Daniel Holth wrote:
> Description: is the only multi-line field in the metadata. It is
> almost never needed at runtime. It would be great for performance and
> simplify the parser to just put it in another file.

License also can be multiline I believe,  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120914/45630080/attachment-0001.html>

From dholth at gmail.com  Fri Sep 14 20:10:14 2012
From: dholth at gmail.com (Daniel Holth)
Date: Fri, 14 Sep 2012 14:10:14 -0400
Subject: [Python-Dev] Edits to Metadata 1.2 to add extras (optional
	dependencies)
In-Reply-To: <636D126DC0FA4514BBC3CE048EF45623@gmail.com>
References: <CAG8k2+5WyDyChg+FX6_2w0_1N5uX6DCMtM+HnTQzjQxjCTTHSQ@mail.gmail.com>
	<CAG8k2+5BE=a6y1yJ8Beh=CmroZSJ3Jx8W+BzmkZHqoB6dU5_bw@mail.gmail.com>
	<CAOTD34akAcFyL3VoMomziHtQEzp9jcSwdndi6kkP00GHoykcew@mail.gmail.com>
	<CAG8k2+4+k9dQhXf7pGZ=wkdHLadvHK40aCq6j-=TCPcyMy2nWw@mail.gmail.com>
	<636D126DC0FA4514BBC3CE048EF45623@gmail.com>
Message-ID: <CAG8k2+7+W6LS_q=4N-_7FcDgTpABLGneqBvsN1iBzNdTCW-oVA@mail.gmail.com>

On Fri, Sep 14, 2012 at 2:03 PM, Donald Stufft <donald.stufft at gmail.com> wrote:
> On Friday, September 14, 2012 at 12:30 PM, Daniel Holth wrote:
>
> Description: is the only multi-line field in the metadata. It is
> almost never needed at runtime. It would be great for performance and
> simplify the parser to just put it in another file.
>
> License also can be multiline I believe,

OK. In practice, Description: is the only field that is likely to be
hundreds of lines long which seems wasteful. The parser complexity is
a non-issue. (I know the PEP says Description: should not be the
instruction manual, but it is wrong because that is the way to get
useful data up on a package's pypi page.)

From p.f.moore at gmail.com  Fri Sep 14 21:37:03 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 14 Sep 2012 20:37:03 +0100
Subject: [Python-Dev] Edits to Metadata 1.2 to add extras (optional
	dependencies)
In-Reply-To: <CAG8k2+4+k9dQhXf7pGZ=wkdHLadvHK40aCq6j-=TCPcyMy2nWw@mail.gmail.com>
References: <CAG8k2+5WyDyChg+FX6_2w0_1N5uX6DCMtM+HnTQzjQxjCTTHSQ@mail.gmail.com>
	<CAG8k2+5BE=a6y1yJ8Beh=CmroZSJ3Jx8W+BzmkZHqoB6dU5_bw@mail.gmail.com>
	<CAOTD34akAcFyL3VoMomziHtQEzp9jcSwdndi6kkP00GHoykcew@mail.gmail.com>
	<CAG8k2+4+k9dQhXf7pGZ=wkdHLadvHK40aCq6j-=TCPcyMy2nWw@mail.gmail.com>
Message-ID: <CACac1F8iiLuV5Xt=1Fu-jmXZGYUjVz_jzkD2N_zBBFziqR9tug@mail.gmail.com>

On 14 September 2012 17:30, Daniel Holth <dholth at gmail.com> wrote:
> we strongly recommend it be named as README.* and be
> utf-8 encoded text.

I'd very strongly recommend that the encoding should be mandated. I'm
happy with UTF-8 (although I expect a lot of "accidental" latin-1
files, particularly from Windows users) but I think it would be a
mistake to have the specification leaving the encoding of *any* string
data unclear. Without the encoding being specified, either in the
metadata itself ("Description-File-Encoding: utf8"? Please, no) or by
the specification mandating it, how is a program expected to read the
description?

Other than this point, I have no opinion on the proposal.

Paul.

From erik.m.bray at gmail.com  Fri Sep 14 21:51:49 2012
From: erik.m.bray at gmail.com (Erik Bray)
Date: Fri, 14 Sep 2012 15:51:49 -0400
Subject: [Python-Dev] Edits to Metadata 1.2 to add extras (optional
	dependencies)
In-Reply-To: <CAG8k2+6TVDVB5zcrnGMNOS+=ioDhpcqdkFmjT6eGvqFn00qbRQ@mail.gmail.com>
References: <CAG8k2+5WyDyChg+FX6_2w0_1N5uX6DCMtM+HnTQzjQxjCTTHSQ@mail.gmail.com>
	<CAG8k2+5BE=a6y1yJ8Beh=CmroZSJ3Jx8W+BzmkZHqoB6dU5_bw@mail.gmail.com>
	<CAOTD34akAcFyL3VoMomziHtQEzp9jcSwdndi6kkP00GHoykcew@mail.gmail.com>
	<CAG8k2+4+k9dQhXf7pGZ=wkdHLadvHK40aCq6j-=TCPcyMy2nWw@mail.gmail.com>
	<CAOTD34YTP8g11QVxkq18dQvDrExCaSkNf+aoaBL5WKB8WXrkEg@mail.gmail.com>
	<CAG8k2+6TVDVB5zcrnGMNOS+=ioDhpcqdkFmjT6eGvqFn00qbRQ@mail.gmail.com>
Message-ID: <CAOTD34Y=PTsqs4p0HrtO-RULmSngGqL9BDQq_KNvi73Ruwrdkg@mail.gmail.com>

On Fri, Sep 14, 2012 at 1:57 PM, Daniel Holth <dholth at gmail.com> wrote:
> On Fri, Sep 14, 2012 at 1:43 PM, Erik Bray <erik.m.bray at gmail.com> wrote:
>> On Fri, Sep 14, 2012 at 12:30 PM, Daniel Holth <dholth at gmail.com> wrote:
>>> Add to metadata 1.3:
>>>
>>> Description-File: README(\..+)?
>>>
>>> Meaning the description should be read from a file in the same
>>> directory as PKG-INFO or METADATA (including in the .dist-info
>>> directories) and we strongly recommend it be named as README.* and be
>>> utf-8 encoded text.
>>>
>>> Description: is the only multi-line field in the metadata. It is
>>> almost never needed at runtime. It would be great for performance and
>>> simplify the parser to just put it in another file.
>>>
>>> Mutually exclusive with Description.
>>>
>>> May beg for a Summary: tag with a one-line description.
>>
>> Can we make Description-File multiple-use?  The meaning of this would
>> be that the Description is formed from concatenating each
>> Description-File in order.  That raises the question: Is ordering
>> guaranteed for multiple-use fields?
>>
>> I ask, because distutils2 supports exactly such a feature, and I've
>> found it useful.  For example, if I have a README.rst and a
>> CHANGELOG.rst I can specify:
>>
>> description-file =
>>     README.rst
>>     CHANGELOG.rst
>>
>> Then the full description, contains my readme and my changelog, which
>> look nice together on PyPI, but I prefer to keep as separate files in
>> the source.
>>
>> My only other concern is that if the value of this field can
>> theoretically be arbitrary, it could conflict with other .dist-info
>> files.  Does the .dist-info format allow subdirectories?  Placing
>> description-files in a subdirectory of .dist-info could be a
>> reasonable workaround.
>>
>> Erik
>
> The .dist-info design asks for every metadata file (the one in all
> caps, not any of the other metadata in .dist-info) to be parsed for
> many packaging operations that do not require the description, such as
> resolving the dependency graph of a package. Description-File would
> give an installer the option to pull Description: out into
> Description-File:. I would expect the concatenation to happen before
> this point.

I understand now. In this case why even allow flexibility in the
description file name?  Just make it description.txt, and the
Description-File field just some boolean indicator of whether or not a
description file exists?

Erik

From victor.stinner at gmail.com  Sat Sep 15 00:55:05 2012
From: victor.stinner at gmail.com (Victor Stinner)
Date: Sat, 15 Sep 2012 00:55:05 +0200
Subject: [Python-Dev] What's New in Python 3.3: missing unittest.mock!
In-Reply-To: <CAMpsgwa9uF2oSFUdcFQa0RF5Zx4gG5uJH69jeg17XapfU8Q2vQ@mail.gmail.com>
References: <CAMpsgwa9uF2oSFUdcFQa0RF5Zx4gG5uJH69jeg17XapfU8Q2vQ@mail.gmail.com>
Message-ID: <CAMpsgwZVf0gg9DVdNDzzLOb9YO8DqCCyMQ81vKRvp1ib+rp_rA@mail.gmail.com>

Hi,

I just noticed that the new unittest.mock module is missing from the
What's New in Python 3.3. Can someone add it? (I cannot right now).

Thanks.
Victor

From tarek at ziade.org  Sat Sep 15 09:27:40 2012
From: tarek at ziade.org (=?ISO-8859-1?Q?Tarek_Ziad=E9?=)
Date: Sat, 15 Sep 2012 09:27:40 +0200
Subject: [Python-Dev] packaging location ?
In-Reply-To: <loom.20120914T165543-69@post.gmane.org>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
	<CADiSq7cDMKmiSW+nE1oLhqL3zgGa1zs2Sh14xmK9JbedQgY=bw@mail.gmail.com>
	<loom.20120914T165543-69@post.gmane.org>
Message-ID: <50542DEC.4050702@ziade.org>

On 9/14/12 5:12 PM, Vinay Sajip wrote:
> Nick Coghlan <ncoghlan <at> gmail.com> writes:
>
>> I like "distcore" or "distlib", though.
>>
> I have set up a BitBucket repo called distlib, at
>
> https://bitbucket.org/vinay.sajip/distlib/
>
> This has the following bits of distutils2 / packaging, updated to run on 2.x and
> 3.x with a single codebase, and including tests (though not docs, yet):
>
> version.py - version specifiers as per PEP 386
> metadata.py - metadata as per PEPs 345, 314 and 241
> markers.py - environment markers as per PEP 345
> database.py - installed distributions as per PEP 376
> depgraph.py - distribution dependency graph logic
> glob.py - globbing functionality
>
> The code was taken at around the time packaging was removed, and may not have
> more recent changes.
>
> Regards,
oh, cool !

maybe we could copy it at hg.python.org ?

>
> Vinay Sajip
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com


From vinay_sajip at yahoo.co.uk  Sat Sep 15 15:43:48 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 15 Sep 2012 13:43:48 +0000 (UTC)
Subject: [Python-Dev] packaging location ?
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
	<CADiSq7cDMKmiSW+nE1oLhqL3zgGa1zs2Sh14xmK9JbedQgY=bw@mail.gmail.com>
	<loom.20120914T165543-69@post.gmane.org>
	<50542DEC.4050702@ziade.org>
Message-ID: <loom.20120915T154205-323@post.gmane.org>

Tarek Ziad? <tarek <at> ziade.org> writes:

> > Regards,
> oh, cool !
> 
> maybe we could copy it at hg.python.org ?
> 

Sure, but I don't know if I can do it. IIUC it needs someone with an account on
the server to create new repositories.

Regards,

Vinay Sajip


From ncoghlan at gmail.com  Sat Sep 15 16:14:00 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 16 Sep 2012 00:14:00 +1000
Subject: [Python-Dev] packaging location ?
In-Reply-To: <loom.20120915T154205-323@post.gmane.org>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
	<CADiSq7cDMKmiSW+nE1oLhqL3zgGa1zs2Sh14xmK9JbedQgY=bw@mail.gmail.com>
	<loom.20120914T165543-69@post.gmane.org>
	<50542DEC.4050702@ziade.org>
	<loom.20120915T154205-323@post.gmane.org>
Message-ID: <CADiSq7dYPTrzhc0PuSZP_Qq-1m2iCwQrow9A5jL7peNeasvSRw@mail.gmail.com>

On Sat, Sep 15, 2012 at 11:43 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Sure, but I don't know if I can do it. IIUC it needs someone with an account on
> the server to create new repositories.

Depends how much you care about a pristine history - you can do a
server side clone of an existing repo and then empty it out. And if
you use http://hg.python.org/buildbot/empty/ as the starting point,
there isn't even anything to empty out.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Sat Sep 15 17:27:28 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 15 Sep 2012 16:27:28 +0100 (BST)
Subject: [Python-Dev] packaging location ?
In-Reply-To: <CADiSq7dYPTrzhc0PuSZP_Qq-1m2iCwQrow9A5jL7peNeasvSRw@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
	<CADiSq7cDMKmiSW+nE1oLhqL3zgGa1zs2Sh14xmK9JbedQgY=bw@mail.gmail.com>
	<loom.20120914T165543-69@post.gmane.org>
	<50542DEC.4050702@ziade.org>
	<loom.20120915T154205-323@post.gmane.org>
	<CADiSq7dYPTrzhc0PuSZP_Qq-1m2iCwQrow9A5jL7peNeasvSRw@mail.gmail.com>
Message-ID: <1347722848.37777.YahooMailNeo@web171406.mail.ir2.yahoo.com>

> Depends how much you care about a pristine history - you can do a

> server side clone of an existing repo and then empty it out. And if
> you use http://hg.python.org/buildbot/empty/ as the starting point,
> there isn't even anything to empty out.

Actually there are some files in there - a Makefile, configure script and some .bat files.

I tried cloning it but I'm not allowed to create a repo at the top level - it says

Please use a secondary level path such as "sandbox/distlib"

Seems a shame not to make it a top-level repo.

Regards,

Vinay Sajip


From solipsis at pitrou.net  Sat Sep 15 19:05:50 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 15 Sep 2012 19:05:50 +0200
Subject: [Python-Dev] packaging location ?
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
	<CADiSq7cDMKmiSW+nE1oLhqL3zgGa1zs2Sh14xmK9JbedQgY=bw@mail.gmail.com>
	<loom.20120914T165543-69@post.gmane.org>
	<50542DEC.4050702@ziade.org>
	<loom.20120915T154205-323@post.gmane.org>
	<CADiSq7dYPTrzhc0PuSZP_Qq-1m2iCwQrow9A5jL7peNeasvSRw@mail.gmail.com>
	<1347722848.37777.YahooMailNeo@web171406.mail.ir2.yahoo.com>
Message-ID: <20120915190550.25b378c8@pitrou.net>

On Sat, 15 Sep 2012 16:27:28 +0100 (BST)
Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:

> > Depends how much you care about a pristine history - you can do a
> 
> > server side clone of an existing repo and then empty it out. And if
> > you use http://hg.python.org/buildbot/empty/ as the starting point,
> > there isn't even anything to empty out.
> 
> Actually there are some files in there - a Makefile, configure script and some .bat files.
> 
> I tried cloning it but I'm not allowed to create a repo at the top level - it says
> 
> Please use a secondary level path such as "sandbox/distlib"
> 
> Seems a shame not to make it a top-level repo.

Well, if you really need it, it can certainly be created.
On the other hand, if you are not using hg.python.org features such as
commits e-mails or buildbots, it's also fine living on bitbucket until
the project matures a bit.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From vinay_sajip at yahoo.co.uk  Sat Sep 15 22:36:12 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 15 Sep 2012 20:36:12 +0000 (UTC)
Subject: [Python-Dev] packaging location ?
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
	<CADiSq7cDMKmiSW+nE1oLhqL3zgGa1zs2Sh14xmK9JbedQgY=bw@mail.gmail.com>
	<loom.20120914T165543-69@post.gmane.org>
	<50542DEC.4050702@ziade.org>
	<loom.20120915T154205-323@post.gmane.org>
	<CADiSq7dYPTrzhc0PuSZP_Qq-1m2iCwQrow9A5jL7peNeasvSRw@mail.gmail.com>
	<1347722848.37777.YahooMailNeo@web171406.mail.ir2.yahoo.com>
	<20120915190550.25b378c8@pitrou.net>
Message-ID: <loom.20120915T223541-866@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

> On the other hand, if you are not using hg.python.org features such as
> commits e-mails or buildbots, it's also fine living on bitbucket until
> the project matures a bit.
> 

I'm fine with that.

Regards,

Vinay Sajip


From rosslagerwall at gmail.com  Sun Sep 16 06:54:21 2012
From: rosslagerwall at gmail.com (Ross Lagerwall)
Date: Sun, 16 Sep 2012 06:54:21 +0200
Subject: [Python-Dev] [Python-checkins] Daily reference leaks
	(1704deb7e6d7): sum=2
In-Reply-To: <E1TD66G-0001y0-PG@ap.vmr.nerim.net>
References: <E1TD66G-0001y0-PG@ap.vmr.nerim.net>
Message-ID: <50555B7D.9040601@gmail.com>

On 09/16/2012 06:04 AM, solipsis at pitrou.net wrote:
> results for 1704deb7e6d7 on branch "default"
> --------------------------------------------
> 
> test_dbm leaked [0, 2, 0] references, sum=2
> 

I've noticed that test_dbm fairly often leaks here although I've never
reproduced it. Does anyone know with which dbm lib this is (gdbm, ndbm,
gdbm_compat, bdb)?


Regards
-- 
Ross Lagerwall

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 291 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120916/81c9f7b4/attachment.pgp>

From solipsis at pitrou.net  Sun Sep 16 12:34:59 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 16 Sep 2012 12:34:59 +0200
Subject: [Python-Dev] [Python-checkins] Daily reference leaks
	(1704deb7e6d7): sum=2
References: <E1TD66G-0001y0-PG@ap.vmr.nerim.net> <50555B7D.9040601@gmail.com>
Message-ID: <20120916123459.6284c0d8@pitrou.net>

On Sun, 16 Sep 2012 06:54:21 +0200
Ross Lagerwall <rosslagerwall at gmail.com> wrote:
> On 09/16/2012 06:04 AM, solipsis at pitrou.net wrote:
> > results for 1704deb7e6d7 on branch "default"
> > --------------------------------------------
> > 
> > test_dbm leaked [0, 2, 0] references, sum=2
> > 
> 
> I've noticed that test_dbm fairly often leaks here although I've never
> reproduced it. Does anyone know with which dbm lib this is (gdbm, ndbm,
> gdbm_compat, bdb)?

It is dbm.dumb.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From chris.jerdonek at gmail.com  Sun Sep 16 12:37:47 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sun, 16 Sep 2012 03:37:47 -0700
Subject: [Python-Dev] packaging location ?
In-Reply-To: <loom.20120914T165543-69@post.gmane.org>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
	<CADiSq7cDMKmiSW+nE1oLhqL3zgGa1zs2Sh14xmK9JbedQgY=bw@mail.gmail.com>
	<loom.20120914T165543-69@post.gmane.org>
Message-ID: <CAOTb1wc-DygAfkQ26OSbR0=yA6=TCm7u7siCdk-g_BBAxsG==w@mail.gmail.com>

On Fri, Sep 14, 2012 at 8:12 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> I have set up a BitBucket repo called distlib, at
>
> https://bitbucket.org/vinay.sajip/distlib/
>
> ...
>
> The code was taken at around the time packaging was removed, and may not have
> more recent changes.

Would it be possible or make sense for this effort to start by forking
the previous Lib/packaging directory of the cpython mirror on
bitbucket (or perhaps do this in a fork of the entire cpython mirror
given that portions of this are meant to be cherry-pick merged back to
cpython)?  That way the continuity with the previous state of
packaging would be clear as well as making later merging easier.

Also, will the previous Doc/packaging docs live in this new repo?

--Chris

From dholth at gmail.com  Sun Sep 16 14:03:47 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 16 Sep 2012 08:03:47 -0400
Subject: [Python-Dev] packaging location ?
In-Reply-To: <CAOTb1wc-DygAfkQ26OSbR0=yA6=TCm7u7siCdk-g_BBAxsG==w@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
	<CADiSq7cDMKmiSW+nE1oLhqL3zgGa1zs2Sh14xmK9JbedQgY=bw@mail.gmail.com>
	<loom.20120914T165543-69@post.gmane.org>
	<CAOTb1wc-DygAfkQ26OSbR0=yA6=TCm7u7siCdk-g_BBAxsG==w@mail.gmail.com>
Message-ID: <FE111638-3B96-4E43-846A-DD78D3D7F709@gmail.com>

On Sep 16, 2012, at 6:37 AM, Chris Jerdonek <chris.jerdonek at gmail.com> wrote:

> On Fri, Sep 14, 2012 at 8:12 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> I have set up a BitBucket repo called distlib, at
>> 
>> https://bitbucket.org/vinay.sajip/distlib/
>> 
>> ...
>> 
>> The code was taken at around the time packaging was removed, and may not have
>> more recent changes.
> 
> Would it be possible or make sense for this effort to start by forking
> the previous Lib/packaging directory of the cpython mirror on
> bitbucket (or perhaps do this in a fork of the entire cpython mirror
> given that portions of this are meant to be cherry-pick merged back to
> cpython)?  That way the continuity with the previous state of
> packaging would be clear as well as making later merging easier.
> 
> Also, will the previous Doc/packaging docs live in this new repo?
> 
> --Chris
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/dholth%40gmail.com

You would also wind up with the entire cpython history in every checkout.

Mercurial allows merging unrelated repositories.

From p.f.moore at gmail.com  Sun Sep 16 14:53:06 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 16 Sep 2012 13:53:06 +0100
Subject: [Python-Dev] packaging location ?
In-Reply-To: <loom.20120914T165543-69@post.gmane.org>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
	<CADiSq7cDMKmiSW+nE1oLhqL3zgGa1zs2Sh14xmK9JbedQgY=bw@mail.gmail.com>
	<loom.20120914T165543-69@post.gmane.org>
Message-ID: <CACac1F9vQn17=x07-3e0uL_7LkqRAthO2CYoVRHcRxdeg---gw@mail.gmail.com>

On 14 September 2012 16:12, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> I have set up a BitBucket repo called distlib, at
>
> https://bitbucket.org/vinay.sajip/distlib/
>
> This has the following bits of distutils2 / packaging, updated to run on 2.x and
> 3.x with a single codebase, and including tests (though not docs, yet):
>
> version.py - version specifiers as per PEP 386
> metadata.py - metadata as per PEPs 345, 314 and 241
> markers.py - environment markers as per PEP 345
> database.py - installed distributions as per PEP 376
> depgraph.py - distribution dependency graph logic
> glob.py - globbing functionality

A nice addition would be an API for managing the RECORD file. I would
imagine functions to read/write the file (hiding the details of how to
open the CSV file correctly in a cross-platform manner), functions to
produce a list of the files installed for a distribution, and
functions to validate (and maybe write) the hashes.

If this would be useful, I'd be willing to write the code, although my
API design skills aren't the best, so some advice on how the API
should look would be nice :-)

Paul

From dholth at gmail.com  Sun Sep 16 21:05:50 2012
From: dholth at gmail.com (Daniel Holth)
Date: Sun, 16 Sep 2012 15:05:50 -0400
Subject: [Python-Dev] packaging location ?
In-Reply-To: <CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
Message-ID: <CAG8k2+6+78Vdca9bp1-JjD7TsrZAJa5Y41-vkESGoo0Jgxr_Ow@mail.gmail.com>

> I agree with Lennart's and Antoine's advice of just move forward with what
> we have. If some PEPs need fixing then let's fix them, but we don't need to
> rock the horse even more by going overboard. Getting the sane, core bits
> into the stdlib as packaging is meant to is plenty to take on. If people
> want to reinvent stuff they can do it elsewhere. I personally don't care if
> it is done inside or outside the stdlib initially or if it stays in
> packaging or goes directly into distutils, but forward movement with what we
> have is the most important thing.

+100

I was excited about packaging in 2010, it is time to document and
implement the specs we have. The sooner we do, the less confusing it
will be for a newcomer who just wants to release a simple printer of
nested lists to pypi.

From ncoghlan at gmail.com  Sun Sep 16 23:09:49 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 17 Sep 2012 07:09:49 +1000
Subject: [Python-Dev] packaging location ?
In-Reply-To: <CAG8k2+6+78Vdca9bp1-JjD7TsrZAJa5Y41-vkESGoo0Jgxr_Ow@mail.gmail.com>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<CAG8k2+6+78Vdca9bp1-JjD7TsrZAJa5Y41-vkESGoo0Jgxr_Ow@mail.gmail.com>
Message-ID: <CADiSq7c4O2zsshZ-x4ME2bw0+O=krZW-Chz6mhGfXUaX3pX0qg@mail.gmail.com>

On Mon, Sep 17, 2012 at 5:05 AM, Daniel Holth <dholth at gmail.com> wrote:
>> I agree with Lennart's and Antoine's advice of just move forward with what
>> we have. If some PEPs need fixing then let's fix them, but we don't need to
>> rock the horse even more by going overboard. Getting the sane, core bits
>> into the stdlib as packaging is meant to is plenty to take on. If people
>> want to reinvent stuff they can do it elsewhere. I personally don't care if
>> it is done inside or outside the stdlib initially or if it stays in
>> packaging or goes directly into distutils, but forward movement with what we
>> have is the most important thing.
>
> +100
>
> I was excited about packaging in 2010, it is time to document and
> implement the specs we have. The sooner we do, the less confusing it
> will be for a newcomer who just wants to release a simple printer of
> nested lists to pypi.

I've been chatting to Chris McDonough a bit as well, and one
potentially useful thing would be to clearly document the
setuptools/distribute metadata precisely as it is generated today.
Currently these formats are entirely implicit in the implementation of
the code that reads and writes them, as far as I can tell anyway. The
distribute docs seem to do a decent job of explaining setup.py and the
various setuptools specific arguments, but *not* what the file formats
will look like inside the metadata directory when installed.

The main advantages of this would be to make it clear:
1. What can setuptools metadata describe, that v1.2 of the official
metadata standard cannot?
2. Does v1.3 allow everything that setuptools can currently describe
(either directly, or as an extension)?
3. Does v1.3 allow some things to be expressed more clearly than they
can be with setuptools?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From vinay_sajip at yahoo.co.uk  Mon Sep 17 00:26:23 2012
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sun, 16 Sep 2012 22:26:23 +0000 (UTC)
Subject: [Python-Dev] packaging location ?
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
	<CADiSq7cDMKmiSW+nE1oLhqL3zgGa1zs2Sh14xmK9JbedQgY=bw@mail.gmail.com>
	<loom.20120914T165543-69@post.gmane.org>
	<CACac1F9vQn17=x07-3e0uL_7LkqRAthO2CYoVRHcRxdeg---gw@mail.gmail.com>
Message-ID: <loom.20120917T001634-295@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:

> A nice addition would be an API for managing the RECORD file. I would
> imagine functions to read/write the file (hiding the details of how to
> open the CSV file correctly in a cross-platform manner), functions to
> produce a list of the files installed for a distribution, and
> functions to validate (and maybe write) the hashes.
> 

Producing the list of files installed for a distribution might be tool-specific
(e.g. a tool might have additional tool-specific files over and above what any
PEPs mandate).

There's a method in database.py - Distribution.list_installed_files() - which
reads and returns the RECORD entries as (path, hash, size) tuples.

I think all that's needed (at the same level of abstraction) is a method
write_installed_files(iterable_of_absolute_file_paths) which writes the file.
This code is already in the distutils2.install_distinfo.install_distinfo.run()
method. I'll pull it out into such a method.

Anyone else, do chip in if you think this is insufficient or sub-optimal.

Perhaps these discussions (on the detail) should continue on distutils-sig. I'll
post there when I've made the changes.

Regards,

Vinay Sajip



From p.f.moore at gmail.com  Mon Sep 17 00:42:39 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 16 Sep 2012 23:42:39 +0100
Subject: [Python-Dev] packaging location ?
In-Reply-To: <loom.20120917T001634-295@post.gmane.org>
References: <50503E19.1030604@ziade.org> <5050DC52.508@netwok.org>
	<CAL0kPAUzY5zyPob==fnyf5NDwhSvwRqrRUhJeBMYNPnkR52UGw@mail.gmail.com>
	<CAP1=2W7r-Zj_kQknok+FS-6DpxoznBnc0gzjowh_xg3zZ+g_=w@mail.gmail.com>
	<20120912224308.56B48250180@webabinitio.net>
	<CADiSq7d67906UWn4LaLqKZv_SyF0XsaJWTBQk0j-a=mSSUwqrw@mail.gmail.com>
	<20120913113857.69dafcb1@pitrou.net> <5051C453.3080003@ziade.org>
	<CAOTb1wdiumKfTZKRw-0n6Z8D6y3gvV+MKE4AkY0h2pQgty6Hag@mail.gmail.com>
	<CADiSq7cQVYSz50msJh9raeFH1L4xZjgdLA3vkaPU4wyvYkTy-A@mail.gmail.com>
	<5051F00D.3040807@netwok.org>
	<CADiSq7cDMKmiSW+nE1oLhqL3zgGa1zs2Sh14xmK9JbedQgY=bw@mail.gmail.com>
	<loom.20120914T165543-69@post.gmane.org>
	<CACac1F9vQn17=x07-3e0uL_7LkqRAthO2CYoVRHcRxdeg---gw@mail.gmail.com>
	<loom.20120917T001634-295@post.gmane.org>
Message-ID: <CACac1F_LBUpPm=DVmjh9KjKFtQsxNyS+=Xi3ab1PYWnRTiWkBg@mail.gmail.com>

On 16 September 2012 23:26, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> I think all that's needed (at the same level of abstraction) is a method
> write_installed_files(iterable_of_absolute_file_paths) which writes the file.
> This code is already in the distutils2.install_distinfo.install_distinfo.run()
> method. I'll pull it out into such a method.

Possibly also useful (if only to standardise the approach) would be a
method to check hashes against installed files.

Paul

From ido at remmon.com  Wed Sep 19 10:51:45 2012
From: ido at remmon.com (Ido Yohanan)
Date: Wed, 19 Sep 2012 11:51:45 +0300
Subject: [Python-Dev] Python 1.5 output file versions
Message-ID: <D58CCF5CAE733B4688D28DF1720AC75B2095F3@sbs2003.Remmon.local>

Hi,

 

I am working with PYTHON 1.5 and want to control versions of every pyo
file.

Is there any way I can assign a file version to every pyo file without
writing a variable inside every py file that holds it?

 

Thanks a lot,

Ido

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120919/efb46055/attachment.html>

From steve at pearwood.info  Wed Sep 19 18:22:03 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 20 Sep 2012 02:22:03 +1000
Subject: [Python-Dev] Python 1.5 output file versions
In-Reply-To: <D58CCF5CAE733B4688D28DF1720AC75B2095F3@sbs2003.Remmon.local>
References: <D58CCF5CAE733B4688D28DF1720AC75B2095F3@sbs2003.Remmon.local>
Message-ID: <5059F12B.3000001@pearwood.info>

On 19/09/12 18:51, Ido Yohanan wrote:
> Hi,
>
>
>
> I am working with PYTHON 1.5 and want to control versions of every pyo
> file.

Python 1.5? Are you serious?

Python 1.5 is now at least 8 versions obsolete, and hasn't been updated
since approximately 1995.

> Is there any way I can assign a file version to every pyo file without
> writing a variable inside every py file that holds it?

I'm afraid I don't understand what you are actually asking here.



-- 
Steven

From brett at python.org  Wed Sep 19 19:03:52 2012
From: brett at python.org (Brett Cannon)
Date: Wed, 19 Sep 2012 13:03:52 -0400
Subject: [Python-Dev] Python 1.5 output file versions
In-Reply-To: <D58CCF5CAE733B4688D28DF1720AC75B2095F3@sbs2003.Remmon.local>
References: <D58CCF5CAE733B4688D28DF1720AC75B2095F3@sbs2003.Remmon.local>
Message-ID: <CAP1=2W78YsnNZQR_ya8sHeu2j6-XWWsQcXBtiokRv21HwNuOBw@mail.gmail.com>

First, this is the wrong mailing list for this question: python-dev is for
discussing the development *of* Python, not *with* it.

Second, the answer to your question is there is no built-in versioning
support.

On Wed, Sep 19, 2012 at 4:51 AM, Ido Yohanan <ido at remmon.com> wrote:

>  Hi,****
>
> ** **
>
> I am working with *PYTHON 1.5* and want to control versions of every pyo
> file.****
>
> Is there any way I can assign a file version to every pyo file without
> writing a variable inside every py file that holds it?****
>
> ** **
>
> Thanks a lot,****
>
> Ido****
>
>
> __________ Information from ESET Smart Security, version of virus
> signature database 7492 (20120918) __________
>
> The message was checked by ESET Smart Security.
>
> http://www.eset.com
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120919/f6de521e/attachment.html>

From dickinsm at gmail.com  Thu Sep 20 13:56:38 2012
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 20 Sep 2012 12:56:38 +0100
Subject: [Python-Dev] TypeError: f() missing 1 required positional argument:
	'x'
Message-ID: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>

I suspect I've missed the boat on this one (certainly for 3.3.0), but
here goes.  The new TypeError reporting for bad function calls is a
huge improvement (thanks Benjamin!), but I have one small nitpick:
what *is* a positional argument?  For example:

    >>> def f(x): pass
    ...
    >>> f()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: f() missing 1 required positional argument: 'x'

I think it's confusing to describe 'x' as a positional argument.  It's
a required formal parameter, certainly.  But a caller of 'f' could
pass 'x' either by position or by 'keyword'.


When running training (generally Python 2.6 or 2.7 based), I
frequently have to devote some time to unravelling student confusion
between 'arguments passed by keyword' on one hand and 'optional formal
parameters' on the other.  The outline of the explanation goes
something like:

(0) Preamble: be careful to separate out details of function calling
from those of function definition; distinguish formal parameters from
actual arguments.
(1) On the function *definition* side, formal parameters may be either
*required* or *optional*.
(2) On the function *calling* side, actual arguments may be passed
either positionally or by keyword.
(3) The notions in (1) and (2) are entirely orthogonal!
(3a) (Although in practice, callers tend to use pass-by-keyword for
optional formal parameters.)

That's all for Python 2;  Python 3, of course, requires a bit more
explanation related to the keyword-only arguments.

There already seems to be a fair amount of confusion in the Python
world about point (3);  I've seen professional Python training slides
that show how to define optional formal parameters under the heading
"keyword arguments".

I submit that the word 'positional' in the TypeError message
exacerbates this confusion, and that little would be lost by simply
dropping it from the exception message.

Thoughts?

Mark

From ncoghlan at gmail.com  Thu Sep 20 14:21:39 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 20 Sep 2012 22:21:39 +1000
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
Message-ID: <CADiSq7fERUAZNe17i=mKa7LzAyNq0yvuwTAN3FVJYG-gZHPqDw@mail.gmail.com>

On Thu, Sep 20, 2012 at 9:56 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> I submit that the word 'positional' in the TypeError message
> exacerbates this confusion, and that little would be lost by simply
> dropping it from the exception message.

+1 for using the unqualified "argument" in these error messages to
mean "positional or keyword argument" (inspect.Parameter spells it out
as POSITIONAL_OR_KEYWORD, but the full phrase is far too verbose for
an error message).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From dickinsm at gmail.com  Thu Sep 20 14:59:16 2012
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 20 Sep 2012 13:59:16 +0100
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CADiSq7fERUAZNe17i=mKa7LzAyNq0yvuwTAN3FVJYG-gZHPqDw@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CADiSq7fERUAZNe17i=mKa7LzAyNq0yvuwTAN3FVJYG-gZHPqDw@mail.gmail.com>
Message-ID: <CAAu3qLXHOs8G4LtxJ6q=TE5NfQ-sRquom88CzCXmzhfVK+OcNA@mail.gmail.com>

On Thu, Sep 20, 2012 at 1:21 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> +1 for using the unqualified "argument" in these error messages to
> mean "positional or keyword argument" (inspect.Parameter spells it out
> as POSITIONAL_OR_KEYWORD, but the full phrase is far too verbose for
> an error message).

Ah yes;  I see that 'positional or keyword' is a more accurate term
(but agree it's unwieldy for an error message).  I also see that I was
naive to think that the 'fix' is as simple as dropping the word
'positional':

    >>> def f(a, *, b):
    ...     pass
    ...
    >>> f()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: f() missing 1 required positional argument: 'a'

If the word 'positional' were dropped here, it would give the
incorrect impression that f only requires one argument.

Perhaps this simply isn't worth worrying about, especially since the
current error messages are all but certain to make it into the 3.3
release.

Mark

From ncoghlan at gmail.com  Thu Sep 20 15:13:56 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 20 Sep 2012 23:13:56 +1000
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CAAu3qLXHOs8G4LtxJ6q=TE5NfQ-sRquom88CzCXmzhfVK+OcNA@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CADiSq7fERUAZNe17i=mKa7LzAyNq0yvuwTAN3FVJYG-gZHPqDw@mail.gmail.com>
	<CAAu3qLXHOs8G4LtxJ6q=TE5NfQ-sRquom88CzCXmzhfVK+OcNA@mail.gmail.com>
Message-ID: <CADiSq7f7b_fgWnft5L1cC_-3-1BftmRUsChyZZ-_u4MXiXT7Ng@mail.gmail.com>

On Thu, Sep 20, 2012 at 10:59 PM, Mark Dickinson <dickinsm at gmail.com> wrote:
> Perhaps this simply isn't worth worrying about, especially since the
> current error messages are all but certain to make it into the 3.3
> release.

No "all but" about it at this point - the earliest they could change
again is 3.3.1. Hopefully the new signature inspection support will
help explain some of the intricacies of binding argument values to
parameter names :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From benjamin at python.org  Thu Sep 20 16:12:04 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 20 Sep 2012 10:12:04 -0400
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
Message-ID: <CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>

2012/9/20 Mark Dickinson <dickinsm at gmail.com>:
> Thoughts?

I tried to define the error messages in terms of the callee's
signature. I call the formals that are not variadic, keyword variadic,
or keyword-only, positional. For example, in

def f(a, b, c, *args, d):
     pass

a, b, and c are positional. Hence the "positional" in error messages.

As you noted in your next message, keyword-only arguments need to be
distinguished from these "positional" arguments somehow. Maybe it
helps to think of "positional" to mean "the only formals you can pass
to with position" (excepting variadic ones).

I'm certainly open to suggestions.



-- 
Regards,
Benjamin

From chris.jerdonek at gmail.com  Thu Sep 20 16:43:34 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Thu, 20 Sep 2012 07:43:34 -0700
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
Message-ID: <CAOTb1wdzvP0+eH_+bLPhZKx6rL9bUMhdfPi+Y3_+H2SEE29y2w@mail.gmail.com>

On Thu, Sep 20, 2012 at 7:12 AM, Benjamin Peterson <benjamin at python.org> wrote:
> def f(a, b, c, *args, d):
>      pass
>
> a, b, and c are positional. Hence the "positional" in error messages.
>
> As you noted in your next message, keyword-only arguments need to be
> distinguished from these "positional" arguments somehow. Maybe it
> helps to think of "positional" to mean "the only formals you can pass
> to with position" (excepting variadic ones).

That is how I think of "positional".

However, the other wrinkle is that some arguments really are
"position-only," for example:

>>> len(s="abc")
Traceback (most recent call last):
 ...
TypeError: len() takes no keyword arguments

I think it is a defect of our documentation that we don't have a way
to distinguish between "positional" and "position-only" arguments in
the function signature notation we use in our documentation, leading
to issues like this one:

"accept keyword arguments on most base type methods and builtins":

http://bugs.python.org/issue8706

--Chris

From solipsis at pitrou.net  Thu Sep 20 16:49:16 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 20 Sep 2012 16:49:16 +0200
Subject: [Python-Dev] TypeError: f() missing 1 required positional
	argument: 'x'
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
Message-ID: <20120920164916.72cc2519@pitrou.net>

On Thu, 20 Sep 2012 10:12:04 -0400
Benjamin Peterson <benjamin at python.org> wrote:
> 2012/9/20 Mark Dickinson <dickinsm at gmail.com>:
> > Thoughts?
> 
> I tried to define the error messages in terms of the callee's
> signature. I call the formals that are not variadic, keyword variadic,
> or keyword-only, positional. For example, in
> 
> def f(a, b, c, *args, d):
>      pass
> 
> a, b, and c are positional. Hence the "positional" in error messages.

But since the error message gives the name of the parameter, there
doesn't seem to be a point to add that it's "positional": it can be
trivially deduced from the function signature.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From dickinsm at gmail.com  Thu Sep 20 16:59:36 2012
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 20 Sep 2012 15:59:36 +0100
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
Message-ID: <CAAu3qLXYvS1BoTNhnO1Ft4B1KnzYEUC_6TiiFcxr2NYiVZZryg@mail.gmail.com>

On Thu, Sep 20, 2012 at 3:12 PM, Benjamin Peterson <benjamin at python.org> wrote:

> As you noted in your next message, keyword-only arguments need to be
> distinguished from these "positional" arguments somehow. Maybe it
> helps to think of "positional" to mean "the only formals you can pass
> to with position" (excepting variadic ones).

And excepting optional ones, too, right?  E.g., the c in

    def foo(a, b, c=1, *args, d):
        pass

can be passed to by position, but isn't "positional".

> I'm certainly open to suggestions.

Yes, I don't have a good alternative suggestion.  If we could find a
suitable word and bless it in the documentation, it might make it
easier to make clear and accurate statements about Python's function
calling.

Mark

From benjamin at python.org  Thu Sep 20 17:14:57 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 20 Sep 2012 11:14:57 -0400
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CAAu3qLXYvS1BoTNhnO1Ft4B1KnzYEUC_6TiiFcxr2NYiVZZryg@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
	<CAAu3qLXYvS1BoTNhnO1Ft4B1KnzYEUC_6TiiFcxr2NYiVZZryg@mail.gmail.com>
Message-ID: <CAPZV6o-j5Z4h2EmVtNVKK=OJBNBtmCG-uSiWaSHfrv58+K1gTA@mail.gmail.com>

2012/9/20 Mark Dickinson <dickinsm at gmail.com>:
> And excepting optional ones, too, right?  E.g., the c in
>
>     def foo(a, b, c=1, *args, d):
>         pass
>
> can be passed to by position, but isn't "positional".

Why not?

>>> def f(a, b, c=3): pass
...
>>> f()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() missing 2 required positional arguments: 'a' and 'b'
>>> f(1, 2, 3, 4)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() takes from 2 to 3 positional arguments but 4 were given



-- 
Regards,
Benjamin

From dickinsm at gmail.com  Thu Sep 20 17:21:40 2012
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 20 Sep 2012 16:21:40 +0100
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CAPZV6o-j5Z4h2EmVtNVKK=OJBNBtmCG-uSiWaSHfrv58+K1gTA@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
	<CAAu3qLXYvS1BoTNhnO1Ft4B1KnzYEUC_6TiiFcxr2NYiVZZryg@mail.gmail.com>
	<CAPZV6o-j5Z4h2EmVtNVKK=OJBNBtmCG-uSiWaSHfrv58+K1gTA@mail.gmail.com>
Message-ID: <CAAu3qLX5gqcybBRBiu6h0y2ZX6GAiEnDaJzK3-ZCQb52xOcnVg@mail.gmail.com>

On Thu, Sep 20, 2012 at 4:14 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2012/9/20 Mark Dickinson <dickinsm at gmail.com>:
>> And excepting optional ones, too, right?  E.g., the c in
>>
>>     def foo(a, b, c=1, *args, d):
>>         pass
>>
>> can be passed to by position, but isn't "positional".
>
> Why not?

Ah, okay;  I was assuming (wrongly) that your definition of
'positional' was intended to exclude these.  My bad.

Mark

From cedric.krier at b2ck.com  Thu Sep 20 17:37:27 2012
From: cedric.krier at b2ck.com (=?utf-8?Q?C=C3=A9dric?= Krier)
Date: Thu, 20 Sep 2012 17:37:27 +0200
Subject: [Python-Dev] Take a look at Issue15421
Message-ID: <20120920153727.GS21714@tetsuo.ced.homedns.org>

Hi,

Could someone take a look at the Issue15421 [1]?
It is a small bug in calendar module but it generates noise on web
application like roundup [2].

Thanks,

[1] http://bugs.python.org/issue15421
[2] http://issues.roundup-tracker.org/issue2550765

-- 
C?dric Krier

B2CK SPRL
Rue de Rotterdam, 4
4000 Li?ge
Belgium
Tel: +32 472 54 46 59
Email/Jabber: cedric.krier at b2ck.com
Website: http://www.b2ck.com/

From guido at python.org  Thu Sep 20 17:52:47 2012
From: guido at python.org (Guido van Rossum)
Date: Thu, 20 Sep 2012 08:52:47 -0700
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
Message-ID: <CAP7+vJKD80Fs2CsFD9iqjcYWKUyAbAz8O2wQprZ=bLCehUpUgw@mail.gmail.com>

On Thu, Sep 20, 2012 at 7:12 AM, Benjamin Peterson <benjamin at python.org> wrote:
> 2012/9/20 Mark Dickinson <dickinsm at gmail.com>:
>> Thoughts?
>
> I tried to define the error messages in terms of the callee's
> signature. I call the formals that are not variadic, keyword variadic,
> or keyword-only, positional. For example, in
>
> def f(a, b, c, *args, d):
>      pass
>
> a, b, and c are positional. Hence the "positional" in error messages.

No -- Mark's point is that (even given this syntax) you *could* pass
them using keywords.

I think Brett's got it right and we should just refer to a and b as
'arguments'. For d, we should use keyword arguments (or, in full,
keyword-only arguments). That's enough of a distinction.

Of course, in a specific call, we can continue to refer to positional
and keyword arguments based on the actual syntax used in the call.

Maybe this is also a good time to start distinguishing between
arguments (what you pass, call syntax) and parameters (what the
function receives, function definition syntax)?

> As you noted in your next message, keyword-only arguments need to be
> distinguished from these "positional" arguments somehow. Maybe it
> helps to think of "positional" to mean "the only formals you can pass
> to with position" (excepting variadic ones).
>
> I'm certainly open to suggestions.

-- 
--Guido van Rossum (python.org/~guido)

From oscar.j.benjamin at gmail.com  Thu Sep 20 17:53:33 2012
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Thu, 20 Sep 2012 16:53:33 +0100
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CAPZV6o-j5Z4h2EmVtNVKK=OJBNBtmCG-uSiWaSHfrv58+K1gTA@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
	<CAAu3qLXYvS1BoTNhnO1Ft4B1KnzYEUC_6TiiFcxr2NYiVZZryg@mail.gmail.com>
	<CAPZV6o-j5Z4h2EmVtNVKK=OJBNBtmCG-uSiWaSHfrv58+K1gTA@mail.gmail.com>
Message-ID: <CAHVvXxQH8F56W--iM=jpw4pUHvcascoJ0oQrvX6KnpCpLNw0Lg@mail.gmail.com>

On 20 September 2012 16:14, Benjamin Peterson <benjamin at python.org> wrote:

> 2012/9/20 Mark Dickinson <dickinsm at gmail.com>:
> > And excepting optional ones, too, right?  E.g., the c in
> >
> >     def foo(a, b, c=1, *args, d):
> >         pass
> >
> > can be passed to by position, but isn't "positional".
>
> Why not?
>
> >>> def f(a, b, c=3): pass
> ...
> >>> f()
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: f() missing 2 required positional arguments: 'a' and 'b'
> >>> f(1, 2, 3, 4)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: f() takes from 2 to 3 positional arguments but 4 were given
>

The difference between c and a,b is that c is optional, whereas a and b are
required.

In Python 2.x there are named arguments and variadic arguments. There are
two types of named arguments: required and optional. There are also two
types of variadic arguments: positional and keyword. i.e.:

named
  required
  not-required
variadic
  positional
  keyword

In Python 2.x all named parameters can be passed by position or by keyword,
so it doesn't make sense to use those concepts to distinguish them. On the
other hand, for variadic parameters that distinction is crucial.

In Python 3.x there are two orthogonal properties for each named parameter.
The parameter can be required or optional as before, and then the parameter
can be keyword-only or positional. There are 4 combinations of these two
properties:

def f(a, b=1, *, c, d=3): pass

                   | required | optional
positional | a              | b
kwonly       | c              | d

Since there are two orthogonal properties of a parameter (requiredness and
positionness) it makes perfect sense to use two adjectives to describe each
parameter as is the case for the error message shown at the start of this
thread:

Mark Dickinson wrote:
>>>> def f(x): pass
>...
>>>> f()
>Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
>TypeError: f() missing 1 required positional argument: 'x'

I would say that the only problem with this terminology is that it would be
good to think of a word to replace "keyword-only" (positionless?).

Oscar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120920/fb93c5ca/attachment-0001.html>

From tjreedy at udel.edu  Thu Sep 20 17:57:10 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 20 Sep 2012 11:57:10 -0400
Subject: [Python-Dev] TypeError: f() missing 1 required positional
	argument: 'x'
In-Reply-To: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
Message-ID: <k3fecs$h8f$1@ger.gmane.org>

On 9/20/2012 7:56 AM, Mark Dickinson wrote:
> I suspect I've missed the boat on this one (certainly for 3.3.0), but
> here goes.  The new TypeError reporting for bad function calls is a
> huge improvement (thanks Benjamin!), but I have one small nitpick:
> what *is* a positional argument?  For example:
>
>      >>> def f(x): pass
>      ...
>      >>> f()
>      Traceback (most recent call last):
>        File "<stdin>", line 1, in <module>
>      TypeError: f() missing 1 required positional argument: 'x'
>
> I think it's confusing to describe 'x' as a positional argument.  It's
> a required formal parameter, certainly.  But a caller of 'f' could
> pass 'x' either by position or by 'keyword'.
>
>
> When running training (generally Python 2.6 or 2.7 based), I
> frequently have to devote some time to unravelling student confusion
> between 'arguments passed by keyword' on one hand and 'optional formal
> parameters' on the other.  The outline of the explanation goes
> something like:
>
> (0) Preamble: be careful to separate out details of function calling
> from those of function definition; distinguish formal parameters from
> actual arguments.
> (1) On the function *definition* side, formal parameters may be either
> *required* or *optional*.

and optional params may or may not have an overt default object

> (2) On the function *calling* side, actual arguments may be passed
> either positionally or by keyword.

Sometimes position is required, sometimes keyword is required, and 
usually both are allowed.

> (3) The notions in (1) and (2) are entirely orthogonal!

Moreover, all six combinations of passing mode and requirement are 
possible, although for Python functions, some combinations require setup 
code in addition to the header. Built-in print accepts an indefinite 
number of optional no-default position-only args followed by up to three 
optional defaulted keyword-only args. print() emits the default end='\n'.

> (3a) (Although in practice, callers tend to use pass-by-keyword for
> optional formal parameters.)
>
> That's all for Python 2;  Python 3, of course, requires a bit more
> explanation related to the keyword-only arguments.
>
> There already seems to be a fair amount of confusion in the Python
> world about point (3);

I have strongly suggested that the docs not adds to the confusion in at 
least one tracker discussion.

>  I've seen professional Python training slides
> that show how to define optional formal parameters under the heading
> "keyword arguments".
>
> I submit that the word 'positional' in the TypeError message
> exacerbates this confusion, and that little would be lost by simply
> dropping it from the exception message.

For this example that would be sufficient, but your later message shows 
that we need a one-word abbreviations for positional-or-keyword: either 
something indicating that its default nature -- 'normal', 'standard', 
'flexible', 'usual', 'typical' -- or something indicating its dual 
nature (possibly coined or metaphorical -- 'pos-key', 'bi-mode', 
'dual-mode', 'Janus-like'.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Thu Sep 20 18:07:09 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 20 Sep 2012 12:07:09 -0400
Subject: [Python-Dev] TypeError: f() missing 1 required positional
	argument: 'x'
In-Reply-To: <CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
Message-ID: <k3fevj$n16$1@ger.gmane.org>

On 9/20/2012 10:12 AM, Benjamin Peterson wrote:
> 2012/9/20 Mark Dickinson <dickinsm at gmail.com>:
>> Thoughts?
>
> I tried to define the error messages in terms of the callee's
> signature. I call the formals that are not variadic, keyword variadic,
> or keyword-only, positional. For example, in
>
> def f(a, b, c, *args, d):
>       pass
>
> a, b, and c are positional. Hence the "positional" in error messages.

They are positional-or-keyword without defaults.

> As you noted in your next message, keyword-only arguments need to be
> distinguished from these "positional" arguments somehow.

Positional-or-keyword and positional-only also need to be distinguished.
'Positional' is ambiguous. One problem for standardized error messages 
is the the header info does not always tell the complete story.

> I'm certainly open to suggestions.

I gave several suggestions for 'positional-or-keyword' in my response to 
Mark.

-- 
Terry Jan Reedy


From dave at dabeaz.com  Thu Sep 20 18:36:29 2012
From: dave at dabeaz.com (David Beazley)
Date: Thu, 20 Sep 2012 11:36:29 -0500
Subject: [Python-Dev] Memoryviews should expose the underlying memory address
Message-ID: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>

I have recently been experimenting with the memoryview() built-in and have come to believe that it really needs to expose the 'buf' attribute of the underlying Py_buffer structure as an integer (see PEP 3118).  Let me explain.

The whole point of PEP 3118 (as I understand it) is to provide a means for exchanging or sharing array data across different libraries such as numpy, PIL, ctypes, Cython, etc.   If you're working with Py_buffer objects at the C level, this works fine.  However, if you're working purely in Python, you're only able to get partial information about memory views such as the shape and size.   You can't get the actual pointer to the underlying memory (unless I've missed something obvious).

This is unfortunate because it means that you can't  write Python code to link memoryviews to other kinds of compiled code that might want to operate on array-oriented data.  For example, you can't pass the raw pointer into a function that you've exposed via ctypes.  Similarly, you can't pass the pointer into functions you've dynamically compiled using libraries such as LLVM-py.     There might be other kinds of applications, but just having that one bit of extra information available would be useful for various advanced programming techniques involving extensions and memory buffers.

Cheers,
Dave







From benjamin at python.org  Thu Sep 20 18:48:02 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 20 Sep 2012 12:48:02 -0400
Subject: [Python-Dev] Memoryviews should expose the underlying memory
	address
In-Reply-To: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
References: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
Message-ID: <CAPZV6o8QnekwCLWeSh4fJXMvE2qXVcEA2LZS3TGMFX+-dYhk=w@mail.gmail.com>

2012/9/20 David Beazley <dave at dabeaz.com>:
> I have recently been experimenting with the memoryview() built-in and have come to believe that it really needs to expose the 'buf' attribute of the underlying Py_buffer structure as an integer (see PEP 3118).  Let me explain.
>
> The whole point of PEP 3118 (as I understand it) is to provide a means for exchanging or sharing array data across different libraries such as numpy, PIL, ctypes, Cython, etc.   If you're working with Py_buffer objects at the C level, this works fine.  However, if you're working purely in Python, you're only able to get partial information about memory views such as the shape and size.   You can't get the actual pointer to the underlying memory (unless I've missed something obvious).
>
> This is unfortunate because it means that you can't  write Python code to link memoryviews to other kinds of compiled code that might want to operate on array-oriented data.  For example, you can't pass the raw pointer into a function that you've exposed via ctypes.  Similarly, you can't pass the pointer into functions you've dynamically compiled using libraries such as LLVM-py.     There might be other kinds of applications, but just having that one bit of extra information available would be useful for various advanced programming techniques involving extensions and memory buffers.

Presumably ctypes should be able to do this conversion for you.



-- 
Regards,
Benjamin

From dave at dabeaz.com  Thu Sep 20 18:53:46 2012
From: dave at dabeaz.com (David Beazley)
Date: Thu, 20 Sep 2012 11:53:46 -0500
Subject: [Python-Dev] Memoryviews should expose the underlying memory
	address
In-Reply-To: <CAPZV6o8QnekwCLWeSh4fJXMvE2qXVcEA2LZS3TGMFX+-dYhk=w@mail.gmail.com>
References: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
	<CAPZV6o8QnekwCLWeSh4fJXMvE2qXVcEA2LZS3TGMFX+-dYhk=w@mail.gmail.com>
Message-ID: <A5245A71-D85F-4763-B82F-7BB076DC0A0F@dabeaz.com>

How?  I must be missing something very obvious.

Cheers,
Dave

On Sep 20, 2012, at 11:48 AM, Benjamin Peterson wrote:

> 2012/9/20 David Beazley <dave at dabeaz.com>:
>> I have recently been experimenting with the memoryview() built-in and have come to believe that it really needs to expose the 'buf' attribute of the underlying Py_buffer structure as an integer (see PEP 3118).  Let me explain.
>> 
>> The whole point of PEP 3118 (as I understand it) is to provide a means for exchanging or sharing array data across different libraries such as numpy, PIL, ctypes, Cython, etc.   If you're working with Py_buffer objects at the C level, this works fine.  However, if you're working purely in Python, you're only able to get partial information about memory views such as the shape and size.   You can't get the actual pointer to the underlying memory (unless I've missed something obvious).
>> 
>> This is unfortunate because it means that you can't  write Python code to link memoryviews to other kinds of compiled code that might want to operate on array-oriented data.  For example, you can't pass the raw pointer into a function that you've exposed via ctypes.  Similarly, you can't pass the pointer into functions you've dynamically compiled using libraries such as LLVM-py.     There might be other kinds of applications, but just having that one bit of extra information available would be useful for various advanced programming techniques involving extensions and memory buffers.
> 
> Presumably ctypes should be able to do this conversion for you.
> 
> 
> 
> -- 
> Regards,
> Benjamin


From chris.jerdonek at gmail.com  Thu Sep 20 19:18:41 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Thu, 20 Sep 2012 10:18:41 -0700
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CAP7+vJKD80Fs2CsFD9iqjcYWKUyAbAz8O2wQprZ=bLCehUpUgw@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
	<CAP7+vJKD80Fs2CsFD9iqjcYWKUyAbAz8O2wQprZ=bLCehUpUgw@mail.gmail.com>
Message-ID: <CAOTb1wdebpyVSa-hgJ=QtS1tsf6uFGZr6fk=_6M6JjqNHoidgw@mail.gmail.com>

On Thu, Sep 20, 2012 at 8:52 AM, Guido van Rossum <guido at python.org> wrote:
> On Thu, Sep 20, 2012 at 7:12 AM, Benjamin Peterson <benjamin at python.org> wrote:
>> I tried to define the error messages in terms of the callee's
>> signature. I call the formals that are not variadic, keyword variadic,
>> or keyword-only, positional. For example, in
>
> Maybe this is also a good time to start distinguishing between
> arguments (what you pass, call syntax) and parameters (what the
> function receives, function definition syntax)?

The glossary is one place to start making this distinction.  It
currently has entries for "argument," "positional argument," and
"keyword argument" that could perhaps use a review from this
discussion.  For example:

http://docs.python.org/dev/glossary.html#term-positional-argument

The entries currently blur the distinction between the calling and
definition perspectives.  Ideally, the glossary definitions of these
terms would match and be consistent with their usage in error
messages.

--Chris

From tjreedy at udel.edu  Thu Sep 20 19:25:28 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 20 Sep 2012 13:25:28 -0400
Subject: [Python-Dev] TypeError: f() missing 1 required positional
	argument: 'x'
In-Reply-To: <CAP7+vJKD80Fs2CsFD9iqjcYWKUyAbAz8O2wQprZ=bLCehUpUgw@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
	<CAP7+vJKD80Fs2CsFD9iqjcYWKUyAbAz8O2wQprZ=bLCehUpUgw@mail.gmail.com>
Message-ID: <k3fjie$3u9$1@ger.gmane.org>

On 9/20/2012 11:52 AM, Guido van Rossum wrote:

> Maybe this is also a good time to start distinguishing between
> arguments (what you pass, call syntax) and parameters (what the
> function receives, function definition syntax)?

One standard usage (and mine) is that parameters are the (local) names 
that arguments get bound to. I *believe* that Knuth used this also, but 
I cannot find a reference. Here is the CS part of
https://en.wikipedia.org/wiki/Parameters
See the last sentence.

"Computer science
Main article: Parameter (computer science)

When the terms formal parameter and actual parameter are used, they 
generally correspond with the definitions used in computer science. In 
the definition of a function such as

     f(x) = x + 2,

x is a formal parameter. When the function is used as in

     y = f(3) + 5 or just the value of f(3),

3 is the actual parameter value that is substituted for the formal 
parameter in the function definition. These concepts are discussed in a 
more precise way in functional programming and its foundational 
disciplines, lambda calculus and combinatory logic.

In computing, parameters are often called arguments, and the two words 
are used interchangeably. However, some computer languages such as C 
define argument to mean actual parameter (i.e., the value), and 
parameter to mean formal parameter."

-- 
Terry Jan Reedy


From shibturn at gmail.com  Thu Sep 20 19:57:37 2012
From: shibturn at gmail.com (Richard Oudkerk)
Date: Thu, 20 Sep 2012 18:57:37 +0100
Subject: [Python-Dev] Memoryviews should expose the underlying memory
	address
In-Reply-To: <A5245A71-D85F-4763-B82F-7BB076DC0A0F@dabeaz.com>
References: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
	<CAPZV6o8QnekwCLWeSh4fJXMvE2qXVcEA2LZS3TGMFX+-dYhk=w@mail.gmail.com>
	<A5245A71-D85F-4763-B82F-7BB076DC0A0F@dabeaz.com>
Message-ID: <k3flek$me2$1@ger.gmane.org>

On 20/09/2012 5:53pm, David Beazley wrote:
> How?  I must be missing something very obvious.

I would not call it obvious, but you can do

     >>> m = memoryview(bytearray(5))
     >>> ctypes.addressof(ctypes.c_char.from_buffer(m))
     149979304

However, this only works for writable memoryviews.  For
read-only memoryviews you could do

     >>> obj = ctypes.py_object(m)
     >>> address = ctypes.c_void_p()
     >>> length = ctypes.c_ssize_t()
     >>> ctypes.pythonapi.PyObject_AsReadBuffer(obj, 
ctypes.byref(address), ctypes.byref(length))
     0
     >>> address, length
     (c_void_p(149979304), c_long(5))



From stefan at bytereef.org  Thu Sep 20 19:58:29 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Thu, 20 Sep 2012 19:58:29 +0200
Subject: [Python-Dev] Memoryviews should expose the underlying memory
 address
In-Reply-To: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
References: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
Message-ID: <20120920175829.GA15391@sleipnir.bytereef.org>

David Beazley <dave at dabeaz.com> wrote:
> I have recently been experimenting with the memoryview() built-in and have
> come to believe that it really needs to expose the 'buf' attribute of the
> underlying Py_buffer structure as an integer (see PEP 3118).  Let me explain.

That sounds quite harmless. People who use the pointer via ctypes etc.
should know the implications. I've opened #15986 for this.


Stefan Krah




From benjamin at python.org  Thu Sep 20 20:20:22 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 20 Sep 2012 14:20:22 -0400
Subject: [Python-Dev] Memoryviews should expose the underlying memory
	address
In-Reply-To: <A5245A71-D85F-4763-B82F-7BB076DC0A0F@dabeaz.com>
References: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
	<CAPZV6o8QnekwCLWeSh4fJXMvE2qXVcEA2LZS3TGMFX+-dYhk=w@mail.gmail.com>
	<A5245A71-D85F-4763-B82F-7BB076DC0A0F@dabeaz.com>
Message-ID: <CAPZV6o9GZsum5CuCy-WUwuBXuxWuavU0MPyZY131=mEwFjBirA@mail.gmail.com>

2012/9/20 David Beazley <dave at dabeaz.com>:
> How?  I must be missing something very obvious.

If you have some ctypes function that requires a pointer and you pass
a memoryview, ctypes should pass the pointer to the raw memory, right?



-- 
Regards,
Benjamin

From dave at dabeaz.com  Thu Sep 20 20:35:46 2012
From: dave at dabeaz.com (David Beazley)
Date: Thu, 20 Sep 2012 13:35:46 -0500
Subject: [Python-Dev] Memoryviews should expose the underlying memory
	address
In-Reply-To: <CAPZV6o9GZsum5CuCy-WUwuBXuxWuavU0MPyZY131=mEwFjBirA@mail.gmail.com>
References: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
	<CAPZV6o8QnekwCLWeSh4fJXMvE2qXVcEA2LZS3TGMFX+-dYhk=w@mail.gmail.com>
	<A5245A71-D85F-4763-B82F-7BB076DC0A0F@dabeaz.com>
	<CAPZV6o9GZsum5CuCy-WUwuBXuxWuavU0MPyZY131=mEwFjBirA@mail.gmail.com>
Message-ID: <AA89697C-F3F5-43D6-A017-AF909AAC0031@dabeaz.com>

Well, if it's supposed to do that, it certainly doesn't work for me in 3.3.  I get a type error about it wanting a ctypes pointer object.    Even if this worked, it still doesn't address the need to get the pointer value possibly for some other purpose such as handling it off to a bunch of code generated via LLVM. 

Cheers,
Dave

On Sep 20, 2012, at 1:20 PM, Benjamin Peterson wrote:

> 2012/9/20 David Beazley <dave at dabeaz.com>:
>> How?  I must be missing something very obvious.
> 
> If you have some ctypes function that requires a pointer and you pass
> a memoryview, ctypes should pass the pointer to the raw memory, right?
> 
> 
> 
> -- 
> Regards,
> Benjamin


From steve at pearwood.info  Thu Sep 20 20:49:46 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 21 Sep 2012 04:49:46 +1000
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CAAu3qLXHOs8G4LtxJ6q=TE5NfQ-sRquom88CzCXmzhfVK+OcNA@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CADiSq7fERUAZNe17i=mKa7LzAyNq0yvuwTAN3FVJYG-gZHPqDw@mail.gmail.com>
	<CAAu3qLXHOs8G4LtxJ6q=TE5NfQ-sRquom88CzCXmzhfVK+OcNA@mail.gmail.com>
Message-ID: <505B654A.90006@pearwood.info>

On 20/09/12 22:59, Mark Dickinson wrote:
> On Thu, Sep 20, 2012 at 1:21 PM, Nick Coghlan<ncoghlan at gmail.com>  wrote:
>> +1 for using the unqualified "argument" in these error messages to
>> mean "positional or keyword argument" (inspect.Parameter spells it out
>> as POSITIONAL_OR_KEYWORD, but the full phrase is far too verbose for
>> an error message).
>
> Ah yes;  I see that 'positional or keyword' is a more accurate term
> (but agree it's unwieldy for an error message).  I also see that I was
> naive to think that the 'fix' is as simple as dropping the word
> 'positional':
>
>      >>>  def f(a, *, b):
>      ...     pass
>      ...
>      >>>  f()
>      Traceback (most recent call last):
>        File "<stdin>", line 1, in<module>
>      TypeError: f() missing 1 required positional argument: 'a'
>
> If the word 'positional' were dropped here, it would give the
> incorrect impression that f only requires one argument.


I don't expect error messages to give a complete catalog of every
problem with a specific function call. If f() reports that required
argument 'a' is missing, that does not imply that no other required
arguments are also missing. I think it is perfectly acceptable to
not report the missing 'b' until the missing 'a' is resolved.

But I do expect error messages to be accurate. +1 to remove the
word "positional" from the message.



-- 
Steven

From steve at pearwood.info  Thu Sep 20 21:02:09 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 21 Sep 2012 05:02:09 +1000
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <20120920164916.72cc2519@pitrou.net>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
	<20120920164916.72cc2519@pitrou.net>
Message-ID: <505B6831.1060207@pearwood.info>

On 21/09/12 00:49, Antoine Pitrou wrote:
> On Thu, 20 Sep 2012 10:12:04 -0400
> Benjamin Peterson<benjamin at python.org>  wrote:
>> 2012/9/20 Mark Dickinson<dickinsm at gmail.com>:
>>> Thoughts?
>>
>> I tried to define the error messages in terms of the callee's
>> signature. I call the formals that are not variadic, keyword variadic,
>> or keyword-only, positional. For example, in
>>
>> def f(a, b, c, *args, d):
>>       pass
>>
>> a, b, and c are positional. Hence the "positional" in error messages.
>
> But since the error message gives the name of the parameter, there
> doesn't seem to be a point to add that it's "positional": it can be
> trivially deduced from the function signature.

Furthermore, since the parameter has a name, it can be given as a
keyword argument. Describing positional-or-keyword as "positional"
is misleading, although I admit that I often do that too. I think that
"positional or keyword argument" is too wordy, and is ambiguous as to
whether the argument can be given as either positional or keyword, or
we're unsure which of the two it is.

"Named positional argument" is more accurate, but also too wordy, and
it relies on the reader knowing enough about Python's calling semantics
to infer that therefore it can be given as positional or keyword style.

Since this is way too complicated to encapsulate in a short error
message, I'm with Nick and Mark that "positional" should be dropped
unless the argument is positional-only.



-- 
Steven

From steve at pearwood.info  Thu Sep 20 21:09:19 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 21 Sep 2012 05:09:19 +1000
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CAHVvXxQH8F56W--iM=jpw4pUHvcascoJ0oQrvX6KnpCpLNw0Lg@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
	<CAAu3qLXYvS1BoTNhnO1Ft4B1KnzYEUC_6TiiFcxr2NYiVZZryg@mail.gmail.com>
	<CAPZV6o-j5Z4h2EmVtNVKK=OJBNBtmCG-uSiWaSHfrv58+K1gTA@mail.gmail.com>
	<CAHVvXxQH8F56W--iM=jpw4pUHvcascoJ0oQrvX6KnpCpLNw0Lg@mail.gmail.com>
Message-ID: <505B69DF.70400@pearwood.info>

On 21/09/12 01:53, Oscar Benjamin wrote:

> Mark Dickinson wrote:
>>>>> def f(x): pass
>> ...
>>>>> f()
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in<module>
>> TypeError: f() missing 1 required positional argument: 'x'
>
> I would say that the only problem with this terminology is that it would be
> good to think of a word to replace "keyword-only" (positionless?).

I disagree completely. I think keyword-only is the right terminology to
use for arguments which can only be passed by keyword. It is *positional*
that is questionable, since named positional arguments can be given by
keyword.

I would like to see error messages reserve the terms:

1) "positional" for explicitly positional-only parameters;
2) "keyword" for explicitly keyword-only parameters;

(I don't mind whether or not they use "-only" as a suffix)

For normal, named-positional-or-keyword arguments, just use an unqualified
"argument".



-- 
Steven

From chris.jerdonek at gmail.com  Thu Sep 20 21:27:51 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Thu, 20 Sep 2012 12:27:51 -0700
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CAOTb1wdebpyVSa-hgJ=QtS1tsf6uFGZr6fk=_6M6JjqNHoidgw@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
	<CAP7+vJKD80Fs2CsFD9iqjcYWKUyAbAz8O2wQprZ=bLCehUpUgw@mail.gmail.com>
	<CAOTb1wdebpyVSa-hgJ=QtS1tsf6uFGZr6fk=_6M6JjqNHoidgw@mail.gmail.com>
Message-ID: <CAOTb1wdMxq-iw+3mnXUwiq85+TCso+seGrHuneCi0mfbnd_8yg@mail.gmail.com>

On Thu, Sep 20, 2012 at 10:18 AM, Chris Jerdonek
<chris.jerdonek at gmail.com> wrote:
> On Thu, Sep 20, 2012 at 8:52 AM, Guido van Rossum <guido at python.org> wrote:
>> On Thu, Sep 20, 2012 at 7:12 AM, Benjamin Peterson <benjamin at python.org> wrote:
>>> I tried to define the error messages in terms of the callee's
>>> signature. I call the formals that are not variadic, keyword variadic,
>>> or keyword-only, positional. For example, in
>>
>> Maybe this is also a good time to start distinguishing between
>> arguments (what you pass, call syntax) and parameters (what the
>> function receives, function definition syntax)?
>
> The glossary is one place to start making this distinction.  It
> currently has entries for "argument," "positional argument," and
> "keyword argument" that could perhaps use a review from this
> discussion.  For example:
>
> http://docs.python.org/dev/glossary.html#term-positional-argument
>
> The entries currently blur the distinction between the calling and
> definition perspectives.  Ideally, the glossary definitions of these
> terms would match and be consistent with their usage in error
> messages.

I took the liberty to create an issue in the tracker to settle on and
document preferred terminology in the area of positional/keyword
arguments/parameters, etc.  The issue is here:

http://bugs.python.org/issue15990

--Chris

From ethan at stoneleaf.us  Thu Sep 20 21:45:25 2012
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 20 Sep 2012 12:45:25 -0700
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <505B654A.90006@pearwood.info>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>	<CADiSq7fERUAZNe17i=mKa7LzAyNq0yvuwTAN3FVJYG-gZHPqDw@mail.gmail.com>	<CAAu3qLXHOs8G4LtxJ6q=TE5NfQ-sRquom88CzCXmzhfVK+OcNA@mail.gmail.com>
	<505B654A.90006@pearwood.info>
Message-ID: <505B7255.2060707@stoneleaf.us>

Steven D'Aprano wrote:
> On 20/09/12 22:59, Mark Dickinson wrote:
>> On Thu, Sep 20, 2012 at 1:21 PM, Nick Coghlan<ncoghlan at gmail.com>  wrote:
>>> +1 for using the unqualified "argument" in these error messages to
>>> mean "positional or keyword argument" (inspect.Parameter spells it out
>>> as POSITIONAL_OR_KEYWORD, but the full phrase is far too verbose for
>>> an error message).
>>
>> Ah yes;  I see that 'positional or keyword' is a more accurate term
>> (but agree it's unwieldy for an error message).  I also see that I was
>> naive to think that the 'fix' is as simple as dropping the word
>> 'positional':
>>
>>      >>>  def f(a, *, b):
>>      ...     pass
>>      ...
>>      >>>  f()
>>      Traceback (most recent call last):
>>        File "<stdin>", line 1, in<module>
>>      TypeError: f() missing 1 required positional argument: 'a'
>>
>> If the word 'positional' were dropped here, it would give the
>> incorrect impression that f only requires one argument.
> 
> I don't expect error messages to give a complete catalog of every
> problem with a specific function call. If f() reports that required
> argument 'a' is missing, that does not imply that no other required
> arguments are also missing. I think it is perfectly acceptable to
> not report the missing 'b' until the missing 'a' is resolved.

I disagree.  There is no reason (that I'm aware of ;) that the missing 
'b' cannot be noticed and reported at the same time as the missing 'a'.


> But I do expect error messages to be accurate. +1 to remove the
> word "positional" from the message.

And then it's still not accurate as 'b' is also a required argument that 
is missing.  Unless and until all error messages adopt your proposed 
'positional argument', 'argument', 'keyword argument' *and* describe 
_all_ the problems with the call confusion will reign supreme.

So, ideally, the above example would be:

      >>>  def f(a, *, b):
      ...     pass
      ...
      >>>  f()
      Traceback (most recent call last):
        File "<stdin>", line 1, in<module>
      TypeError: f() missing 2 required arguments: positional: 'a', 
keyword: 'b'

~Ethan~

P.S.
Also, a big thank-you -- the error messages *are* getting better all the 
time!

From ethan at stoneleaf.us  Thu Sep 20 21:30:05 2012
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 20 Sep 2012 12:30:05 -0700
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <505B69DF.70400@pearwood.info>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>	<CAAu3qLXYvS1BoTNhnO1Ft4B1KnzYEUC_6TiiFcxr2NYiVZZryg@mail.gmail.com>	<CAPZV6o-j5Z4h2EmVtNVKK=OJBNBtmCG-uSiWaSHfrv58+K1gTA@mail.gmail.com>	<CAHVvXxQH8F56W--iM=jpw4pUHvcascoJ0oQrvX6KnpCpLNw0Lg@mail.gmail.com>
	<505B69DF.70400@pearwood.info>
Message-ID: <505B6EBD.8080105@stoneleaf.us>

Steven D'Aprano wrote:
> I would like to see error messages reserve the terms:
> 
> 1) "positional" for explicitly positional-only parameters;
> 2) "keyword" for explicitly keyword-only parameters;

+1

From steve at pearwood.info  Thu Sep 20 22:21:54 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Fri, 21 Sep 2012 06:21:54 +1000
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <505B7255.2060707@stoneleaf.us>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>	<CADiSq7fERUAZNe17i=mKa7LzAyNq0yvuwTAN3FVJYG-gZHPqDw@mail.gmail.com>	<CAAu3qLXHOs8G4LtxJ6q=TE5NfQ-sRquom88CzCXmzhfVK+OcNA@mail.gmail.com>
	<505B654A.90006@pearwood.info> <505B7255.2060707@stoneleaf.us>
Message-ID: <505B7AE2.3000103@pearwood.info>

On 21/09/12 05:45, Ethan Furman wrote:

>> I don't expect error messages to give a complete catalog of every
>> problem with a specific function call. If f() reports that required
>> argument 'a' is missing, that does not imply that no other required
>> arguments are also missing. I think it is perfectly acceptable to
>> not report the missing 'b' until the missing 'a' is resolved.
>
> I disagree. There is no reason (that I'm aware of ;) that the missing
>'b' cannot be noticed and reported at the same time as the missing 'a'.

Listing every missing argument does not scale well as the number of
arguments increases.

def f(spam, ham, cheese, aardvark, halibut, *,  shrubbery, parrot, wafer_thin_mint):
     pass

f()

I would be -0 on an error message like:

TypeError: f() needs arguments 'spam', 'ham', 'cheese', 'aardvark', 'halibut' and keyword-only arguments 'shrubbery', 'parrot', 'wafer_thin_mint'

but wouldn't strongly object. I think it is acceptable (although not
ideal) if calling f() only reported the first missing argument it
noticed.

But I do think that we should not make any language guarantees about
error messages being "complete" or not.


-- 
Steven

From ncoghlan at gmail.com  Fri Sep 21 00:01:06 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 21 Sep 2012 08:01:06 +1000
Subject: [Python-Dev] TypeError: f() missing 1 required positional
 argument: 'x'
In-Reply-To: <CAOTb1wdMxq-iw+3mnXUwiq85+TCso+seGrHuneCi0mfbnd_8yg@mail.gmail.com>
References: <CAAu3qLUDriU=t+pOX9WWW5JYego1xz=0izBoh-Kg5tMivPu-Mw@mail.gmail.com>
	<CAPZV6o85m=5qKoD-v4xZASSu4gKbTT1BdaVBwvazFDSw+huiWQ@mail.gmail.com>
	<CAP7+vJKD80Fs2CsFD9iqjcYWKUyAbAz8O2wQprZ=bLCehUpUgw@mail.gmail.com>
	<CAOTb1wdebpyVSa-hgJ=QtS1tsf6uFGZr6fk=_6M6JjqNHoidgw@mail.gmail.com>
	<CAOTb1wdMxq-iw+3mnXUwiq85+TCso+seGrHuneCi0mfbnd_8yg@mail.gmail.com>
Message-ID: <CADiSq7dSDhpdsjgLLJ24K=2k-1SLwbh4kyigeqgALkwgUXWfow@mail.gmail.com>

We've already had this terminology discussion and documented the results in
PEP 362. The rest of the docs may require updates to be brought in line
with that.

Cheers,
Nick.

--
Sent from my phone, thus the relative brevity :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120921/4427d080/attachment.html>

From glyph at twistedmatrix.com  Fri Sep 21 01:16:03 2012
From: glyph at twistedmatrix.com (Glyph)
Date: Thu, 20 Sep 2012 16:16:03 -0700
Subject: [Python-Dev] Memoryviews should expose the underlying memory
	address
In-Reply-To: <AA89697C-F3F5-43D6-A017-AF909AAC0031@dabeaz.com>
References: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
	<CAPZV6o8QnekwCLWeSh4fJXMvE2qXVcEA2LZS3TGMFX+-dYhk=w@mail.gmail.com>
	<A5245A71-D85F-4763-B82F-7BB076DC0A0F@dabeaz.com>
	<CAPZV6o9GZsum5CuCy-WUwuBXuxWuavU0MPyZY131=mEwFjBirA@mail.gmail.com>
	<AA89697C-F3F5-43D6-A017-AF909AAC0031@dabeaz.com>
Message-ID: <6EAE4F99-A1F3-4654-B88E-32D9417C86CF@twistedmatrix.com>

Le Sep 20, 2012 ? 11:35 AM, David Beazley <dave at dabeaz.com> a ?crit :

> Well, if it's supposed to do that, it certainly doesn't work for me in 3.3.  I get a type error about it wanting a ctypes pointer object.    Even if this worked, it still doesn't address the need to get the pointer value possibly for some other purpose such as handling it off to a bunch of code generated via LLVM. 

It seems like there's no reason to need to get the pointer value out as a Python integer.  If you are trying to get a pointer from a memoryview into some C code, or into some LLVM generated code, you still need to do the Python int object ? C integer-of-some-kind ? C pointer type conversion.  Better to just go straight from Python memoryview object ? C pointer in one supported API call.  Isn't this what the y* w* s* format codes are for?

Every time I have something that's a big number and I need to turn it into a pointer, I have to stare at the table in <http://en.wikipedia.org/wiki/64_bit#64-bit_data_models> for like 30 seconds.  I'd rather have some Python API do the staring for me.  David, I realize that table is probably permanently visible in the heads-up display that your cybernetic implants afford you, but some of us need to make our way through C code with humbler faculties ;-).

-g


From dave at dabeaz.com  Fri Sep 21 01:37:21 2012
From: dave at dabeaz.com (David Beazley)
Date: Thu, 20 Sep 2012 18:37:21 -0500
Subject: [Python-Dev] Memoryviews should expose the underlying memory
	address
In-Reply-To: <6EAE4F99-A1F3-4654-B88E-32D9417C86CF@twistedmatrix.com>
References: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
	<CAPZV6o8QnekwCLWeSh4fJXMvE2qXVcEA2LZS3TGMFX+-dYhk=w@mail.gmail.com>
	<A5245A71-D85F-4763-B82F-7BB076DC0A0F@dabeaz.com>
	<CAPZV6o9GZsum5CuCy-WUwuBXuxWuavU0MPyZY131=mEwFjBirA@mail.gmail.com>
	<AA89697C-F3F5-43D6-A017-AF909AAC0031@dabeaz.com>
	<6EAE4F99-A1F3-4654-B88E-32D9417C86CF@twistedmatrix.com>
Message-ID: <8B294F19-DDF9-4168-8114-A6CE52C7C880@dabeaz.com>

A memory address is a number.   I think an integer is fine--if you're working at this level, you're already on your own and expected to know what you're doing.  I'd prefer to just get the raw address without yet another level of indirection.

Other parts of the library already do this.   For instance array.buffer_info().

Cheers
Dave

Sent from cell 

On Sep 20, 2012, at 6:16 PM, Glyph <glyph at twistedmatrix.com> wrote:

> Le Sep 20, 2012 ? 11:35 AM, David Beazley <dave at dabeaz.com> a ?crit :
> 
>> Well, if it's supposed to do that, it certainly doesn't work for me in 3.3.  I get a type error about it wanting a ctypes pointer object.    Even if this worked, it still doesn't address the need to get the pointer value possibly for some other purpose such as handling it off to a bunch of code generated via LLVM. 
> 
> It seems like there's no reason to need to get the pointer value out as a Python integer.  If you are trying to get a pointer from a memoryview into some C code, or into some LLVM generated code, you still need to do the Python int object ? C integer-of-some-kind ? C pointer type conversion.  Better to just go straight from Python memoryview object ? C pointer in one supported API call.  Isn't this what the y* w* s* format codes are for?
> 
> Every time I have something that's a big number and I need to turn it into a pointer, I have to stare at the table in <http://en.wikipedia.org/wiki/64_bit#64-bit_data_models> for like 30 seconds.  I'd rather have some Python API do the staring for me.  David, I realize that table is probably permanently visible in the heads-up display that your cybernetic implants afford you, but some of us need to make our way through C code with humbler faculties ;-).
> 
> -g
> 

From ncoghlan at gmail.com  Fri Sep 21 05:19:06 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 21 Sep 2012 13:19:06 +1000
Subject: [Python-Dev] Memoryviews should expose the underlying memory
	address
In-Reply-To: <8B294F19-DDF9-4168-8114-A6CE52C7C880@dabeaz.com>
References: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
	<CAPZV6o8QnekwCLWeSh4fJXMvE2qXVcEA2LZS3TGMFX+-dYhk=w@mail.gmail.com>
	<A5245A71-D85F-4763-B82F-7BB076DC0A0F@dabeaz.com>
	<CAPZV6o9GZsum5CuCy-WUwuBXuxWuavU0MPyZY131=mEwFjBirA@mail.gmail.com>
	<AA89697C-F3F5-43D6-A017-AF909AAC0031@dabeaz.com>
	<6EAE4F99-A1F3-4654-B88E-32D9417C86CF@twistedmatrix.com>
	<8B294F19-DDF9-4168-8114-A6CE52C7C880@dabeaz.com>
Message-ID: <CADiSq7c3ES0BXqmQfcw5hDq1L4W_YzzbknHMkNgbHsH8q8DUJw@mail.gmail.com>

On Fri, Sep 21, 2012 at 9:37 AM, David Beazley <dave at dabeaz.com> wrote:
> A memory address is a number.   I think an integer is fine--if you're working at this level, you're already on your own and expected to know what you're doing.  I'd prefer to just get the raw address without yet another level of indirection.
>
> Other parts of the library already do this.   For instance array.buffer_info().

I'm fine with exposing a memoryview.buffer_address attribute in 3.4.
The idea had never come up before, as the idea of using *Python code*
(rather than C) to provide the shim between a PEP 3118 exporter and a
consumer that doesn't understand that API isn't a use case we had even
considered. memoryview has instead been more focused on *interpreting*
the contents of exported buffers as ordinary Python objects.

(We already know we still need to define an API to let classes defined
in Python implement the buffer API, though)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From greg.ewing at canterbury.ac.nz  Fri Sep 21 08:12:31 2012
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 21 Sep 2012 18:12:31 +1200
Subject: [Python-Dev] Memoryviews should expose the underlying memory
 address
In-Reply-To: <CADiSq7c3ES0BXqmQfcw5hDq1L4W_YzzbknHMkNgbHsH8q8DUJw@mail.gmail.com>
References: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
	<CAPZV6o8QnekwCLWeSh4fJXMvE2qXVcEA2LZS3TGMFX+-dYhk=w@mail.gmail.com>
	<A5245A71-D85F-4763-B82F-7BB076DC0A0F@dabeaz.com>
	<CAPZV6o9GZsum5CuCy-WUwuBXuxWuavU0MPyZY131=mEwFjBirA@mail.gmail.com>
	<AA89697C-F3F5-43D6-A017-AF909AAC0031@dabeaz.com>
	<6EAE4F99-A1F3-4654-B88E-32D9417C86CF@twistedmatrix.com>
	<8B294F19-DDF9-4168-8114-A6CE52C7C880@dabeaz.com>
	<CADiSq7c3ES0BXqmQfcw5hDq1L4W_YzzbknHMkNgbHsH8q8DUJw@mail.gmail.com>
Message-ID: <505C054F.8020802@canterbury.ac.nz>

Nick Coghlan wrote:
> I'm fine with exposing a memoryview.buffer_address attribute in 3.4.

What about objects whose buffer address can change
when the buffer isn't locked?

-- 
Greg

From ncoghlan at gmail.com  Fri Sep 21 09:08:28 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 21 Sep 2012 17:08:28 +1000
Subject: [Python-Dev] Memoryviews should expose the underlying memory
	address
In-Reply-To: <505C054F.8020802@canterbury.ac.nz>
References: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
	<CAPZV6o8QnekwCLWeSh4fJXMvE2qXVcEA2LZS3TGMFX+-dYhk=w@mail.gmail.com>
	<A5245A71-D85F-4763-B82F-7BB076DC0A0F@dabeaz.com>
	<CAPZV6o9GZsum5CuCy-WUwuBXuxWuavU0MPyZY131=mEwFjBirA@mail.gmail.com>
	<AA89697C-F3F5-43D6-A017-AF909AAC0031@dabeaz.com>
	<6EAE4F99-A1F3-4654-B88E-32D9417C86CF@twistedmatrix.com>
	<8B294F19-DDF9-4168-8114-A6CE52C7C880@dabeaz.com>
	<CADiSq7c3ES0BXqmQfcw5hDq1L4W_YzzbknHMkNgbHsH8q8DUJw@mail.gmail.com>
	<505C054F.8020802@canterbury.ac.nz>
Message-ID: <CADiSq7fqveygbRGdkqAvq9pW6i-i1QiCAdxNk+5g2_inmWTjqA@mail.gmail.com>

On Fri, Sep 21, 2012 at 4:12 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Nick Coghlan wrote:
>>
>> I'm fine with exposing a memoryview.buffer_address attribute in 3.4.
>
>
> What about objects whose buffer address can change
> when the buffer isn't locked?

Managing the lifecycle issues will be up to the application. If they
let the memoryview object go away, then the buffer it references may
also go away. This isn't any different from the situation with
array.buffer_info() - the address from that is only valid as long as
the array object itself is still around.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From fijall at gmail.com  Fri Sep 21 11:45:18 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 21 Sep 2012 11:45:18 +0200
Subject: [Python-Dev] Memoryviews should expose the underlying memory
	address
In-Reply-To: <6EAE4F99-A1F3-4654-B88E-32D9417C86CF@twistedmatrix.com>
References: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
	<CAPZV6o8QnekwCLWeSh4fJXMvE2qXVcEA2LZS3TGMFX+-dYhk=w@mail.gmail.com>
	<A5245A71-D85F-4763-B82F-7BB076DC0A0F@dabeaz.com>
	<CAPZV6o9GZsum5CuCy-WUwuBXuxWuavU0MPyZY131=mEwFjBirA@mail.gmail.com>
	<AA89697C-F3F5-43D6-A017-AF909AAC0031@dabeaz.com>
	<6EAE4F99-A1F3-4654-B88E-32D9417C86CF@twistedmatrix.com>
Message-ID: <CAK5idxQgf4RK-wD407kssCbjY5=_AbnhaoxtkY8afhr-PFovkQ@mail.gmail.com>

On Fri, Sep 21, 2012 at 1:16 AM, Glyph <glyph at twistedmatrix.com> wrote:
> Le Sep 20, 2012 ? 11:35 AM, David Beazley <dave at dabeaz.com> a ?crit :
>
>> Well, if it's supposed to do that, it certainly doesn't work for me in 3.3.  I get a type error about it wanting a ctypes pointer object.    Even if this worked, it still doesn't address the need to get the pointer value possibly for some other purpose such as handling it off to a bunch of code generated via LLVM.
>
> It seems like there's no reason to need to get the pointer value out as a Python integer.  If you are trying to get a pointer from a memoryview into some C code, or into some LLVM generated code, you still need to do the Python int object ? C integer-of-some-kind ? C pointer type conversion.  Better to just go straight from Python memoryview object ? C pointer in one supported API call.  Isn't this what the y* w* s* format codes are for?
>
> Every time I have something that's a big number and I need to turn it into a pointer, I have to stare at the table in <http://en.wikipedia.org/wiki/64_bit#64-bit_data_models> for like 30 seconds.  I'd rather have some Python API do the staring for me.  David, I realize that table is probably permanently visible in the heads-up display that your cybernetic implants afford you, but some of us need to make our way through C code with humbler faculties ;-).
>
> -g

This is also kind of a problem with PyPy and CFFI, where we actively
discourage people from using C. Passing address as an int sounds like
a very reasonable solution.

Cheers,
fijal

From dave at dabeaz.com  Fri Sep 21 14:07:09 2012
From: dave at dabeaz.com (David Beazley)
Date: Fri, 21 Sep 2012 07:07:09 -0500
Subject: [Python-Dev] Memoryviews should expose the underlying memory
	address
In-Reply-To: <CAK5idxQgf4RK-wD407kssCbjY5=_AbnhaoxtkY8afhr-PFovkQ@mail.gmail.com>
References: <10F9C059-8EC1-4C5F-9C24-6CF646CCCF08@dabeaz.com>
	<CAPZV6o8QnekwCLWeSh4fJXMvE2qXVcEA2LZS3TGMFX+-dYhk=w@mail.gmail.com>
	<A5245A71-D85F-4763-B82F-7BB076DC0A0F@dabeaz.com>
	<CAPZV6o9GZsum5CuCy-WUwuBXuxWuavU0MPyZY131=mEwFjBirA@mail.gmail.com>
	<AA89697C-F3F5-43D6-A017-AF909AAC0031@dabeaz.com>
	<6EAE4F99-A1F3-4654-B88E-32D9417C86CF@twistedmatrix.com>
	<CAK5idxQgf4RK-wD407kssCbjY5=_AbnhaoxtkY8afhr-PFovkQ@mail.gmail.com>
Message-ID: <715A4A9D-0A18-431A-AD0B-D2A89E29B1D3@dabeaz.com>


On Sep 21, 2012, at 4:45 AM, Maciej Fijalkowski wrote:
> 
> This is also kind of a problem with PyPy and CFFI, where we actively
> discourage people from using C. Passing address as an int sounds like
> a very reasonable solution.
> 

I just wanted to add that getting the address as an integer is useful because one might actually want to do math with it.   Since memoryviews also expose the shape, itemsize, and other information, it's conceivable that one might combine these with the base address to compute locations within the array.  Example: take a memoryview, slice up the buffer into partitions and hand them off to worker functions running in a thread pool.

Cheers,
Dave


From brett at python.org  Fri Sep 21 14:37:47 2012
From: brett at python.org (Brett Cannon)
Date: Fri, 21 Sep 2012 08:37:47 -0400
Subject: [Python-Dev] [Python-checkins] peps: add stub for wheel spec
In-Reply-To: <3XNCfp3DGCzQnp@mail.python.org>
References: <3XNCfp3DGCzQnp@mail.python.org>
Message-ID: <CAP1=2W4rXrw7-Wtq7CyKjbz74MLN+EjXpB6ukrJJsWER4-kATg@mail.gmail.com>

We don't put stubs of PEPs into the repository, especially when they have
not been vetted on python-ideas or some other public mailing list that is
appropriate for the subject.

You also mis-capitalized PyPy in this commit.

On Thu, Sep 20, 2012 at 6:44 PM, daniel.holth <python-checkins at python.org>wrote:

> http://hg.python.org/peps/rev/4ac055623f2a
> changeset:   4514:4ac055623f2a
> user:        Daniel Holth <dholth at fastmail.fm>
> date:        Thu Sep 20 18:43:53 2012 -0400
> summary:
>   add stub for wheel spec
>
> files:
>   pep-0425.txt |   7 +++----
>   pep-0427.txt |  30 ++++++++++++++++++++++++++++++
>   2 files changed, 33 insertions(+), 4 deletions(-)
>
>
> diff --git a/pep-0425.txt b/pep-0425.txt
> --- a/pep-0425.txt
> +++ b/pep-0425.txt
> @@ -81,10 +81,9 @@
>
>  Other Python implementations should use `sys.implementation.name`.
>
> -The version is `py_version_nodot`.  CPython gets away with no dot, but
> -if one is needed the underscore `_` is used instead.  Pypy uses versions
> that
> -do not track the Python language version and should probably use its own
> -versions here `pp18`, `pp19`.
> +The version is `py_version_nodot`.  CPython gets away with no dot,
> +but if one is needed the underscore `_` is used instead.  Pypy should
> +probably use its own versions here `pp18`, `pp19`.
>
>  The version can be just the major version `2` or `3` `py2`, `py3` for
>  many pure-Python distributions.
> diff --git a/pep-0427.txt b/pep-0427.txt
> new file mode 100644
> --- /dev/null
> +++ b/pep-0427.txt
> @@ -0,0 +1,30 @@
> +PEP: 426
> +Title: The Wheel Binary Package Format 1.0
> +Version: $Revision$
> +Last-Modified: $Date$
> +Author: Daniel Holth <dholth at fastmail.fm>
> +Discussions-To: Distutils SIG
> +Status: Draft
> +Type: Standards Track
> +Content-Type: text/x-rst
> +Created: 20 Sep 2012
> +
> +
> +Abstract
> +========
> +
> +This PEP describes a binary format for Python called wheel.
> +
> +Copyright
> +=========
> +
> +This document has been placed in the public domain.
> +
> +
> +..
> +   Local Variables:
> +   mode: indented-text
> +   indent-tabs-mode: nil
> +   sentence-end-double-space: t
> +   fill-column: 70
> +   End:
>
> --
> Repository URL: http://hg.python.org/peps
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120921/5db6e76d/attachment.html>

From status at bugs.python.org  Fri Sep 21 18:07:08 2012
From: status at bugs.python.org (Python tracker)
Date: Fri, 21 Sep 2012 18:07:08 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20120921160708.96CF61CC78@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2012-09-14 - 2012-09-21)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3731 (+30)
  closed 24045 (+25)
  total  27776 (+55)

Open issues with patches: 1650 


Issues opened (44)
==================

#15945: memoryview + bytes fails
http://bugs.python.org/issue15945  opened by exarkun

#15947: Assigning new values to instance of pointer types does not che
http://bugs.python.org/issue15947  opened by facundobatista

#15948: Unchecked return value of I/O functions
http://bugs.python.org/issue15948  opened by christian.heimes

#15949: docs.python.org not getting updated
http://bugs.python.org/issue15949  opened by cjerdonek

#15951: string.Formatter returns str for empty unicode template
http://bugs.python.org/issue15951  opened by Aleksey.Sivokon

#15952: format(value) and value.__format__() behave differently with u
http://bugs.python.org/issue15952  opened by cjerdonek

#15953: Incorrect some fields declaration in the PyTypeObject document
http://bugs.python.org/issue15953  opened by storchaka

#15954: No error checking after using of the wcsxfrm()
http://bugs.python.org/issue15954  opened by storchaka

#15955: gzip, bz2, lzma: add method to get decompressed size
http://bugs.python.org/issue15955  opened by christian.heimes

#15956: backreference to named group does not work
http://bugs.python.org/issue15956  opened by steve.newcomb

#15957: README.txt points to broken "contributing" url in python wiki
http://bugs.python.org/issue15957  opened by jcd

#15958: bytes.join() should allow arbitrary buffer objects
http://bugs.python.org/issue15958  opened by pitrou

#15961: Missing return value in ``system_message``
http://bugs.python.org/issue15961  opened by malthe

#15962: Windows STDIN/STDOUT Redirection is actually FIXED
http://bugs.python.org/issue15962  opened by Nacsa.Krist??f

#15963: Improve ./configure's support for 32/64-bit debug|release|prof
http://bugs.python.org/issue15963  opened by trent

#15964: SyntaxError in asdl when building 2.7 with system Python 3
http://bugs.python.org/issue15964  opened by cjerdonek

#15966: concurrent.futures: Executor.submit keyword arguments may not 
http://bugs.python.org/issue15966  opened by mark.dickinson

#15967: Slaves don't seem to clean up their /tmp mess if a step fails.
http://bugs.python.org/issue15967  opened by trent

#15968: Incorporate Tcl/Tk/Tix into the Windows build process
http://bugs.python.org/issue15968  opened by jkloth

#15969: faulthandler: drop S from dump_tracebacks_later() function nam
http://bugs.python.org/issue15969  opened by haypo

#15971: Sporadic failure in test_dump_tracebacks_later_file (test_faul
http://bugs.python.org/issue15971  opened by ezio.melotti

#15972: wrong error message for os.path.getsize
http://bugs.python.org/issue15972  opened by jftuga

#15973: Segmentation fault on timezone comparison
http://bugs.python.org/issue15973  opened by lanhel

#15974: Optional compact and colored output for regrest
http://bugs.python.org/issue15974  opened by brett.cannon

#15976: Inconsistent behavior of search_for_exec_prefix() results in s
http://bugs.python.org/issue15976  opened by foutrelis

#15977: Memory leak in _ssl.c
http://bugs.python.org/issue15977  opened by Daniel.Sommermann

#15978: asyncore: included batteries don't fit
http://bugs.python.org/issue15978  opened by chrysn

#15979: Improve timeit documentation
http://bugs.python.org/issue15979  opened by ezio.melotti

#15982: asyncore.dispatcher does not handle windows socket error code 
http://bugs.python.org/issue15982  opened by McNetic

#15983: multiprocessing JoinableQueue's join function with timeout
http://bugs.python.org/issue15983  opened by legordian

#15984: Wrong documentation for PyUnicode_FromObject()
http://bugs.python.org/issue15984  opened by storchaka

#15986: memoryview: expose 'buf' attribute
http://bugs.python.org/issue15986  opened by skrah

#15987: Provide a way to compare AST nodes for equality recursively
http://bugs.python.org/issue15987  opened by Julian

#15988: Inconsistency in overflow error messages of integer argument
http://bugs.python.org/issue15988  opened by storchaka

#15989: Possible integer overflow of PyLong_AsLong() results
http://bugs.python.org/issue15989  opened by storchaka

#15990: solidify argument/parameter terminology
http://bugs.python.org/issue15990  opened by cjerdonek

#15991: BaseHTTPServer with ThreadingMixIn serving wrong data sometime
http://bugs.python.org/issue15991  opened by theamk

#15992: Strict aliasing violations in Objects/unicodeobject.c
http://bugs.python.org/issue15992  opened by mark.dickinson

#15993: Windows: 3.3.0-rc2.msi: test_buffer fails
http://bugs.python.org/issue15993  opened by skrah

#15994: memoryview to freed memory can cause segfault
http://bugs.python.org/issue15994  opened by sbt

#15996: pow() for complex numbers is rough around the edges
http://bugs.python.org/issue15996  opened by mattip

#15997: NotImplemented needs to be documented
http://bugs.python.org/issue15997  opened by max

#15998: Python GUI idle not working
http://bugs.python.org/issue15998  opened by sreeni

#15999: Using new 'bool' format character
http://bugs.python.org/issue15999  opened by storchaka



Most recent 15 issues with no replies (15)
==========================================

#15994: memoryview to freed memory can cause segfault
http://bugs.python.org/issue15994

#15992: Strict aliasing violations in Objects/unicodeobject.c
http://bugs.python.org/issue15992

#15991: BaseHTTPServer with ThreadingMixIn serving wrong data sometime
http://bugs.python.org/issue15991

#15988: Inconsistency in overflow error messages of integer argument
http://bugs.python.org/issue15988

#15987: Provide a way to compare AST nodes for equality recursively
http://bugs.python.org/issue15987

#15986: memoryview: expose 'buf' attribute
http://bugs.python.org/issue15986

#15984: Wrong documentation for PyUnicode_FromObject()
http://bugs.python.org/issue15984

#15982: asyncore.dispatcher does not handle windows socket error code 
http://bugs.python.org/issue15982

#15976: Inconsistent behavior of search_for_exec_prefix() results in s
http://bugs.python.org/issue15976

#15969: faulthandler: drop S from dump_tracebacks_later() function nam
http://bugs.python.org/issue15969

#15968: Incorporate Tcl/Tk/Tix into the Windows build process
http://bugs.python.org/issue15968

#15962: Windows STDIN/STDOUT Redirection is actually FIXED
http://bugs.python.org/issue15962

#15958: bytes.join() should allow arbitrary buffer objects
http://bugs.python.org/issue15958

#15955: gzip, bz2, lzma: add method to get decompressed size
http://bugs.python.org/issue15955

#15953: Incorrect some fields declaration in the PyTypeObject document
http://bugs.python.org/issue15953



Most recent 15 issues waiting for review (15)
=============================================

#15999: Using new 'bool' format character
http://bugs.python.org/issue15999

#15983: multiprocessing JoinableQueue's join function with timeout
http://bugs.python.org/issue15983

#15977: Memory leak in _ssl.c
http://bugs.python.org/issue15977

#15973: Segmentation fault on timezone comparison
http://bugs.python.org/issue15973

#15972: wrong error message for os.path.getsize
http://bugs.python.org/issue15972

#15969: faulthandler: drop S from dump_tracebacks_later() function nam
http://bugs.python.org/issue15969

#15966: concurrent.futures: Executor.submit keyword arguments may not 
http://bugs.python.org/issue15966

#15961: Missing return value in ``system_message``
http://bugs.python.org/issue15961

#15957: README.txt points to broken "contributing" url in python wiki
http://bugs.python.org/issue15957

#15956: backreference to named group does not work
http://bugs.python.org/issue15956

#15954: No error checking after using of the wcsxfrm()
http://bugs.python.org/issue15954

#15953: Incorrect some fields declaration in the PyTypeObject document
http://bugs.python.org/issue15953

#15952: format(value) and value.__format__() behave differently with u
http://bugs.python.org/issue15952

#15951: string.Formatter returns str for empty unicode template
http://bugs.python.org/issue15951

#15939: make *.rst files in Doc/ parseable by doctest
http://bugs.python.org/issue15939



Top 10 most discussed issues (10)
=================================

#11454: email.message import time
http://bugs.python.org/issue11454  18 msgs

#15903: Make rawiobase_read() read directly to bytes object
http://bugs.python.org/issue15903  16 msgs

#15973: Segmentation fault on timezone comparison
http://bugs.python.org/issue15973  16 msgs

#14262: Allow using decimals as arguments to `timedelta`
http://bugs.python.org/issue14262  12 msgs

#15944: memoryviews and ctypes
http://bugs.python.org/issue15944  12 msgs

#11664: Add patch method to unittest.TestCase
http://bugs.python.org/issue11664  11 msgs

#15972: wrong error message for os.path.getsize
http://bugs.python.org/issue15972  10 msgs

#15276: unicode format does not really work in Python 2.x
http://bugs.python.org/issue15276   9 msgs

#15629: Add to regrtest the ability to run Lib and Doc doctests
http://bugs.python.org/issue15629   8 msgs

#15951: string.Formatter returns str for empty unicode template
http://bugs.python.org/issue15951   8 msgs



Issues closed (22)
==================

#14873: Windows devguide: clarification for build errors due to missin
http://bugs.python.org/issue14873  closed by ezio.melotti

#15039: module/ found before module.py when both are in the CWD
http://bugs.python.org/issue15039  closed by eric.smith

#15144: Possible integer overflow in operations with addresses and siz
http://bugs.python.org/issue15144  closed by pitrou

#15304: Wrong path in test.support.temp_cwd() error message
http://bugs.python.org/issue15304  closed by ezio.melotti

#15421: Calendar.itermonthdates OverflowError
http://bugs.python.org/issue15421  closed by ezio.melotti

#15548: Mention all new os functions in What's New in Python 3.3
http://bugs.python.org/issue15548  closed by haypo

#15789: mention shell-like parts of the std lib in the subprocess docs
http://bugs.python.org/issue15789  closed by ezio.melotti

#15796: Fix readline() docstrings
http://bugs.python.org/issue15796  closed by ezio.melotti

#15896: Sporadic EINVAL in nonblocking pipe os.read when forked child 
http://bugs.python.org/issue15896  closed by vitaly

#15920: make howto/regex.rst doctests pass
http://bugs.python.org/issue15920  closed by ezio.melotti

#15932: files in the csv documentation's examples are not closed
http://bugs.python.org/issue15932  closed by ezio.melotti

#15946: Windows 8 x64 - IO-Error
http://bugs.python.org/issue15946  closed by brian.curtin

#15950: open() should not accept bool argument
http://bugs.python.org/issue15950  closed by r.david.murray

#15959: Declaration mismatch of quick integer allocation counters
http://bugs.python.org/issue15959  closed by mark.dickinson

#15960: logging.shutdown should cope with IO errors from handler.relea
http://bugs.python.org/issue15960  closed by vinay.sajip

#15965: AT_FDCWD is 0xffd19553 on Solaris 10, resulting in compiler wa
http://bugs.python.org/issue15965  closed by trent

#15970: ElementTree HTML serialization incorrect for <meta>, <param>
http://bugs.python.org/issue15970  closed by ezio.melotti

#15975: Allow multiplying timedelta by Decimal
http://bugs.python.org/issue15975  closed by mark.dickinson

#15980: Non-escaped '\n' in docstrings
http://bugs.python.org/issue15980  closed by ezio.melotti

#15981: improve documentation of __hash__
http://bugs.python.org/issue15981  closed by r.david.murray

#15995: Windows: 3.3.0-rc2.msi: test_lzma fails
http://bugs.python.org/issue15995  closed by skrah

#15985: round() has wrong argument names
http://bugs.python.org/issue15985  closed by mark.dickinson

From yselivanov.ml at gmail.com  Fri Sep 21 21:10:22 2012
From: yselivanov.ml at gmail.com (Yury Selivanov)
Date: Fri, 21 Sep 2012 15:10:22 -0400
Subject: [Python-Dev] 3.3 release
Message-ID: <FB01BDB3-B5B2-4817-B7E2-83AD17BB1C91@gmail.com>

Hello,

What's the status of 3.3?  Is it going to be released tomorrow?

(I see that we still have a few release-blockers in the tracker,
and the whatsnew section is probably due to be updated by Raymond...)

-
Yury

From g.brandl at gmx.net  Sat Sep 22 13:18:03 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 22 Sep 2012 13:18:03 +0200
Subject: [Python-Dev] 3.3 release
In-Reply-To: <FB01BDB3-B5B2-4817-B7E2-83AD17BB1C91@gmail.com>
References: <FB01BDB3-B5B2-4817-B7E2-83AD17BB1C91@gmail.com>
Message-ID: <k3k6p7$9uk$1@ger.gmane.org>

On 09/21/2012 09:10 PM, Yury Selivanov wrote:
> Hello,
> 
> What's the status of 3.3?  Is it going to be released tomorrow?
> 
> (I see that we still have a few release-blockers in the tracker,
> and the whatsnew section is probably due to be updated by Raymond...)

There will be a candidate 3 this weekend, with the final scheduled
for next weekend.

The whatsnew update is not immediately blocking the release, since
most people read the docs online anyway, where they are updated daily.

Georg


From georg at python.org  Mon Sep 24 08:18:47 2012
From: georg at python.org (Georg Brandl)
Date: Mon, 24 Sep 2012 08:18:47 +0200
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
Message-ID: <505FFB47.30901@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team, I'm delighted to announce the
third release candidate of Python 3.3.0.

This is a preview release, and its use is not recommended in
production settings.

Python 3.3 includes a range of improvements of the 3.x series, as well
as easier porting between 2.x and 3.x.  Major new features and changes
in the 3.3 release series are:

* PEP 380, syntax for delegating to a subgenerator ("yield from")
* PEP 393, flexible string representation (doing away with the
    distinction between "wide" and "narrow" Unicode builds)
* A C implementation of the "decimal" module, with up to 80x speedup
    for decimal-heavy applications
* The import system (__import__) now based on importlib by default
* The new "lzma" module with LZMA/XZ support
* PEP 397, a Python launcher for Windows
* PEP 405, virtual environment support in core
* PEP 420, namespace package support
* PEP 3151, reworking the OS and IO exception hierarchy
* PEP 3155, qualified name for classes and functions
* PEP 409, suppressing exception context
* PEP 414, explicit Unicode literals to help with porting
* PEP 418, extended platform-independent clocks in the "time" module
* PEP 412, a new key-sharing dictionary implementation that
    significantly saves memory for object-oriented code
* PEP 362, the function-signature object
* The new "faulthandler" module that helps diagnosing crashes
* The new "unittest.mock" module
* The new "ipaddress" module
* The "sys.implementation" attribute
* A policy framework for the email package, with a provisional (see
    PEP 411) policy that adds much improved unicode support for email
    header parsing
* A "collections.ChainMap" class for linking mappings to a single unit
* Wrappers for many more POSIX functions in the "os" and "signal"
    modules, as well as other useful functions such as "sendfile()"
* Hash randomization, introduced in earlier bugfix releases, is now
    switched on by default

In total, almost 500 API items are new or improved in Python 3.3.
For a more extensive list of changes in 3.3.0, see

      http://docs.python.org/3.3/whatsnew/3.3.html

To download Python 3.3.0 visit:

      http://www.python.org/download/releases/3.3.0/

Please consider trying Python 3.3.0 with your code and reporting any bugs
you may notice to:

      http://bugs.python.org/


Enjoy!

- -- 
Georg Brandl, Release Manager
georg at python.org
(on behalf of the entire python-dev team and 3.3's contributors)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iEYEARECAAYFAlBf+0cACgkQN9GcIYhpnLBqfgCglbN63XUr2m4Ya4ff8Hza1Axl
SgMAniQZRJi8uYfeqltf5/G4QV/+SdWT
=KXTo
-----END PGP SIGNATURE-----

From breamoreboy at yahoo.co.uk  Mon Sep 24 10:26:25 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 24 Sep 2012 09:26:25 +0100
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <505FFB47.30901@python.org>
References: <505FFB47.30901@python.org>
Message-ID: <k3p5cc$uoh$1@ger.gmane.org>

On 24/09/2012 07:18, Georg Brandl wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On behalf of the Python development team, I'm delighted to announce the
> third release candidate of Python 3.3.0.
>
> This is a preview release, and its use is not recommended in
> production settings.
>
> Python 3.3 includes a range of improvements of the 3.x series, as well
> as easier porting between 2.x and 3.x.  Major new features and changes
> in the 3.3 release series are:
>
> * PEP 380, syntax for delegating to a subgenerator ("yield from")
> * PEP 393, flexible string representation (doing away with the
>      distinction between "wide" and "narrow" Unicode builds)
> * A C implementation of the "decimal" module, with up to 80x speedup
>      for decimal-heavy applications
> * The import system (__import__) now based on importlib by default
> * The new "lzma" module with LZMA/XZ support
> * PEP 397, a Python launcher for Windows
> * PEP 405, virtual environment support in core
> * PEP 420, namespace package support
> * PEP 3151, reworking the OS and IO exception hierarchy
> * PEP 3155, qualified name for classes and functions
> * PEP 409, suppressing exception context
> * PEP 414, explicit Unicode literals to help with porting
> * PEP 418, extended platform-independent clocks in the "time" module
> * PEP 412, a new key-sharing dictionary implementation that
>      significantly saves memory for object-oriented code
> * PEP 362, the function-signature object
> * The new "faulthandler" module that helps diagnosing crashes
> * The new "unittest.mock" module
> * The new "ipaddress" module
> * The "sys.implementation" attribute
> * A policy framework for the email package, with a provisional (see
>      PEP 411) policy that adds much improved unicode support for email
>      header parsing
> * A "collections.ChainMap" class for linking mappings to a single unit
> * Wrappers for many more POSIX functions in the "os" and "signal"
>      modules, as well as other useful functions such as "sendfile()"
> * Hash randomization, introduced in earlier bugfix releases, is now
>      switched on by default
>
> In total, almost 500 API items are new or improved in Python 3.3.
> For a more extensive list of changes in 3.3.0, see
>
>        http://docs.python.org/3.3/whatsnew/3.3.html
>
> To download Python 3.3.0 visit:
>
>        http://www.python.org/download/releases/3.3.0/
>
> Please consider trying Python 3.3.0 with your code and reporting any bugs
> you may notice to:
>
>        http://bugs.python.org/
>
>
> Enjoy!
>
> - --
> Georg Brandl, Release Manager
> georg at python.org
> (on behalf of the entire python-dev team and 3.3's contributors)
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.19 (GNU/Linux)
>
> iEYEARECAAYFAlBf+0cACgkQN9GcIYhpnLBqfgCglbN63XUr2m4Ya4ff8Hza1Axl
> SgMAniQZRJi8uYfeqltf5/G4QV/+SdWT
> =KXTo
> -----END PGP SIGNATURE-----
>

Yes, but apart from all that, what have the python devs ever done for 
us?  Nothing :)

-- 
Cheers.

Mark Lawrence.


From ethan at stoneleaf.us  Mon Sep 24 15:13:09 2012
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 24 Sep 2012 06:13:09 -0700
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <k3p5cc$uoh$1@ger.gmane.org>
References: <505FFB47.30901@python.org> <k3p5cc$uoh$1@ger.gmane.org>
Message-ID: <50605C65.80202@stoneleaf.us>

Mark Lawrence wrote:
> On 24/09/2012 07:18, Georg Brandl wrote:

[snip impressive list of improvements]

> Yes, but apart from all that, what have the python devs ever done for 
> us?  Nothing :)

I'll take that kind of nothing any day of the week!  ;)

~Ethan~


From solipsis at pitrou.net  Mon Sep 24 17:27:49 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 24 Sep 2012 17:27:49 +0200
Subject: [Python-Dev] Better HTTP 1.1 support in http.server?
Message-ID: <20120924172749.459a9195@pitrou.net>


Hello,

I was wondering if it would be worth providing better HTTP 1.1 support
in http.server. The way I envision it, there would be a separate
HTTP11RequestHandler which would provide:

- a smart wfile with automatic chunk encoding (which relieves the API
  user from manually handling chunk encoding or content length
  calculation)
- keep-alive enabled by default
- HTTP 1.1 by default (BaseHTTPRequestHandler defaults to 1.0)

What do you think?

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From christian at python.org  Mon Sep 24 18:39:49 2012
From: christian at python.org (Christian Heimes)
Date: Mon, 24 Sep 2012 18:39:49 +0200
Subject: [Python-Dev] Better HTTP 1.1 support in http.server?
In-Reply-To: <20120924172749.459a9195@pitrou.net>
References: <20120924172749.459a9195@pitrou.net>
Message-ID: <50608CD5.20200@python.org>

Am 24.09.2012 17:27, schrieb Antoine Pitrou:
> 
> Hello,
> 
> I was wondering if it would be worth providing better HTTP 1.1 support
> in http.server. The way I envision it, there would be a separate
> HTTP11RequestHandler which would provide:
> 
> - a smart wfile with automatic chunk encoding (which relieves the API
>   user from manually handling chunk encoding or content length
>   calculation)
> - keep-alive enabled by default
> - HTTP 1.1 by default (BaseHTTPRequestHandler defaults to 1.0)
> 
> What do you think?

+1 for 3.4

You proposed gave me another idea. What do you think about SPDY support
in the stdlib? It's the next step after HTTP 1.1.


From dirkjan at ochtman.nl  Mon Sep 24 18:49:30 2012
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Mon, 24 Sep 2012 18:49:30 +0200
Subject: [Python-Dev] Better HTTP 1.1 support in http.server?
In-Reply-To: <50608CD5.20200@python.org>
References: <20120924172749.459a9195@pitrou.net> <50608CD5.20200@python.org>
Message-ID: <CAKmKYaC965XGspE4TUcFu+JiHEiDAvih4GFf4NsutxBC-TncDw@mail.gmail.com>

On Mon, Sep 24, 2012 at 6:39 PM, Christian Heimes <christian at python.org> wrote:
> You proposed gave me another idea. What do you think about SPDY support
> in the stdlib? It's the next step after HTTP 1.1.

I'd wait it out a bit. SPDY is currently iterating, and there's an
effort to define HTTP 2 that will likely supersede SPDY (and may
incorporate many of its ideas).

http://trac.tools.ietf.org/wg/httpbis/trac/wiki/Http2CfI

On the other hand, some WebSockets support might be useful.

Cheers,

Dirkjan

From senthil at uthcode.com  Mon Sep 24 18:51:39 2012
From: senthil at uthcode.com (Senthil Kumaran)
Date: Mon, 24 Sep 2012 09:51:39 -0700
Subject: [Python-Dev] Better HTTP 1.1 support in http.server?
In-Reply-To: <50608CD5.20200@python.org>
References: <20120924172749.459a9195@pitrou.net>
	<50608CD5.20200@python.org>
Message-ID: <CAPOVWOQGWiXyh=U+t9iSSPtDB+jXmrjdu4ye43uuDSykpzJDHg@mail.gmail.com>

On Mon, Sep 24, 2012 at 9:39 AM, Christian Heimes <christian at python.org> wrote:
>
> +1 for 3.4
>
> You proposed gave me another idea. What do you think about SPDY support
> in the stdlib? It's the next step after HTTP 1.1.

Yeah, it is a good idea. it should live along side with HTTP 1.1 as
another protocol layer.
Guido had given some suggestions to takle that, but I have been lazing
and lagging for while now in terms of contributions, but I am excited
about the idea.

-- 
Senthil

From felipecruz at loogica.net  Mon Sep 24 18:59:43 2012
From: felipecruz at loogica.net (Felipe Cruz)
Date: Mon, 24 Sep 2012 13:59:43 -0300
Subject: [Python-Dev] Better HTTP 1.1 support in http.server?
In-Reply-To: <CAPOVWOQGWiXyh=U+t9iSSPtDB+jXmrjdu4ye43uuDSykpzJDHg@mail.gmail.com>
References: <20120924172749.459a9195@pitrou.net> <50608CD5.20200@python.org>
	<CAPOVWOQGWiXyh=U+t9iSSPtDB+jXmrjdu4ye43uuDSykpzJDHg@mail.gmail.com>
Message-ID: <CAEynwxtpS8R6ny4N2teM3Yh_xSnjqCqHkwtg82tbKQOruM2z=Q@mail.gmail.com>

+1 for 3.4

And I'll be available to help with HTTP(websockets and SPDY) related stuff!



2012/9/24 Senthil Kumaran <senthil at uthcode.com>

> On Mon, Sep 24, 2012 at 9:39 AM, Christian Heimes <christian at python.org>
> wrote:
> >
> > +1 for 3.4
> >
> > You proposed gave me another idea. What do you think about SPDY support
> > in the stdlib? It's the next step after HTTP 1.1.
>
> Yeah, it is a good idea. it should live along side with HTTP 1.1 as
> another protocol layer.
> Guido had given some suggestions to takle that, but I have been lazing
> and lagging for while now in terms of contributions, but I am excited
> about the idea.
>
> --
> Senthil
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/felipecruz%40loogica.net
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120924/58e3a766/attachment.html>

From dw at botanicus.net  Tue Sep 25 15:02:10 2012
From: dw at botanicus.net (David Wilson)
Date: Tue, 25 Sep 2012 16:02:10 +0300
Subject: [Python-Dev] Better HTTP 1.1 support in http.server?
In-Reply-To: <50608CD5.20200@python.org>
References: <20120924172749.459a9195@pitrou.net> <50608CD5.20200@python.org>
Message-ID: <CABzuZRNt4eCJy8y_9LYT3b2_8-ctf_w4zcS5GS31pNsw=PP7aw@mail.gmail.com>

On 24 September 2012 19:39, Christian Heimes <christian at python.org> wrote:
> Am 24.09.2012 17:27, schrieb Antoine Pitrou:
>>
>> Hello,
>>
>> I was wondering if it would be worth providing better HTTP 1.1 support
>> in http.server. The way I envision it, there would be a separate
>> HTTP11RequestHandler which would provide:
>>
>> - a smart wfile with automatic chunk encoding (which relieves the API
>>   user from manually handling chunk encoding or content length
>>   calculation)
>> - keep-alive enabled by default
>> - HTTP 1.1 by default (BaseHTTPRequestHandler defaults to 1.0)
>>
>> What do you think?
>
> +1 for 3.4
>
> You proposed gave me another idea. What do you think about SPDY support
> in the stdlib? It's the next step after HTTP 1.1.

Given the rate SPDY is changing, complexity of implementation, and
that some prerequisites aren't in place (zlib module needs to support
specifying the dictionary), this probably isn't a good idea right
away. It's also easy to get a similar effect using a SPDY-capable
reverse proxy.

>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/dw%40botanicus.net

From ncoghlan at gmail.com  Wed Sep 26 05:16:29 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 26 Sep 2012 13:16:29 +1000
Subject: [Python-Dev] [Python-checkins] cpython: don't depend on
 __debug__ because it's baked in at freeze time (issue #16046)
In-Reply-To: <3XR5dd1CFWzQpL@mail.python.org>
References: <3XR5dd1CFWzQpL@mail.python.org>
Message-ID: <CADiSq7fNXaZe9CtFSkEtCXN6+EbVY4p59SrAbB-peK-k86iFMg@mail.gmail.com>

On Wed, Sep 26, 2012 at 1:23 AM, benjamin.peterson
<python-checkins at python.org> wrote:
>      If debug_override is not None, then it must be a boolean and is taken as
> -    the value of __debug__ instead.
> +    the value of bool(sys.flags.optimize) instead.

s/bool(sys.flags.optimize)/"not sys.flags.optimize"/?

It may be better to just rephrase this sentence entirely to better
account for the fact that we're now checking the runtime
sys.flags.optimize value rather than the compile time __debug__ value.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From benjamin at python.org  Wed Sep 26 06:25:26 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 26 Sep 2012 00:25:26 -0400
Subject: [Python-Dev] [Python-checkins] cpython: don't depend on
 __debug__ because it's baked in at freeze time (issue #16046)
In-Reply-To: <CADiSq7fNXaZe9CtFSkEtCXN6+EbVY4p59SrAbB-peK-k86iFMg@mail.gmail.com>
References: <3XR5dd1CFWzQpL@mail.python.org>
	<CADiSq7fNXaZe9CtFSkEtCXN6+EbVY4p59SrAbB-peK-k86iFMg@mail.gmail.com>
Message-ID: <CAPZV6o8yY+=i5HGFKeenV9rFuCqvGzRUBG_KYNN9zRZxGS_b1g@mail.gmail.com>

2012/9/25 Nick Coghlan <ncoghlan at gmail.com>:
> On Wed, Sep 26, 2012 at 1:23 AM, benjamin.peterson
> <python-checkins at python.org> wrote:
>>      If debug_override is not None, then it must be a boolean and is taken as
>> -    the value of __debug__ instead.
>> +    the value of bool(sys.flags.optimize) instead.
>
> s/bool(sys.flags.optimize)/"not sys.flags.optimize"/?
>
> It may be better to just rephrase this sentence entirely to better
> account for the fact that we're now checking the runtime
> sys.flags.optimize value rather than the compile time __debug__ value.

Good point; I rephrased it.


-- 
Regards,
Benjamin

From walter at livinglogic.de  Wed Sep 26 17:02:16 2012
From: walter at livinglogic.de (=?UTF-8?B?V2FsdGVyIETDtnJ3YWxk?=)
Date: Wed, 26 Sep 2012 17:02:16 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Add a few entries to
	whatsnew/3.3.rst.
In-Reply-To: <3XRhjY5yt1zQv1@mail.python.org>
References: <3XRhjY5yt1zQv1@mail.python.org>
Message-ID: <506318F8.5010303@livinglogic.de>

On 26.09.12 16:43, ezio.melotti wrote:

> http://hg.python.org/cpython/rev/36f61661f71e
> changeset:   79194:36f61661f71e
> user:        Ezio Melotti <ezio.melotti at gmail.com>
> date:        Wed Sep 26 17:43:23 2012 +0300
> summary:
>    Add a few entries to whatsnew/3.3.rst.
> [...]
> +
> +A new :data:`~html.entities.html5` dictionary that maps HTML5 named character
> +references to the equivalent Unicode character(s) (e.g. ``html5['gt;'] == '>'``)
> +has been added to the :mod:`html.entities` module.  The dictionary is now also
> +used by :class:`~html.parser.HTMLParser`.

Is there a reason why the trailing ';' is included in the entity names?

Servus,
    Walter



From ezio.melotti at gmail.com  Wed Sep 26 18:30:41 2012
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Wed, 26 Sep 2012 19:30:41 +0300
Subject: [Python-Dev] [Python-checkins] cpython: Add a few entries to
	whatsnew/3.3.rst.
In-Reply-To: <506318F8.5010303@livinglogic.de>
References: <3XRhjY5yt1zQv1@mail.python.org> <506318F8.5010303@livinglogic.de>
Message-ID: <CACBhJdH41wwNUwLOgNrk5fQ6PSMGJ17x6U4gcewrDqnaWkib9Q@mail.gmail.com>

On Wed, Sep 26, 2012 at 6:02 PM, Walter D?rwald <walter at livinglogic.de> wrote:
> On 26.09.12 16:43, ezio.melotti wrote:
>
>> http://hg.python.org/cpython/rev/36f61661f71e
>> changeset:   79194:36f61661f71e
>> user:        Ezio Melotti <ezio.melotti at gmail.com>
>> date:        Wed Sep 26 17:43:23 2012 +0300
>> summary:
>>    Add a few entries to whatsnew/3.3.rst.
>> [...]
>>
>> +
>> +A new :data:`~html.entities.html5` dictionary that maps HTML5 named character
>> +references to the equivalent Unicode character(s) (e.g. ``html5['gt;'] == '>'``)
>> +has been added to the :mod:`html.entities` module.  The dictionary is now also
>> +used by :class:`~html.parser.HTMLParser`.
>
>
> Is there a reason why the trailing ';' is included in the entity names?
>

Yes, to quote <http://bugs.python.org/issue11113#msg163706>:

"""
The problem is that the standard allows some charref to end without a
';', but not all of them.

So both "&Eacuteric" and &Eacute;ric" will be parsed as "?ric", but
only "&alpha;centauri" will result in "?centauri" -- "&alphacentauri"
will be returned unchanged.
"""

To preserve this I included them both, in the same way they are listed
at <http://www.w3.org/TR/html5/named-character-references.html>.
This is also explained at
<http://docs.python.org/dev/library/html.entities.html#html.entities.html5>.

Best Regards,
Ezio Melotti

> Servus,
>    Walter

From benjamin at python.org  Thu Sep 27 01:00:53 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 26 Sep 2012 19:00:53 -0400
Subject: [Python-Dev] buildbot with -O
Message-ID: <CAPZV6o-NAcrUm9z4cpP=jn+FcKcfoKGeCB9q1kaH0dhi9rMnEw@mail.gmail.com>

In light of issue #16046, I think it would be helpful to have a
buildbot running the testsuite with -O enabled.

-- 
Regards,
Benjamin

From fijall at gmail.com  Thu Sep 27 01:10:28 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 27 Sep 2012 01:10:28 +0200
Subject: [Python-Dev] buildbot with -O
In-Reply-To: <CAPZV6o-NAcrUm9z4cpP=jn+FcKcfoKGeCB9q1kaH0dhi9rMnEw@mail.gmail.com>
References: <CAPZV6o-NAcrUm9z4cpP=jn+FcKcfoKGeCB9q1kaH0dhi9rMnEw@mail.gmail.com>
Message-ID: <CAK5idxTAk_Q=Oi29=K0ihe1+2-BQM=TpYEQx70DezpF=HK_fUw@mail.gmail.com>

On Thu, Sep 27, 2012 at 1:00 AM, Benjamin Peterson <benjamin at python.org> wrote:
> In light of issue #16046, I think it would be helpful to have a
> buildbot running the testsuite with -O enabled.

How about deprecating -O completely instead? It does not seem to give
any speedups and definitely gives headaches.

From rdmurray at bitdance.com  Thu Sep 27 01:12:11 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 26 Sep 2012 19:12:11 -0400
Subject: [Python-Dev] buildbot with -O
In-Reply-To: <CAPZV6o-NAcrUm9z4cpP=jn+FcKcfoKGeCB9q1kaH0dhi9rMnEw@mail.gmail.com>
References: <CAPZV6o-NAcrUm9z4cpP=jn+FcKcfoKGeCB9q1kaH0dhi9rMnEw@mail.gmail.com>
Message-ID: <20120926231213.0F470250177@webabinitio.net>

On Wed, 26 Sep 2012 19:00:53 -0400, Benjamin Peterson <benjamin at python.org> wrote:
> In light of issue #16046, I think it would be helpful to have a
> buildbot running the testsuite with -O enabled.

I've got some (slow) hardware I could dedicate to this, though
it may take me a few weeks to find time to set it up.

--David

From benjamin at python.org  Thu Sep 27 01:13:42 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 26 Sep 2012 19:13:42 -0400
Subject: [Python-Dev] buildbot with -O
In-Reply-To: <CAK5idxTAk_Q=Oi29=K0ihe1+2-BQM=TpYEQx70DezpF=HK_fUw@mail.gmail.com>
References: <CAPZV6o-NAcrUm9z4cpP=jn+FcKcfoKGeCB9q1kaH0dhi9rMnEw@mail.gmail.com>
	<CAK5idxTAk_Q=Oi29=K0ihe1+2-BQM=TpYEQx70DezpF=HK_fUw@mail.gmail.com>
Message-ID: <CAPZV6o9WdrwEcttAVahXPQG43YrYrndOJN4o1jJawj=U890bRg@mail.gmail.com>

2012/9/26 Maciej Fijalkowski <fijall at gmail.com>:
> On Thu, Sep 27, 2012 at 1:00 AM, Benjamin Peterson <benjamin at python.org> wrote:
>> In light of issue #16046, I think it would be helpful to have a
>> buildbot running the testsuite with -O enabled.
>
> How about deprecating -O completely instead? It does not seem to give
> any speedups and definitely gives headaches.

I'm all for that, but I'd rather have it tested in the meantime.


-- 
Regards,
Benjamin

From trent at snakebite.org  Thu Sep 27 02:09:30 2012
From: trent at snakebite.org (Trent Nelson)
Date: Wed, 26 Sep 2012 20:09:30 -0400
Subject: [Python-Dev] buildbot with -O
In-Reply-To: <CAPZV6o9WdrwEcttAVahXPQG43YrYrndOJN4o1jJawj=U890bRg@mail.gmail.com>
References: <CAPZV6o-NAcrUm9z4cpP=jn+FcKcfoKGeCB9q1kaH0dhi9rMnEw@mail.gmail.com>
	<CAK5idxTAk_Q=Oi29=K0ihe1+2-BQM=TpYEQx70DezpF=HK_fUw@mail.gmail.com>
	<CAPZV6o9WdrwEcttAVahXPQG43YrYrndOJN4o1jJawj=U890bRg@mail.gmail.com>
Message-ID: <20120927000930.GA68527@snakebite.org>

On Wed, Sep 26, 2012 at 04:13:42PM -0700, Benjamin Peterson wrote:
> 2012/9/26 Maciej Fijalkowski <fijall at gmail.com>:
> > On Thu, Sep 27, 2012 at 1:00 AM, Benjamin Peterson <benjamin at python.org> wrote:
> >> In light of issue #16046, I think it would be helpful to have a
> >> buildbot running the testsuite with -O enabled.
> >
> > How about deprecating -O completely instead? It does not seem to give
> > any speedups and definitely gives headaches.
> 
> I'm all for that, but I'd rather have it tested in the meantime.

    You can use Snakebite for this sort of stuff.  Grab a slave ID off
    Antoine, then log in to whatever slave you want and set up a new
    instance (say, ~/buildslave-with-O).  (Look at existing slave
    details in ~/buildslave/buildbot.tac for hints on password etc.)

    And as a general side bar: all committers automatically have full
    access to Snakebite.  You don't need to register keys* or sign up or
    anything.  I've tried to make it as painless as possible to connect
    to:
        % cd ~
        % svn co http://svn.snakebite.net/.snakebite
        % cd .snakebite
        % sh snakebite.subr

            [*]: just to clarify: Snakebite's already been primed
                 with whatever ssh key you use to commit to hg.

    From then on, just run ~/.snakebite/sb to get access to the host
    menu.  That will work out of the box for all committers.  (Next set
    of enhancements will focus on linking a local source directory (on
    your laptop/desktop) to Snakebite and spinning up a buildbot master
    + slaves on demand, so you won't even have to log in and faff around
    with priming a build environment like you do now.  (I, uh, sort of
    need to do a little funding/donation drive before then though, which
    is what I'm currently organizing.))


        Trent.

From stefan_ml at behnel.de  Thu Sep 27 18:35:08 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 27 Sep 2012 18:35:08 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Add a few entries to
	whatsnew/3.3.rst.
In-Reply-To: <CACBhJdH41wwNUwLOgNrk5fQ6PSMGJ17x6U4gcewrDqnaWkib9Q@mail.gmail.com>
References: <3XRhjY5yt1zQv1@mail.python.org> <506318F8.5010303@livinglogic.de>
	<CACBhJdH41wwNUwLOgNrk5fQ6PSMGJ17x6U4gcewrDqnaWkib9Q@mail.gmail.com>
Message-ID: <k41v7r$6pf$1@ger.gmane.org>

Ezio Melotti, 26.09.2012 18:30:
> """
> The problem is that the standard allows some charref to end without a
> ';', but not all of them.
> 
> So both "&Eacuteric" and &Eacute;ric" will be parsed as "?ric", but
> only "&alpha;centauri" will result in "?centauri" -- "&alphacentauri"
> will be returned unchanged.
> """
> 
> To preserve this I included them both, in the same way they are listed
> at <http://www.w3.org/TR/html5/named-character-references.html>.

Interesting. Seems to be missing on dailywtf, though. Maybe just an oversight.

Stefan



From merwok at netwok.org  Fri Sep 28 00:30:49 2012
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Thu, 27 Sep 2012 18:30:49 -0400
Subject: [Python-Dev] Python Bug Day in October
Message-ID: <5064D399.5050208@netwok.org>

Hi all,

The Montreal-Python user group would like to host a bug day on October
27 (to be confirmed) at a partner university in Montreal.  It would be
cool to do a bug day on IRC like we used to (and in other physical
locations if people want to!) to get new contributors and close bugs.
What do you think?

Cheers

From brian at python.org  Fri Sep 28 00:41:37 2012
From: brian at python.org (Brian Curtin)
Date: Thu, 27 Sep 2012 17:41:37 -0500
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <5064D399.5050208@netwok.org>
References: <5064D399.5050208@netwok.org>
Message-ID: <CAD+XWwog59QuNGHkmWE_+8BMaejyqJSbu-K1PB+p03dTBPz46w@mail.gmail.com>

On Thu, Sep 27, 2012 at 5:30 PM, ?ric Araujo <merwok at netwok.org> wrote:
> Hi all,
>
> The Montreal-Python user group would like to host a bug day on October
> 27 (to be confirmed) at a partner university in Montreal.  It would be
> cool to do a bug day on IRC like we used to (and in other physical
> locations if people want to!) to get new contributors and close bugs.
> What do you think?

Sounds good to me, and I'll see if I can get some interest in having a
meetup in Chicago. Would be nice to try and get all of the
implementations to join in.

Whenever a final date is picked out, I'll write something up and
publicize it on the PSF blog, dev blog, pythonsprints.com, etc.

From tarek at ziade.org  Fri Sep 28 01:37:01 2012
From: tarek at ziade.org (=?UTF-8?B?VGFyZWsgWmlhZMOp?=)
Date: Fri, 28 Sep 2012 01:37:01 +0200
Subject: [Python-Dev] [Distutils] distlib updated with resources API
In-Reply-To: <loom.20120928T004648-800@post.gmane.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
Message-ID: <5064E31D.80601@ziade.org>

On 9/28/12 12:55 AM, Antoine Pitrou wrote:
>> Last but not least, distlib is the plan forward endorsed by python-dev,
> Is it? I haven't seen a PEP or an official decision about that. Just because
> someone proposed it on a mailing-list doesn't mean it is "endorsed by
> python-dev".

We discussed about this with Vinay, Nick and al on python-dev, based on 
Nick's document
that describes what 'distlib' is.

The document has changed since then, 
http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packaging_api.html

But the idea was to create a subset of 4 or 5 modules that implement the 
various PEPs.

Vinay started to work on this and made progress.

When I said "endorsed", I mean that most of the people in python-dev 
that care about packaging agreed or did not disagree.

Now, if you disagree please say it. Or if you need an official decision, 
we need to first declare who is the current packaging BDFL maybe ?

And since you seem interested in the topic maybe you could take that role ?

Cheers
Tarek


From rdmurray at bitdance.com  Fri Sep 28 02:10:50 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 27 Sep 2012 20:10:50 -0400
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <5064D399.5050208@netwok.org>
References: <5064D399.5050208@netwok.org>
Message-ID: <20120928001051.732A0250173@webabinitio.net>

On Thu, 27 Sep 2012 18:30:49 -0400, merwok at netwok.org wrote:
> The Montreal-Python user group would like to host a bug day on October
> 27 (to be confirmed) at a partner university in Montreal.  It would be
> cool to do a bug day on IRC like we used to (and in other physical
> locations if people want to!) to get new contributors and close bugs.
> What do you think?

All that is needed for an IRC bug day to happen is for someone to
coordinate and advertise it.  It sounds like you and Brian between you
have that covered.  Of course, it does help to have committers on hand.
I think I for one will probably be available during the day (EST) on
the 27th.

--David

From ncoghlan at gmail.com  Fri Sep 28 04:25:37 2012
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 28 Sep 2012 12:25:37 +1000
Subject: [Python-Dev] [Distutils] distlib updated with resources API
In-Reply-To: <5064E31D.80601@ziade.org>
References: <loom.20120923T174554-748@post.gmane.org>
	<50636845.8080204@ziade.org>
	<loom.20120926T233609-992@post.gmane.org>
	<50645C34.6070606@ziade.org>
	<1348756086.3495.1.camel@localhost.localdomain>
	<50646CFF.3010601@ziade.org>
	<loom.20120927T173348-617@post.gmane.org>
	<50647599.3030507@ziade.org>
	<loom.20120928T004648-800@post.gmane.org>
	<5064E31D.80601@ziade.org>
Message-ID: <CADiSq7et7kT7NKSjNJ-D3oWAWcbNNF43=nqaj=VrCkThJoyYXA@mail.gmail.com>

On Fri, Sep 28, 2012 at 9:37 AM, Tarek Ziad? <tarek at ziade.org> wrote:
> On 9/28/12 12:55 AM, Antoine Pitrou wrote:
>>>
>>> Last but not least, distlib is the plan forward endorsed by python-dev,
>>
>> Is it? I haven't seen a PEP or an official decision about that. Just
>> because
>> someone proposed it on a mailing-list doesn't mean it is "endorsed by
>> python-dev".
>
>
> We discussed about this with Vinay, Nick and al on python-dev, based on
> Nick's document
> that describes what 'distlib' is.
>
> The document has changed since then,
> http://python-notes.boredomandlaziness.org/en/latest/pep_ideas/core_packaging_api.html

Yeah, don't read too much into the current state of that - it will
eventually become a proposal for a standardised *in-memory* data
structure to better support metadata interoperability between
packaging tools, but it isn't there yet (although scrubbing every
reference to "JSON file" and replacing it with "API data structure"
would get you close - think of the overall idea as "like dictConfig,
but for distribution metadata rather than logging configurations". We
need something like that in order to allow import hooks to correctly
supply distribution metadata).

The original email thread from the removal of packaging from 3.3 is
probably a better point of reference, with a concrete "distlib" PEP
still on the todo list.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From petri at digip.org  Fri Sep 28 08:39:51 2012
From: petri at digip.org (Petri Lehtinen)
Date: Fri, 28 Sep 2012 09:39:51 +0300
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <5064D399.5050208@netwok.org>
References: <5064D399.5050208@netwok.org>
Message-ID: <20120928063951.GD20102@p29>

?ric Araujo wrote:
> Hi all,
> 
> The Montreal-Python user group would like to host a bug day on October
> 27 (to be confirmed) at a partner university in Montreal.  It would be
> cool to do a bug day on IRC like we used to (and in other physical
> locations if people want to!) to get new contributors and close bugs.
> What do you think?

Sounds good to me.

I'm currently gathering a list [1] of easy and easyish issues for the
PyCon Finland sprint, because finding things to work on is (in my
opinion) the hardest and most time consuming part of bug fixing.

I'm going to organize the list in the same way the stdlib
documentation [2] is split into sections, so there should be something
to work on for everyone, regardless of their interest area.

PyCon Finland is on October 22-23, and I expect most of the list
remain unused, so maybe it could also be used in the bug day
afterwards. What do you think?

Petri


[1] http://piratepad.net/pyconfi-sprint-issues
[2] http://docs.python.org/dev/library/

From merwok at netwok.org  Fri Sep 28 16:48:41 2012
From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=)
Date: Fri, 28 Sep 2012 10:48:41 -0400
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <20120928063951.GD20102@p29>
References: <5064D399.5050208@netwok.org> <20120928063951.GD20102@p29>
Message-ID: <5065B8C9.4020808@netwok.org>

Hi,

Cool to see the positive response!  I?m waiting for confirmation of our
venue in Montreal and will follow-up on this list and update the wiki page.

Petri, your lists of bugs will definitely be helpful.

Thanks all!

From stefan at bytereef.org  Fri Sep 28 17:40:42 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Fri, 28 Sep 2012 17:40:42 +0200
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <505FFB47.30901@python.org>
References: <505FFB47.30901@python.org>
Message-ID: <20120928154042.GA19170@sleipnir.bytereef.org>

Georg Brandl <georg at python.org> wrote:
> * A C implementation of the "decimal" module, with up to 80x speedup
>     for decimal-heavy applications

Could you bump up the factor to 120x in the final announcement? There were
a couple of performance improvements in the meantime, and this is what I'm
consistently measuring now.


Thanks,

Stefan Krah




From status at bugs.python.org  Fri Sep 28 18:06:29 2012
From: status at bugs.python.org (Python tracker)
Date: Fri, 28 Sep 2012 18:06:29 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20120928160629.9B3D51C976@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2012-09-21 - 2012-09-28)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3754 (+23)
  closed 24106 (+61)
  total  27860 (+84)

Open issues with patches: 1662 


Issues opened (52)
==================

#16000: test_curses should use unittest
http://bugs.python.org/issue16000  opened by chris.jerdonek

#16001: small ints: cache string representation
http://bugs.python.org/issue16001  opened by haypo

#16003: thread_PyThread_start_new_thread fails with a generic error
http://bugs.python.org/issue16003  opened by kristjan.jonsson

#16004: Add `make touch` to 2.7 Makefile
http://bugs.python.org/issue16004  opened by chris.jerdonek

#16005: smtplib.SMTP().sendmail() and rset()
http://bugs.python.org/issue16005  opened by DDarko

#16007: Improved Error message for failing re expressions
http://bugs.python.org/issue16007  opened by lingster

#16009: Json error messages could provide more information about the e
http://bugs.python.org/issue16009  opened by Luka.Rahne

#16011: "in" should be consistent with return value of __contains__
http://bugs.python.org/issue16011  opened by nparikh

#16013: small csv reader bug
http://bugs.python.org/issue16013  opened by arigo

#16023: IDLE freezes on ^5 or ^6 (Un-)Tabify Region with OS X Cocoa Tk
http://bugs.python.org/issue16023  opened by szellmeyer

#16024: Doc cleanup regarding path=fd, dir_fd, follow_symlinks, etc
http://bugs.python.org/issue16024  opened by larry

#16025: Minor corrections to the zipfile documentation
http://bugs.python.org/issue16025  opened by storchaka

#16026: csv.DictReader argument names documented incorrectly
http://bugs.python.org/issue16026  opened by petere

#16027: pkgutil doesn't support frozen modules
http://bugs.python.org/issue16027  opened by lemburg

#16029: pickle.dumps(xrange(sys.maxsize)) produces xrange(0)
http://bugs.python.org/issue16029  opened by akira

#16030: xrange repr broken for large arguments
http://bugs.python.org/issue16030  opened by mark.dickinson

#16034: bz2 module appears slower in Python 3.x versus Python 2.x
http://bugs.python.org/issue16034  opened by victorhooi

#16036: simplify int() signature docs
http://bugs.python.org/issue16036  opened by chris.jerdonek

#16037: httplib: header parsing is not delimited
http://bugs.python.org/issue16037  opened by christian.heimes

#16038: ftplib: unlimited readline() from connection
http://bugs.python.org/issue16038  opened by christian.heimes

#16039: imaplib: unlimited readline() from connection
http://bugs.python.org/issue16039  opened by christian.heimes

#16040: nntplib: unlimited readline() from connection
http://bugs.python.org/issue16040  opened by christian.heimes

#16041: poplib: unlimited readline() from connection
http://bugs.python.org/issue16041  opened by christian.heimes

#16042: smtplib: unlimited readline() from connection
http://bugs.python.org/issue16042  opened by christian.heimes

#16043: xmlrpc: gzip_decode has unlimited read()
http://bugs.python.org/issue16043  opened by christian.heimes

#16044: xml.etree.ElementTree.Element: append method iterator param is
http://bugs.python.org/issue16044  opened by kirpit

#16045: add more unit tests for built-in int()
http://bugs.python.org/issue16045  opened by chris.jerdonek

#16047: Tools/freeze no longer works in Python 3
http://bugs.python.org/issue16047  opened by lemburg

#16048: Tutorial-classes-remarks: replace paragragh
http://bugs.python.org/issue16048  opened by terry.reedy

#16049: Create abstract base classes by inheritance rather than a dire
http://bugs.python.org/issue16049  opened by rhettinger

#16050: ctypes: callback from C++ to Python fails with Illegal Instruc
http://bugs.python.org/issue16050  opened by Opilki_Inside

#16053: "strict" parameter is not documented in csv module
http://bugs.python.org/issue16053  opened by storchaka

#16055: incorrect error text for int(base=1000, x='1')
http://bugs.python.org/issue16055  opened by chris.jerdonek

#16056: shadowed test names in std lib regression tests
http://bugs.python.org/issue16056  opened by xdegaye

#16057: Subclasses of JSONEncoder should not be insturcted to call JSO
http://bugs.python.org/issue16057  opened by Justin.Lebar

#16058: ConfigParser no longer deepcopy compatible in 2.7
http://bugs.python.org/issue16058  opened by rbcollins

#16059: Serialize MD5 computation-state and resume later
http://bugs.python.org/issue16059  opened by Ye.Yuan

#16061: performance regression in string replace for 3.3
http://bugs.python.org/issue16061  opened by BreamoreBoy

#16062: Socket closed prematurely in httplib for https
http://bugs.python.org/issue16062  opened by ABR

#16063: HMAC trans_5C is a string, causing a TypeError
http://bugs.python.org/issue16063  opened by Adam.Glenn

#16065: Python/distutils setup.py: passing --prefix / makes --root ign
http://bugs.python.org/issue16065  opened by arschficker

#16066: Truncated POST data in CGI script on Windows 7
http://bugs.python.org/issue16066  opened by Alexander.Martin

#16067: UAC prompt for installation shows temporary file name
http://bugs.python.org/issue16067  opened by breyed

#16070: Structure and native Structure (LittleEndianStructure on Windo
http://bugs.python.org/issue16070  opened by hct

#16071: fix link to email.message.Message in mailbox docs
http://bugs.python.org/issue16071  opened by chris.jerdonek

#16074: bad error in rename
http://bugs.python.org/issue16074  opened by v+python

#16076: xml.etree.ElementTree.Element is no longer pickleable
http://bugs.python.org/issue16076  opened by einarfd

#16077: fix code example in docs for built-in reduce()
http://bugs.python.org/issue16077  opened by chris.jerdonek

#16079: list duplicate test names with patchcheck
http://bugs.python.org/issue16079  opened by xdegaye

#16081: Fix compile warnings in thread_pthread.h
http://bugs.python.org/issue16081  opened by bbrazil

#16082: xml.etree.ElementTree.tostringlist does conform to it's docume
http://bugs.python.org/issue16082  opened by einarfd

#16083: HTTPServer does not correctly handle bad headers
http://bugs.python.org/issue16083  opened by maker



Most recent 15 issues with no replies (15)
==========================================

#16081: Fix compile warnings in thread_pthread.h
http://bugs.python.org/issue16081

#16079: list duplicate test names with patchcheck
http://bugs.python.org/issue16079

#16076: xml.etree.ElementTree.Element is no longer pickleable
http://bugs.python.org/issue16076

#16074: bad error in rename
http://bugs.python.org/issue16074

#16071: fix link to email.message.Message in mailbox docs
http://bugs.python.org/issue16071

#16070: Structure and native Structure (LittleEndianStructure on Windo
http://bugs.python.org/issue16070

#16067: UAC prompt for installation shows temporary file name
http://bugs.python.org/issue16067

#16063: HMAC trans_5C is a string, causing a TypeError
http://bugs.python.org/issue16063

#16058: ConfigParser no longer deepcopy compatible in 2.7
http://bugs.python.org/issue16058

#16057: Subclasses of JSONEncoder should not be insturcted to call JSO
http://bugs.python.org/issue16057

#16053: "strict" parameter is not documented in csv module
http://bugs.python.org/issue16053

#16047: Tools/freeze no longer works in Python 3
http://bugs.python.org/issue16047

#16041: poplib: unlimited readline() from connection
http://bugs.python.org/issue16041

#16040: nntplib: unlimited readline() from connection
http://bugs.python.org/issue16040

#16039: imaplib: unlimited readline() from connection
http://bugs.python.org/issue16039



Most recent 15 issues waiting for review (15)
=============================================

#16083: HTTPServer does not correctly handle bad headers
http://bugs.python.org/issue16083

#16081: Fix compile warnings in thread_pthread.h
http://bugs.python.org/issue16081

#16079: list duplicate test names with patchcheck
http://bugs.python.org/issue16079

#16056: shadowed test names in std lib regression tests
http://bugs.python.org/issue16056

#16055: incorrect error text for int(base=1000, x='1')
http://bugs.python.org/issue16055

#16050: ctypes: callback from C++ to Python fails with Illegal Instruc
http://bugs.python.org/issue16050

#16042: smtplib: unlimited readline() from connection
http://bugs.python.org/issue16042

#16036: simplify int() signature docs
http://bugs.python.org/issue16036

#16034: bz2 module appears slower in Python 3.x versus Python 2.x
http://bugs.python.org/issue16034

#16029: pickle.dumps(xrange(sys.maxsize)) produces xrange(0)
http://bugs.python.org/issue16029

#16025: Minor corrections to the zipfile documentation
http://bugs.python.org/issue16025

#16024: Doc cleanup regarding path=fd, dir_fd, follow_symlinks, etc
http://bugs.python.org/issue16024

#16013: small csv reader bug
http://bugs.python.org/issue16013

#16004: Add `make touch` to 2.7 Makefile
http://bugs.python.org/issue16004

#16001: small ints: cache string representation
http://bugs.python.org/issue16001



Top 10 most discussed issues (10)
=================================

#8425: a -= b should be fast if a is a small set and b is a large set
http://bugs.python.org/issue8425  14 msgs

#10044: small int optimization
http://bugs.python.org/issue10044  10 msgs

#15958: bytes.join() should allow arbitrary buffer objects
http://bugs.python.org/issue15958   9 msgs

#15964: SyntaxError in asdl when building 2.7 with system Python 3
http://bugs.python.org/issue15964   9 msgs

#16029: pickle.dumps(xrange(sys.maxsize)) produces xrange(0)
http://bugs.python.org/issue16029   9 msgs

#16036: simplify int() signature docs
http://bugs.python.org/issue16036   9 msgs

#16055: incorrect error text for int(base=1000, x='1')
http://bugs.python.org/issue16055   8 msgs

#16056: shadowed test names in std lib regression tests
http://bugs.python.org/issue16056   8 msgs

#16001: small ints: cache string representation
http://bugs.python.org/issue16001   7 msgs

#16005: smtplib.SMTP().sendmail() and rset()
http://bugs.python.org/issue16005   7 msgs



Issues closed (58)
==================

#2126: BaseHTTPServer.py fails long POST from IE
http://bugs.python.org/issue2126  closed by pitrou

#2600: BindingHTTPConnectionWithTimeout and BindingHTTPHandlerWithTim
http://bugs.python.org/issue2600  closed by pitrou

#6027: test_xmlrpc_net fails when the ISP returns "302 Found"
http://bugs.python.org/issue6027  closed by ezio.melotti

#7844: Add -3 warning for absolute imports.
http://bugs.python.org/issue7844  closed by mark.dickinson

#10980: http.server Header Unicode Bug
http://bugs.python.org/issue10980  closed by pitrou

#11103: Python 3.2 installer doesn't register file extensions on Windo
http://bugs.python.org/issue11103  closed by serwy

#12376: unittest.TextTestResult.__init__ does not pass on its init arg
http://bugs.python.org/issue12376  closed by michael.foord

#13263: Group some os functions in submodules
http://bugs.python.org/issue13263  closed by ezio.melotti

#14295: PEP 417: adding mock module
http://bugs.python.org/issue14295  closed by michael.foord

#15270: "Economy of Expression" section outdated
http://bugs.python.org/issue15270  closed by rhettinger

#15323: Provide target name in output message when Mock.assert_called_
http://bugs.python.org/issue15323  closed by python-dev

#15379: Charmap decoding of no-BMP characters
http://bugs.python.org/issue15379  closed by pitrou

#15593: urlparse.parse_qs documentation wrong re: urlencode
http://bugs.python.org/issue15593  closed by python-dev

#15815: Add numerator to ZeroDivisionError messages
http://bugs.python.org/issue15815  closed by terry.reedy

#15882: _decimal.Decimal constructed from tuple
http://bugs.python.org/issue15882  closed by georg.brandl

#15895: PyRun_SimpleFileExFlags() can leak a FILE pointer
http://bugs.python.org/issue15895  closed by georg.brandl

#15900: Memory leak in PyUnicode_TranslateCharmap()
http://bugs.python.org/issue15900  closed by georg.brandl

#15925: email.utils.parsedate() and email.utils.parsedate_tz() should 
http://bugs.python.org/issue15925  closed by georg.brandl

#15926: Segmentation fault after multiple reinitializations
http://bugs.python.org/issue15926  closed by georg.brandl

#15935: clarify argparse docs re: add_argument() type and default argu
http://bugs.python.org/issue15935  closed by barry

#15949: docs.python.org not getting updated
http://bugs.python.org/issue15949  closed by ezio.melotti

#15953: Incorrect some fields declaration in the PyTypeObject document
http://bugs.python.org/issue15953  closed by python-dev

#15969: faulthandler: drop S from dump_tracebacks_later() function nam
http://bugs.python.org/issue15969  closed by georg.brandl

#15973: Segmentation fault on timezone comparison
http://bugs.python.org/issue15973  closed by georg.brandl

#15978: asyncore: included batteries don't fit
http://bugs.python.org/issue15978  closed by benjamin.peterson

#15993: Windows: 3.3.0-rc2.msi: test_buffer fails
http://bugs.python.org/issue15993  closed by loewis

#16002: AIX buildbot compile errors
http://bugs.python.org/issue16002  closed by skrah

#16006: Spelling mistake in PEP 213
http://bugs.python.org/issue16006  closed by ezio.melotti

#16008: [OS bug] DragonFly buildbot locale failures
http://bugs.python.org/issue16008  closed by skrah

#16010: Some Unicode in identifiers improperly rejected
http://bugs.python.org/issue16010  closed by r.david.murray

#16012: pyexpat: Argument for UseForeignDTD should be optional
http://bugs.python.org/issue16012  closed by christian.heimes

#16014: Incomplete link in docs
http://bugs.python.org/issue16014  closed by ezio.melotti

#16015: Incorrect startup header in tutorial
http://bugs.python.org/issue16015  closed by chris.jerdonek

#16016: Maybe make one author in ipaddress HOWTO
http://bugs.python.org/issue16016  closed by ezio.melotti

#16017: "urls" should be capital in tutorial
http://bugs.python.org/issue16017  closed by ezio.melotti

#16018: Impossible ... line in tutorial
http://bugs.python.org/issue16018  closed by ezio.melotti

#16019: Python highlighting in tutorial;should be text highlighting
http://bugs.python.org/issue16019  closed by ezio.melotti

#16020: Missing >>> in Python code example
http://bugs.python.org/issue16020  closed by ezio.melotti

#16021: In a table in PEP 235, brrrrrrrr should be N/A
http://bugs.python.org/issue16021  closed by mark.dickinson

#16022: Mistake in "What's New in Python 3.3"
http://bugs.python.org/issue16022  closed by python-dev

#16028: break in finally discards exception
http://bugs.python.org/issue16028  closed by mark.dickinson

#16031: relative import headaches
http://bugs.python.org/issue16031  closed by ncoghlan

#16032: IDLE 3.2 is crashing multiple times
http://bugs.python.org/issue16032  closed by ned.deily

#16033: IDLE crashes when debugging and saving
http://bugs.python.org/issue16033  closed by serwy

#16035: Segmentation fault in test suite of apsw
http://bugs.python.org/issue16035  closed by georg.brandl

#16046: python -O does not find *.pyo files
http://bugs.python.org/issue16046  closed by georg.brandl

#16051: Documentation missing pipes.quote()
http://bugs.python.org/issue16051  closed by r.david.murray

#16052: Typo in "What's new in 3.3"
http://bugs.python.org/issue16052  closed by python-dev

#16054: IDLE Preferences Crashes on click
http://bugs.python.org/issue16054  closed by ned.deily

#16060: Double decref and dereferencing after decref in int()
http://bugs.python.org/issue16060  closed by georg.brandl

#16064: unittest -m claims executable is "python", not "python3"
http://bugs.python.org/issue16064  closed by python-dev

#16068: with statement executes type(obj).__exit__ rather than getattr
http://bugs.python.org/issue16068  closed by Paul.Wiseman

#16069: packaging shows up on docs.python.org/dev
http://bugs.python.org/issue16069  closed by georg.brandl

#16072: fix documentation of string.replace() signature
http://bugs.python.org/issue16072  closed by asvetlov

#16073: fix map() statement in list comprehension example
http://bugs.python.org/issue16073  closed by chris.jerdonek

#16075: incoming.cia hook error when pushing commits
http://bugs.python.org/issue16075  closed by georg.brandl

#16078: Calendar.leapdays(y1,y2) bug
http://bugs.python.org/issue16078  closed by larry

#16080: test_decimal causes other tests to fail with LC_ALL=fr_FR
http://bugs.python.org/issue16080  closed by skrah

From brett at python.org  Fri Sep 28 19:53:33 2012
From: brett at python.org (Brett Cannon)
Date: Fri, 28 Sep 2012 13:53:33 -0400
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <20120928154042.GA19170@sleipnir.bytereef.org>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
Message-ID: <CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>

On Fri, Sep 28, 2012 at 11:40 AM, Stefan Krah <stefan at bytereef.org> wrote:

> Georg Brandl <georg at python.org> wrote:
> > * A C implementation of the "decimal" module, with up to 80x speedup
> >     for decimal-heavy applications
>
> Could you bump up the factor to 120x in the final announcement? There were
> a couple of performance improvements in the meantime, and this is what I'm
> consistently measuring now.
>

Is that based on Modules/_decimal/tests/bench.py or some other benchmark?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120928/69e74092/attachment.html>

From stefan at bytereef.org  Fri Sep 28 20:19:25 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Fri, 28 Sep 2012 20:19:25 +0200
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
Message-ID: <20120928181925.GA21350@sleipnir.bytereef.org>

Brett Cannon <brett at python.org> wrote:
>     Georg Brandl <georg at python.org> wrote:
>     > * A C implementation of the "decimal" module, with up to 80x speedup
>     > ? ? for decimal-heavy applications
> 
>     Could you bump up the factor to 120x in the final announcement? There were
>     a couple of performance improvements in the meantime, and this is what I'm
>     consistently measuring now.
> 
> 
> Is that based on Modules/_decimal/tests/bench.py or some other benchmark?

It's the pi benchmark from bench.py. This is what I'm typically getting
on a Core 2 Duo 3.16 GHz:


Precision: 9 decimal digits

float:
result: 3.1415926535897927
time: 0.113188s

cdecimal:
result: 3.14159265
time: 0.158313s

decimal:
result: 3.14159265
time: 18.671457s


Precision: 19 decimal digits

float:
result: 3.1415926535897927
time: 0.112874s

cdecimal:
result: 3.141592653589793236
time: 0.348100s

decimal:
result: 3.141592653589793236
time: 43.241220s



Stefan Krah



From p.f.moore at gmail.com  Fri Sep 28 22:51:39 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 28 Sep 2012 21:51:39 +0100
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <20120928181925.GA21350@sleipnir.bytereef.org>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
Message-ID: <CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>

On 28 September 2012 19:19, Stefan Krah <stefan at bytereef.org> wrote:
> Brett Cannon <brett at python.org> wrote:
>>     Georg Brandl <georg at python.org> wrote:
>>     > * A C implementation of the "decimal" module, with up to 80x speedup
>>     >     for decimal-heavy applications
>>
>>     Could you bump up the factor to 120x in the final announcement? There were
>>     a couple of performance improvements in the meantime, and this is what I'm
>>     consistently measuring now.
>>
>>
>> Is that based on Modules/_decimal/tests/bench.py or some other benchmark?
>
> It's the pi benchmark from bench.py. This is what I'm typically getting
> on a Core 2 Duo 3.16 GHz:
>
>
> Precision: 9 decimal digits
>
> float:
> result: 3.1415926535897927
> time: 0.113188s
>
> cdecimal:
> result: 3.14159265
> time: 0.158313s
>
> decimal:
> result: 3.14159265
> time: 18.671457s
>
>
> Precision: 19 decimal digits
>
> float:
> result: 3.1415926535897927
> time: 0.112874s
>
> cdecimal:
> result: 3.141592653589793236
> time: 0.348100s
>
> decimal:
> result: 3.141592653589793236
> time: 43.241220s

Wow! I had no idea cdecimal was that close in speed to float. That's
seriously impressive.

Paul

From soltysh at gmail.com  Fri Sep 28 22:52:59 2012
From: soltysh at gmail.com (Maciej Szulik)
Date: Fri, 28 Sep 2012 22:52:59 +0200
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <5064D399.5050208@netwok.org>
References: <5064D399.5050208@netwok.org>
Message-ID: <50660E2B.5000209@gmail.com>

On 09/28/2012 12:30 AM, ?ric Araujo wrote:
> Hi all,
>
> The Montreal-Python user group would like to host a bug day on October
> 27 (to be confirmed) at a partner university in Montreal.  It would be
> cool to do a bug day on IRC like we used to (and in other physical
> locations if people want to!) to get new contributors and close bugs.
> What do you think?
>

Great idea, I'll talk to ?ukasz Langa about possibilities to gather
a group of Poles interested in this topic. ?ukasz could be the
leader of our group.

Regards,
Maciej


From storchaka at gmail.com  Fri Sep 28 23:48:54 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sat, 29 Sep 2012 00:48:54 +0300
Subject: [Python-Dev] Deprecating "U" open mode
Message-ID: <k45609$kmn$1@ger.gmane.org>

Since Python 2.3 many open-like functions supports "Universal line mode" 
(PEP 278 [1]). Since 3.0 (and 2.6) PEP 3116 [2] suggests better 
alternative -- io.TextWrapper.

In issue15204 [3] proposed to deprecate "U" mode in open-like functions. 
In fact I found only three places where "U" mode is mentioned in the 
standard library:

* builtin open (io.open) (almost ignored);
* fileinput (transparently passed to open);
* zipfile (inefficient and inconsistent implementation).

Is someone uses "U" mode and has objections?

[1] http://www.python.org/dev/peps/pep-0278/
[2] http://www.python.org/dev/peps/pep-3116/
[3] http://bugs.python.org/issue15204


From timothy.c.delaney at gmail.com  Fri Sep 28 23:50:39 2012
From: timothy.c.delaney at gmail.com (Tim Delaney)
Date: Sat, 29 Sep 2012 07:50:39 +1000
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
Message-ID: <CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>

On 29 September 2012 06:51, Paul Moore <p.f.moore at gmail.com> wrote:

>
> Wow! I had no idea cdecimal was that close in speed to float. That's
> seriously impressive.
>

If those numbers are similar in other benchmarks, would it be accurate
and/or reasonable to include a statement along the lines of:

"comparable to float performance - usually no more than 3x for calculations
within the range of numbers covered by float"

or is there not enough evidence to conclude that?

Substitute suitable factor for 3x.

Obviously
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120929/26e14a3c/attachment.html>

From timothy.c.delaney at gmail.com  Fri Sep 28 23:52:51 2012
From: timothy.c.delaney at gmail.com (Tim Delaney)
Date: Sat, 29 Sep 2012 07:52:51 +1000
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
	<CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
Message-ID: <CAN8CLgmdb-z8y-NnSjRzbsGXOk8TGjLNn8Sk+42+8Jpor6nbPA@mail.gmail.com>

On 29 September 2012 07:50, Tim Delaney <timothy.c.delaney at gmail.com> wrote:

> On 29 September 2012 06:51, Paul Moore <p.f.moore at gmail.com> wrote:
>
>>
>> Wow! I had no idea cdecimal was that close in speed to float. That's
>> seriously impressive.
>>
>
> If those numbers are similar in other benchmarks, would it be accurate
> and/or reasonable to include a statement along the lines of:
>
> "comparable to float performance - usually no more than 3x for
> calculations within the range of numbers covered by float"
>
> or is there not enough evidence to conclude that?
>
> Substitute suitable factor for 3x.
>
> Obviously
>

And I meant to say congratulations - that's a really impressive result.

Oh-oh. The PSU (which does not exist) has found me. I must hi

Tim Delaney
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120929/79233f1c/attachment-0001.html>

From solipsis at pitrou.net  Sat Sep 29 00:08:51 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 29 Sep 2012 00:08:51 +0200
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
Message-ID: <20120929000851.329e9ee5@pitrou.net>

On Fri, 28 Sep 2012 21:51:39 +0100
Paul Moore <p.f.moore at gmail.com> wrote:
> On 28 September 2012 19:19, Stefan Krah <stefan at bytereef.org> wrote:
> > Brett Cannon <brett at python.org> wrote:
> >>     Georg Brandl <georg at python.org> wrote:
> >>     > * A C implementation of the "decimal" module, with up to 80x speedup
> >>     >     for decimal-heavy applications
> >>
> >>     Could you bump up the factor to 120x in the final announcement? There were
> >>     a couple of performance improvements in the meantime, and this is what I'm
> >>     consistently measuring now.
> >>
> >>
> >> Is that based on Modules/_decimal/tests/bench.py or some other benchmark?
> >
> > It's the pi benchmark from bench.py. This is what I'm typically getting
> > on a Core 2 Duo 3.16 GHz:
> >
> >
> > Precision: 9 decimal digits
> >
> > float:
> > result: 3.1415926535897927
> > time: 0.113188s
> >
> > cdecimal:
> > result: 3.14159265
> > time: 0.158313s
> >
> > decimal:
> > result: 3.14159265
> > time: 18.671457s
> >
> >
> > Precision: 19 decimal digits
> >
> > float:
> > result: 3.1415926535897927
> > time: 0.112874s
> >
> > cdecimal:
> > result: 3.141592653589793236
> > time: 0.348100s
> >
> > decimal:
> > result: 3.141592653589793236
> > time: 43.241220s
> 
> Wow! I had no idea cdecimal was that close in speed to float. That's
> seriously impressive.

I think this means the performance difference is on the same order
of magnitude as the CPython interpretation overhead. Still, it's
impressive indeed.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From solipsis at pitrou.net  Sat Sep 29 01:30:07 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 29 Sep 2012 01:30:07 +0200
Subject: [Python-Dev] Deprecating "U" open mode
References: <k45609$kmn$1@ger.gmane.org>
Message-ID: <20120929013007.073ba51d@pitrou.net>

On Sat, 29 Sep 2012 00:48:54 +0300
Serhiy Storchaka <storchaka at gmail.com> wrote:
> Since Python 2.3 many open-like functions supports "Universal line mode" 
> (PEP 278 [1]). Since 3.0 (and 2.6) PEP 3116 [2] suggests better 
> alternative -- io.TextWrapper.
> 
> In issue15204 [3] proposed to deprecate "U" mode in open-like functions. 
> In fact I found only three places where "U" mode is mentioned in the 
> standard library:
> 
> * builtin open (io.open) (almost ignored);
> * fileinput (transparently passed to open);
> * zipfile (inefficient and inconsistent implementation).

I'm ok on the principle, I just think we should minimize disruption (I
don't think the "U" mode adds a lot of complexity to the code).

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From wolfson at gmail.com  Sat Sep 29 02:30:14 2012
From: wolfson at gmail.com (Ben Wolfson)
Date: Fri, 28 Sep 2012 17:30:14 -0700
Subject: [Python-Dev] str.format bug again
Message-ID: <CAPc-aXkWQ0BrkP-aK0oECWkdMM1QanSwPjvpsE=n6UsWm5pswA@mail.gmail.com>

There's a patch for this bug: http://bugs.python.org/issue12014 that
needs reviewed. Petri Lehtinen made some (very minor) comments back in
May; otherwise it's been neglected. I've brought this up occasionally
here in the past; it would be great if someone could just give it a
thumbs up or down (or say what needs to be changed, or whatever).

-- 
Ben Wolfson
"Human kind has used its intelligence to vary the flavour of drinks,
which may be sweet, aromatic, fermented or spirit-based. ... Family
and social life also offer numerous other occasions to consume drinks
for pleasure." [Larousse, "Drink" entry]

From stefan at bytereef.org  Sat Sep 29 11:17:29 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Sat, 29 Sep 2012 11:17:29 +0200
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
	<CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
Message-ID: <20120929091729.GA4859@sleipnir.bytereef.org>

Tim Delaney <timothy.c.delaney at gmail.com> wrote:
> If those numbers are similar in other benchmarks, would it be accurate and/or
> reasonable to include a statement along the lines of:
> 
> "comparable to float performance - usually no more than 3x for calculations
> within the range of numbers covered by float"

For numerical programs, 1.4x (9 digits) to 3x (19 digits) slower would be
accurate. On Windows the difference is even less.

For output formatting, cdecimal is faster than float (at least it was when
I posted a benchmark a couple of months ago).


Stefan Krah



From stefan at bytereef.org  Sat Sep 29 11:22:03 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Sat, 29 Sep 2012 11:22:03 +0200
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <20120929000851.329e9ee5@pitrou.net>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
	<20120929000851.329e9ee5@pitrou.net>
Message-ID: <20120929092203.GB4859@sleipnir.bytereef.org>

Antoine Pitrou <solipsis at pitrou.net> wrote:
> > Wow! I had no idea cdecimal was that close in speed to float. That's
> > seriously impressive.
> 
> I think this means the performance difference is on the same order
> of magnitude as the CPython interpretation overhead. Still, it's
> impressive indeed.

Of course, if you compare a pure C program that uses libmpdec to a C program
that uses floats, the difference will be much higher.


Stefan Krah




From georg at python.org  Sat Sep 29 14:18:54 2012
From: georg at python.org (Georg Brandl)
Date: Sat, 29 Sep 2012 14:18:54 +0200
Subject: [Python-Dev] [RELEASED] Python 3.3.0
Message-ID: <5066E72E.2010100@python.org>

On behalf of the Python development team, I'm delighted to announce the
Python 3.3.0 final release.

Python 3.3 includes a range of improvements of the 3.x series, as well
as easier porting between 2.x and 3.x.  Major new features and changes
in the 3.3 release series are:

* PEP 380, syntax for delegating to a subgenerator ("yield from")
* PEP 393, flexible string representation (doing away with the
    distinction between "wide" and "narrow" Unicode builds)
* A C implementation of the "decimal" module, with up to 120x speedup
    for decimal-heavy applications
* The import system (__import__) now based on importlib by default
* The new "lzma" module with LZMA/XZ support
* PEP 397, a Python launcher for Windows
* PEP 405, virtual environment support in core
* PEP 420, namespace package support
* PEP 3151, reworking the OS and IO exception hierarchy
* PEP 3155, qualified name for classes and functions
* PEP 409, suppressing exception context
* PEP 414, explicit Unicode literals to help with porting
* PEP 418, extended platform-independent clocks in the "time" module
* PEP 412, a new key-sharing dictionary implementation that
    significantly saves memory for object-oriented code
* PEP 362, the function-signature object
* The new "faulthandler" module that helps diagnosing crashes
* The new "unittest.mock" module
* The new "ipaddress" module
* The "sys.implementation" attribute
* A policy framework for the email package, with a provisional (see
    PEP 411) policy that adds much improved unicode support for email
    header parsing
* A "collections.ChainMap" class for linking mappings to a single unit
* Wrappers for many more POSIX functions in the "os" and "signal"
    modules, as well as other useful functions such as "sendfile()"
* Hash randomization, introduced in earlier bugfix releases, is now
    switched on by default

In total, almost 500 API items are new or improved in Python 3.3.
For a more extensive list of changes in 3.3.0, see

      http://docs.python.org/3.3/whatsnew/3.3.html

To download Python 3.3.0 visit:

      http://www.python.org/download/releases/3.3.0/

This is a production release, please report any bugs to

       http://bugs.python.org/


Enjoy!

--
Georg Brandl, Release Manager
georg at python.org
(on behalf of the entire python-dev team and 3.3's contributors)

From amitsaha.in at gmail.com  Sat Sep 29 14:23:49 2012
From: amitsaha.in at gmail.com (Amit Saha)
Date: Sat, 29 Sep 2012 22:23:49 +1000
Subject: [Python-Dev] [RELEASED] Python 3.3.0
In-Reply-To: <5066E72E.2010100@python.org>
References: <5066E72E.2010100@python.org>
Message-ID: <CANODV3k0Ff84wcDwviNWpdOT8L6H6hnFfaXc9iG8k3So8Hqx0A@mail.gmail.com>

On Sat, Sep 29, 2012 at 10:18 PM, Georg Brandl <georg at python.org> wrote:
> On behalf of the Python development team, I'm delighted to announce the
> Python 3.3.0 final release.
>
> Python 3.3 includes a range of improvements of the 3.x series, as well
> as easier porting between 2.x and 3.x.  Major new features and changes
> in the 3.3 release series are:
>
> * PEP 380, syntax for delegating to a subgenerator ("yield from")
> * PEP 393, flexible string representation (doing away with the
>    distinction between "wide" and "narrow" Unicode builds)
> * A C implementation of the "decimal" module, with up to 120x speedup
>    for decimal-heavy applications
> * The import system (__import__) now based on importlib by default
> * The new "lzma" module with LZMA/XZ support
> * PEP 397, a Python launcher for Windows
> * PEP 405, virtual environment support in core
> * PEP 420, namespace package support
> * PEP 3151, reworking the OS and IO exception hierarchy
> * PEP 3155, qualified name for classes and functions
> * PEP 409, suppressing exception context
> * PEP 414, explicit Unicode literals to help with porting
> * PEP 418, extended platform-independent clocks in the "time" module
> * PEP 412, a new key-sharing dictionary implementation that
>    significantly saves memory for object-oriented code
> * PEP 362, the function-signature object
> * The new "faulthandler" module that helps diagnosing crashes
> * The new "unittest.mock" module
> * The new "ipaddress" module
> * The "sys.implementation" attribute
> * A policy framework for the email package, with a provisional (see
>    PEP 411) policy that adds much improved unicode support for email
>    header parsing
> * A "collections.ChainMap" class for linking mappings to a single unit
> * Wrappers for many more POSIX functions in the "os" and "signal"
>    modules, as well as other useful functions such as "sendfile()"
> * Hash randomization, introduced in earlier bugfix releases, is now
>    switched on by default
>
> In total, almost 500 API items are new or improved in Python 3.3.
> For a more extensive list of changes in 3.3.0, see
>
>      http://docs.python.org/3.3/whatsnew/3.3.html

Redirects to http://docs.python.org/py3k/whatsnew/3.3.html: 404 Not Found.

Cheers,
Amit.

-- 
http://echorand.me

From amitsaha.in at gmail.com  Sat Sep 29 14:45:45 2012
From: amitsaha.in at gmail.com (Amit Saha)
Date: Sat, 29 Sep 2012 22:45:45 +1000
Subject: [Python-Dev] [RELEASED] Python 3.3.0
In-Reply-To: <5066EB7C.10407@davea.name>
References: <5066E72E.2010100@python.org>
	<CANODV3k0Ff84wcDwviNWpdOT8L6H6hnFfaXc9iG8k3So8Hqx0A@mail.gmail.com>
	<5066EB7C.10407@davea.name>
Message-ID: <CANODV3mPCmULdgvtHCNYMxezDe1oo7y3swWOmvWL2XSKNZp4kg@mail.gmail.com>

On Sat, Sep 29, 2012 at 10:37 PM, Dave Angel <d at davea.name> wrote:
> On 09/29/2012 08:23 AM, Amit Saha wrote:
>> On Sat, Sep 29, 2012 at 10:18 PM, Georg Brandl <georg at python.org> wrote:
>>> <snip>
>>>
>>> For a more extensive list of changes in 3.3.0, see
>>>
>>>      http://docs.python.org/3.3/whatsnew/3.3.html
>> Redirects to http://docs.python.org/py3k/whatsnew/3.3.html: 404 Not Found.
>>
>>
> Works for me.  Perhaps a momentary glitch.

Yes, I clicked too soon, i guess..

-Amit.
-- 
http://echorand.me

From p.f.moore at gmail.com  Sat Sep 29 15:07:15 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 29 Sep 2012 14:07:15 +0100
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <20120929091729.GA4859@sleipnir.bytereef.org>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
	<CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
	<20120929091729.GA4859@sleipnir.bytereef.org>
Message-ID: <CACac1F8HNLeCSMmmk8-AnL1NWjW=cNQrbLwBk5JJ1q=Mf0hdow@mail.gmail.com>

On 29 September 2012 10:17, Stefan Krah <stefan at bytereef.org> wrote:
> Tim Delaney <timothy.c.delaney at gmail.com> wrote:
>> If those numbers are similar in other benchmarks, would it be accurate and/or
>> reasonable to include a statement along the lines of:
>>
>> "comparable to float performance - usually no more than 3x for calculations
>> within the range of numbers covered by float"
>
> For numerical programs, 1.4x (9 digits) to 3x (19 digits) slower would be
> accurate. On Windows the difference is even less.
>
> For output formatting, cdecimal is faster than float (at least it was when
> I posted a benchmark a couple of months ago).

To me, this means that the key point is that for the casual user,
float is no longer the "obvious" choice. You'd choose float for the
convenience of a built in type, and Decimal for the more natural
rounding and precision semantics. If you are sufficiently interested
in performance for it to matter, you're no longer a "casual" user. (Up
until now, I'd have said use float unless your need for the better
behaviour justifies the performance loss - that's no longer the case).

Paul.

From eliben at gmail.com  Sat Sep 29 15:24:46 2012
From: eliben at gmail.com (Eli Bendersky)
Date: Sat, 29 Sep 2012 06:24:46 -0700
Subject: [Python-Dev] [RELEASED] Python 3.3.0
In-Reply-To: <5066E72E.2010100@python.org>
References: <5066E72E.2010100@python.org>
Message-ID: <CAF-Rda-z4LOzT98vS4vpdwP4sNGD5YF6-=JDZgtti9f7H53j0g@mail.gmail.com>

On Sat, Sep 29, 2012 at 5:18 AM, Georg Brandl <georg at python.org> wrote:
> On behalf of the Python development team, I'm delighted to announce the
> Python 3.3.0 final release.
>

Yay :)

From p.f.moore at gmail.com  Sat Sep 29 15:51:28 2012
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 29 Sep 2012 14:51:28 +0100
Subject: [Python-Dev] [RELEASED] Python 3.3.0
In-Reply-To: <CAF-Rda-z4LOzT98vS4vpdwP4sNGD5YF6-=JDZgtti9f7H53j0g@mail.gmail.com>
References: <5066E72E.2010100@python.org>
	<CAF-Rda-z4LOzT98vS4vpdwP4sNGD5YF6-=JDZgtti9f7H53j0g@mail.gmail.com>
Message-ID: <CACac1F8miQd3oKBb3JkCnzuB0KWjBzfqrQ6TNb-__fhFSAHyhg@mail.gmail.com>

On 29 September 2012 14:24, Eli Bendersky <eliben at gmail.com> wrote:
> On Sat, Sep 29, 2012 at 5:18 AM, Georg Brandl <georg at python.org> wrote:
>> On behalf of the Python development team, I'm delighted to announce the
>> Python 3.3.0 final release.
>>
>
> Yay :)

Agreed - this is a really nice release, thanks to all who put it together.
Paul

From guido at python.org  Sat Sep 29 17:52:48 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 29 Sep 2012 08:52:48 -0700
Subject: [Python-Dev] [RELEASED] Python 3.3.0
In-Reply-To: <5066E72E.2010100@python.org>
References: <5066E72E.2010100@python.org>
Message-ID: <CAP7+vJJmQPfSi+0++RO6=AG400+OMCyKSe12paLf64Xi41Z8EA@mail.gmail.com>

Congrats Georg and team! I am incredibly proud of you all for
producing such a great release. As the marketeers would say, "Python
3.3 is the best Python ever!" The feature list is amazing.

--Guido

On Sat, Sep 29, 2012 at 5:18 AM, Georg Brandl <georg at python.org> wrote:
> On behalf of the Python development team, I'm delighted to announce the
> Python 3.3.0 final release.
>
> Python 3.3 includes a range of improvements of the 3.x series, as well
> as easier porting between 2.x and 3.x.  Major new features and changes
> in the 3.3 release series are:
>
> * PEP 380, syntax for delegating to a subgenerator ("yield from")
> * PEP 393, flexible string representation (doing away with the
>    distinction between "wide" and "narrow" Unicode builds)
> * A C implementation of the "decimal" module, with up to 120x speedup
>    for decimal-heavy applications
> * The import system (__import__) now based on importlib by default
> * The new "lzma" module with LZMA/XZ support
> * PEP 397, a Python launcher for Windows
> * PEP 405, virtual environment support in core
> * PEP 420, namespace package support
> * PEP 3151, reworking the OS and IO exception hierarchy
> * PEP 3155, qualified name for classes and functions
> * PEP 409, suppressing exception context
> * PEP 414, explicit Unicode literals to help with porting
> * PEP 418, extended platform-independent clocks in the "time" module
> * PEP 412, a new key-sharing dictionary implementation that
>    significantly saves memory for object-oriented code
> * PEP 362, the function-signature object
> * The new "faulthandler" module that helps diagnosing crashes
> * The new "unittest.mock" module
> * The new "ipaddress" module
> * The "sys.implementation" attribute
> * A policy framework for the email package, with a provisional (see
>    PEP 411) policy that adds much improved unicode support for email
>    header parsing
> * A "collections.ChainMap" class for linking mappings to a single unit
> * Wrappers for many more POSIX functions in the "os" and "signal"
>    modules, as well as other useful functions such as "sendfile()"
> * Hash randomization, introduced in earlier bugfix releases, is now
>    switched on by default
>
> In total, almost 500 API items are new or improved in Python 3.3.
> For a more extensive list of changes in 3.3.0, see
>
>      http://docs.python.org/3.3/whatsnew/3.3.html
>
> To download Python 3.3.0 visit:
>
>      http://www.python.org/download/releases/3.3.0/
>
> This is a production release, please report any bugs to
>
>       http://bugs.python.org/
>
>
> Enjoy!
>
> --
> Georg Brandl, Release Manager
> georg at python.org
> (on behalf of the entire python-dev team and 3.3's contributors)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org



-- 
--Guido van Rossum (python.org/~guido)

From d at davea.name  Sat Sep 29 14:37:16 2012
From: d at davea.name (Dave Angel)
Date: Sat, 29 Sep 2012 08:37:16 -0400
Subject: [Python-Dev] [RELEASED] Python 3.3.0
In-Reply-To: <CANODV3k0Ff84wcDwviNWpdOT8L6H6hnFfaXc9iG8k3So8Hqx0A@mail.gmail.com>
References: <5066E72E.2010100@python.org>
	<CANODV3k0Ff84wcDwviNWpdOT8L6H6hnFfaXc9iG8k3So8Hqx0A@mail.gmail.com>
Message-ID: <5066EB7C.10407@davea.name>

On 09/29/2012 08:23 AM, Amit Saha wrote:
> On Sat, Sep 29, 2012 at 10:18 PM, Georg Brandl <georg at python.org> wrote:
>> <snip>
>>
>> For a more extensive list of changes in 3.3.0, see
>>
>>      http://docs.python.org/3.3/whatsnew/3.3.html
> Redirects to http://docs.python.org/py3k/whatsnew/3.3.html: 404 Not Found.
>
>
Works for me.  Perhaps a momentary glitch.



-- 

DaveA


From python at bdurham.com  Sat Sep 29 17:14:18 2012
From: python at bdurham.com (python at bdurham.com)
Date: Sat, 29 Sep 2012 11:14:18 -0400
Subject: [Python-Dev] [RELEASED] Python 3.3.0
In-Reply-To: <CACac1F8miQd3oKBb3JkCnzuB0KWjBzfqrQ6TNb-__fhFSAHyhg@mail.gmail.com>
References: <5066E72E.2010100@python.org>
	<CAF-Rda-z4LOzT98vS4vpdwP4sNGD5YF6-=JDZgtti9f7H53j0g@mail.gmail.com>
	<CACac1F8miQd3oKBb3JkCnzuB0KWjBzfqrQ6TNb-__fhFSAHyhg@mail.gmail.com>
Message-ID: <1348931658.18629.140661134295829.195F63B3@webmail.messagingengine.com>

> Agreed - this is a really nice release, thanks to all who put it together.

+1

Thank you!
Malcolm

From solipsis at pitrou.net  Sat Sep 29 18:53:41 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 29 Sep 2012 18:53:41 +0200
Subject: [Python-Dev] [RELEASED] Python 3.3.0
References: <5066E72E.2010100@python.org>
Message-ID: <20120929185341.641ab269@pitrou.net>


Hello,

I've created a 3.3 category on the buildbots:
http://buildbot.python.org/3.3/
http://buildbot.python.org/3.3.stable/

Someone will have to update the following HTML page:
http://python.org/dev/buildbot/

Regards

Antoine.



On Sat, 29 Sep 2012 14:18:54 +0200
Georg Brandl <georg at python.org> wrote:
> On behalf of the Python development team, I'm delighted to announce the
> Python 3.3.0 final release.
> 
> Python 3.3 includes a range of improvements of the 3.x series, as well
> as easier porting between 2.x and 3.x.  Major new features and changes
> in the 3.3 release series are:
> 
> * PEP 380, syntax for delegating to a subgenerator ("yield from")
> * PEP 393, flexible string representation (doing away with the
>     distinction between "wide" and "narrow" Unicode builds)
> * A C implementation of the "decimal" module, with up to 120x speedup
>     for decimal-heavy applications
> * The import system (__import__) now based on importlib by default
> * The new "lzma" module with LZMA/XZ support
> * PEP 397, a Python launcher for Windows
> * PEP 405, virtual environment support in core
> * PEP 420, namespace package support
> * PEP 3151, reworking the OS and IO exception hierarchy
> * PEP 3155, qualified name for classes and functions
> * PEP 409, suppressing exception context
> * PEP 414, explicit Unicode literals to help with porting
> * PEP 418, extended platform-independent clocks in the "time" module
> * PEP 412, a new key-sharing dictionary implementation that
>     significantly saves memory for object-oriented code
> * PEP 362, the function-signature object
> * The new "faulthandler" module that helps diagnosing crashes
> * The new "unittest.mock" module
> * The new "ipaddress" module
> * The "sys.implementation" attribute
> * A policy framework for the email package, with a provisional (see
>     PEP 411) policy that adds much improved unicode support for email
>     header parsing
> * A "collections.ChainMap" class for linking mappings to a single unit
> * Wrappers for many more POSIX functions in the "os" and "signal"
>     modules, as well as other useful functions such as "sendfile()"
> * Hash randomization, introduced in earlier bugfix releases, is now
>     switched on by default
> 
> In total, almost 500 API items are new or improved in Python 3.3.
> For a more extensive list of changes in 3.3.0, see
> 
>       http://docs.python.org/3.3/whatsnew/3.3.html
> 
> To download Python 3.3.0 visit:
> 
>       http://www.python.org/download/releases/3.3.0/
> 
> This is a production release, please report any bugs to
> 
>        http://bugs.python.org/
> 
> 
> Enjoy!
> 
> --
> Georg Brandl, Release Manager
> georg at python.org
> (on behalf of the entire python-dev team and 3.3's contributors)



-- 
Software development and contracting: http://pro.pitrou.net



From v+python at g.nevcal.com  Sat Sep 29 19:46:37 2012
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sat, 29 Sep 2012 10:46:37 -0700
Subject: [Python-Dev] [RELEASED] Python 3.3.0
In-Reply-To: <CAP7+vJJmQPfSi+0++RO6=AG400+OMCyKSe12paLf64Xi41Z8EA@mail.gmail.com>
References: <5066E72E.2010100@python.org>
	<CAP7+vJJmQPfSi+0++RO6=AG400+OMCyKSe12paLf64Xi41Z8EA@mail.gmail.com>
Message-ID: <506733FD.6050701@g.nevcal.com>

On Sat, Sep 29, 2012 at 5:18 AM, Georg Brandl <georg at python.org> wrote:
>> In total, almost 500 API items are new or improved in Python 3.3.
>> For a more extensive list of changes in 3.3.0, see
>>
>>       http://docs.python.org/3.3/whatsnew/3.3.html
Reading this to see if I missed anything while downloading the new release:

I found:
> For the common user, this change should result in no visible change in 
> semantics. Any possible changes required in one?s code to handle this 
> change should read the Porting Python code 
> <http://docs.python.org/py3k/whatsnew/3.3.html#porting-python-code> 
> section of this document to see what needs to be changed, but it will 
> only affect those that currently manipulate import or try calling it 
> programmatically.

Sentence two in this paragraph has bizarre structure, probably due to 
being changed from one perspective to another.  Suggestion (which turns 
out to be briefer):

For the common user, this change should result in no visible change in 
semantics. Any code changes required are described in the Porting Python 
code <http://docs.python.org/py3k/whatsnew/3.3.html#porting-python-code> 
section of this document; it will only affect code that currently 
manipulates import or calls it programmatically.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120929/0f65e943/attachment.html>

From benjamin at python.org  Sat Sep 29 19:59:59 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 29 Sep 2012 13:59:59 -0400
Subject: [Python-Dev] str.format bug again
In-Reply-To: <CAPc-aXkWQ0BrkP-aK0oECWkdMM1QanSwPjvpsE=n6UsWm5pswA@mail.gmail.com>
References: <CAPc-aXkWQ0BrkP-aK0oECWkdMM1QanSwPjvpsE=n6UsWm5pswA@mail.gmail.com>
Message-ID: <CAPZV6o-V-a8qDknRd9MmgcG8Novah4nVOK-+MY+vDgw25eq2eA@mail.gmail.com>

2012/9/28 Ben Wolfson <wolfson at gmail.com>:
> There's a patch for this bug: http://bugs.python.org/issue12014 that
> needs reviewed. Petri Lehtinen made some (very minor) comments back in
> May; otherwise it's been neglected. I've brought this up occasionally
> here in the past; it would be great if someone could just give it a
> thumbs up or down (or say what needs to be changed, or whatever).

It seems like Eric Smith is the one needs to make a decision.



-- 
Regards,
Benjamin

From wolfson at gmail.com  Sat Sep 29 20:11:43 2012
From: wolfson at gmail.com (Ben Wolfson)
Date: Sat, 29 Sep 2012 11:11:43 -0700
Subject: [Python-Dev] str.format bug again
In-Reply-To: <CAPZV6o-V-a8qDknRd9MmgcG8Novah4nVOK-+MY+vDgw25eq2eA@mail.gmail.com>
References: <CAPc-aXkWQ0BrkP-aK0oECWkdMM1QanSwPjvpsE=n6UsWm5pswA@mail.gmail.com>
	<CAPZV6o-V-a8qDknRd9MmgcG8Novah4nVOK-+MY+vDgw25eq2eA@mail.gmail.com>
Message-ID: <CAPc-aXkm4n2rX38-qc26U3_=Htkgk-zrRGx6RefyBOOQsSKOog@mail.gmail.com>

On Sat, Sep 29, 2012 at 10:59 AM, Benjamin Peterson <benjamin at python.org> wrote:
> 2012/9/28 Ben Wolfson <wolfson at gmail.com>:
>> There's a patch for this bug: http://bugs.python.org/issue12014 that
>> needs reviewed. Petri Lehtinen made some (very minor) comments back in
>> May; otherwise it's been neglected. I've brought this up occasionally
>> here in the past; it would be great if someone could just give it a
>> thumbs up or down (or say what needs to be changed, or whatever).
>
> It seems like Eric Smith is the one needs to make a decision.

Yes, but one of the things he could decide is that someone else should
make the decision because PEP 420 is too involved (though it was last
modified in June, so who knows).

-- 
Ben Wolfson
"Human kind has used its intelligence to vary the flavour of drinks,
which may be sweet, aromatic, fermented or spirit-based. ... Family
and social life also offer numerous other occasions to consume drinks
for pleasure." [Larousse, "Drink" entry]

From brett at python.org  Sat Sep 29 20:26:50 2012
From: brett at python.org (Brett Cannon)
Date: Sat, 29 Sep 2012 14:26:50 -0400
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <CACac1F8HNLeCSMmmk8-AnL1NWjW=cNQrbLwBk5JJ1q=Mf0hdow@mail.gmail.com>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
	<CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
	<20120929091729.GA4859@sleipnir.bytereef.org>
	<CACac1F8HNLeCSMmmk8-AnL1NWjW=cNQrbLwBk5JJ1q=Mf0hdow@mail.gmail.com>
Message-ID: <CAP1=2W6BmKgZC6kU+gfD-863hufVDbvw-2-zgi-MyME2y5d=zw@mail.gmail.com>

On Sat, Sep 29, 2012 at 9:07 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 29 September 2012 10:17, Stefan Krah <stefan at bytereef.org> wrote:
> > Tim Delaney <timothy.c.delaney at gmail.com> wrote:
> >> If those numbers are similar in other benchmarks, would it be accurate
> and/or
> >> reasonable to include a statement along the lines of:
> >>
> >> "comparable to float performance - usually no more than 3x for
> calculations
> >> within the range of numbers covered by float"
> >
> > For numerical programs, 1.4x (9 digits) to 3x (19 digits) slower would be
> > accurate. On Windows the difference is even less.
> >
> > For output formatting, cdecimal is faster than float (at least it was
> when
> > I posted a benchmark a couple of months ago).
>
> To me, this means that the key point is that for the casual user,
> float is no longer the "obvious" choice. You'd choose float for the
> convenience of a built in type, and Decimal for the more natural
> rounding and precision semantics. If you are sufficiently interested
> in performance for it to matter, you're no longer a "casual" user. (Up
> until now, I'd have said use float unless your need for the better
> behaviour justifies the performance loss - that's no longer the case)
>

Does this mean we want to re-open the discussion about decimal constants?
Last time this came up I think we decided that we wanted to wait for
cdecimal (which is obviously here) and work out how to handle contexts, the
syntax, etc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120929/b49d9830/attachment.html>

From rdmurray at bitdance.com  Sat Sep 29 20:37:21 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Sat, 29 Sep 2012 14:37:21 -0400
Subject: [Python-Dev] [RELEASED] Python 3.3.0
In-Reply-To: <506733FD.6050701@g.nevcal.com>
References: <5066E72E.2010100@python.org>
	<CAP7+vJJmQPfSi+0++RO6=AG400+OMCyKSe12paLf64Xi41Z8EA@mail.gmail.com>
	<506733FD.6050701@g.nevcal.com>
Message-ID: <20120929183721.D766B250054@webabinitio.net>

On Sat, 29 Sep 2012 10:46:37 -0700, Glenn Linderman <v+python at g.nevcal.com> wrote:
> On Sat, Sep 29, 2012 at 5:18 AM, Georg Brandl <georg at python.org> wrote:
> >> In total, almost 500 API items are new or improved in Python 3.3.
> >> For a more extensive list of changes in 3.3.0, see
> >>
> >>       http://docs.python.org/3.3/whatsnew/3.3.html
> Reading this to see if I missed anything while downloading the new release:
> 
> I found:
> > For the common user, this change should result in no visible change in 
> > semantics. Any possible changes required in one???s code to handle this 
> > change should read the Porting Python code 
> > <http://docs.python.org/py3k/whatsnew/3.3.html#porting-python-code> 
> > section of this document to see what needs to be changed, but it will 
> > only affect those that currently manipulate import or try calling it 
> > programmatically.
> 
> Sentence two in this paragraph has bizarre structure, probably due to 
> being changed from one perspective to another.  Suggestion (which turns 
> out to be briefer):
> 
> For the common user, this change should result in no visible change in 
> semantics. Any code changes required are described in the Porting Python 
> code <http://docs.python.org/py3k/whatsnew/3.3.html#porting-python-code> 
> section of this document; it will only affect code that currently 
> manipulates import or calls it programmatically.

I fixed this, though with a different wording change.

--David

From guido at python.org  Sat Sep 29 20:38:21 2012
From: guido at python.org (Guido van Rossum)
Date: Sat, 29 Sep 2012 11:38:21 -0700
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <CAP1=2W6BmKgZC6kU+gfD-863hufVDbvw-2-zgi-MyME2y5d=zw@mail.gmail.com>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
	<CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
	<20120929091729.GA4859@sleipnir.bytereef.org>
	<CACac1F8HNLeCSMmmk8-AnL1NWjW=cNQrbLwBk5JJ1q=Mf0hdow@mail.gmail.com>
	<CAP1=2W6BmKgZC6kU+gfD-863hufVDbvw-2-zgi-MyME2y5d=zw@mail.gmail.com>
Message-ID: <CAP7+vJ+zpbvnHsGGEspeh5znXcMyT_9dbMEjcGtYwN3jC+cKcw@mail.gmail.com>

On Sat, Sep 29, 2012 at 11:26 AM, Brett Cannon <brett at python.org> wrote:
>
>
> On Sat, Sep 29, 2012 at 9:07 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>>
>> On 29 September 2012 10:17, Stefan Krah <stefan at bytereef.org> wrote:
>> > Tim Delaney <timothy.c.delaney at gmail.com> wrote:
>> >> If those numbers are similar in other benchmarks, would it be accurate
>> >> and/or
>> >> reasonable to include a statement along the lines of:
>> >>
>> >> "comparable to float performance - usually no more than 3x for
>> >> calculations
>> >> within the range of numbers covered by float"
>> >
>> > For numerical programs, 1.4x (9 digits) to 3x (19 digits) slower would
>> > be
>> > accurate. On Windows the difference is even less.
>> >
>> > For output formatting, cdecimal is faster than float (at least it was
>> > when
>> > I posted a benchmark a couple of months ago).
>>
>> To me, this means that the key point is that for the casual user,
>> float is no longer the "obvious" choice. You'd choose float for the
>> convenience of a built in type, and Decimal for the more natural
>> rounding and precision semantics. If you are sufficiently interested
>> in performance for it to matter, you're no longer a "casual" user. (Up
>> until now, I'd have said use float unless your need for the better
>> behaviour justifies the performance loss - that's no longer the case)
>
>
> Does this mean we want to re-open the discussion about decimal constants?
> Last time this came up I think we decided that we wanted to wait for
> cdecimal (which is obviously here) and work out how to handle contexts, the
> syntax, etc.

I think that ought to be a Python 4 feature if we ever want it to be a
feature. And I'm not saying this to kill the discussion; I just think
it will be a huge change that we have to consider very carefully.
While the existing float definitely has problems for beginners, it is
incredibly useful for advanced users. Consider e.g. the implications
for numpy / scipy, one of the fastest-growing specialized Python user
communities.

Now if you want to introduce a new notation for decimals, e.g. 3.14d
and 1e42d, that would be a fine thing. (Should we also have complex
decimals? 1jd anyone?)

-- 
--Guido van Rossum (python.org/~guido)

From rosuav at gmail.com  Sat Sep 29 20:39:03 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 30 Sep 2012 04:39:03 +1000
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <CAP1=2W6BmKgZC6kU+gfD-863hufVDbvw-2-zgi-MyME2y5d=zw@mail.gmail.com>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
	<CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
	<20120929091729.GA4859@sleipnir.bytereef.org>
	<CACac1F8HNLeCSMmmk8-AnL1NWjW=cNQrbLwBk5JJ1q=Mf0hdow@mail.gmail.com>
	<CAP1=2W6BmKgZC6kU+gfD-863hufVDbvw-2-zgi-MyME2y5d=zw@mail.gmail.com>
Message-ID: <CAPTjJmrrzrRmqwaB5X1jGZhru8vqbEHX7MxvOEVZG8jvND5Rrw@mail.gmail.com>

On Sun, Sep 30, 2012 at 4:26 AM, Brett Cannon <brett at python.org> wrote:
> Does this mean we want to re-open the discussion about decimal constants?
> Last time this came up I think we decided that we wanted to wait for
> cdecimal (which is obviously here) and work out how to handle contexts, the
> syntax, etc.

Just to throw a crazy idea out: How bad a change would it be to make
decimal actually the default?

(Caveat: I've not worked with decimal/cdecimal to any real extent and
don't know its limitations etc.)

ChrisA

From eric at trueblade.com  Sat Sep 29 20:51:26 2012
From: eric at trueblade.com (Eric V. Smith)
Date: Sat, 29 Sep 2012 14:51:26 -0400
Subject: [Python-Dev] str.format bug again
In-Reply-To: <CAPc-aXkm4n2rX38-qc26U3_=Htkgk-zrRGx6RefyBOOQsSKOog@mail.gmail.com>
References: <CAPc-aXkWQ0BrkP-aK0oECWkdMM1QanSwPjvpsE=n6UsWm5pswA@mail.gmail.com>
	<CAPZV6o-V-a8qDknRd9MmgcG8Novah4nVOK-+MY+vDgw25eq2eA@mail.gmail.com>
	<CAPc-aXkm4n2rX38-qc26U3_=Htkgk-zrRGx6RefyBOOQsSKOog@mail.gmail.com>
Message-ID: <305134B1-6234-437A-98BC-4A0079E3F668@trueblade.com>

It's on my list of things to look at. I have a project due next week, then I'll have some time.

--
Eric.

On Sep 29, 2012, at 2:11 PM, Ben Wolfson <wolfson at gmail.com> wrote:

> On Sat, Sep 29, 2012 at 10:59 AM, Benjamin Peterson <benjamin at python.org> wrote:
>> 2012/9/28 Ben Wolfson <wolfson at gmail.com>:
>>> There's a patch for this bug: http://bugs.python.org/issue12014 that
>>> needs reviewed. Petri Lehtinen made some (very minor) comments back in
>>> May; otherwise it's been neglected. I've brought this up occasionally
>>> here in the past; it would be great if someone could just give it a
>>> thumbs up or down (or say what needs to be changed, or whatever).
>> 
>> It seems like Eric Smith is the one needs to make a decision.
> 
> Yes, but one of the things he could decide is that someone else should
> make the decision because PEP 420 is too involved (though it was last
> modified in June, so who knows).
> 
> -- 
> Ben Wolfson
> "Human kind has used its intelligence to vary the flavour of drinks,
> which may be sweet, aromatic, fermented or spirit-based. ... Family
> and social life also offer numerous other occasions to consume drinks
> for pleasure." [Larousse, "Drink" entry]
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com
> 

From greg at krypto.org  Sat Sep 29 21:51:47 2012
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 29 Sep 2012 12:51:47 -0700
Subject: [Python-Dev] make decimal the default non-integer instead of
	float?
Message-ID: <CAGE7PNKQpVXixWq_5xSQHHDBzc_f4Cw109PMvye4ePZVA=X3+A@mail.gmail.com>

-cc: python-dev
+cc: python-ideas

On Sat, Sep 29, 2012 at 11:39 AM, Chris Angelico <rosuav at gmail.com> wrote:

> On Sun, Sep 30, 2012 at 4:26 AM, Brett Cannon <brett at python.org> wrote:
> > Does this mean we want to re-open the discussion about decimal constants?
> > Last time this came up I think we decided that we wanted to wait for
> > cdecimal (which is obviously here) and work out how to handle contexts,
> the
> > syntax, etc.
>
> Just to throw a crazy idea out: How bad a change would it be to make
> decimal actually the default?
>
> (Caveat: I've not worked with decimal/cdecimal to any real extent and
> don't know its limitations etc.)
>

Painful for existing code, unittests and extension modules.  Definitely
python-ideas territory (thread moved there with an appropriate subject).

I'm not surprised at all that a decimal type can be "fast" in an
interpreted language due to the already dominant interpreter overhead.

I wish all spreadsheets had used decimals from day one rather than binary
floating point (blame Lotus?). Think of the trouble that would have saved
the world.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120929/b91d156f/attachment.html>

From timothy.c.delaney at gmail.com  Sat Sep 29 23:51:58 2012
From: timothy.c.delaney at gmail.com (Tim Delaney)
Date: Sun, 30 Sep 2012 07:51:58 +1000
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <CAPTjJmrrzrRmqwaB5X1jGZhru8vqbEHX7MxvOEVZG8jvND5Rrw@mail.gmail.com>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
	<CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
	<20120929091729.GA4859@sleipnir.bytereef.org>
	<CACac1F8HNLeCSMmmk8-AnL1NWjW=cNQrbLwBk5JJ1q=Mf0hdow@mail.gmail.com>
	<CAP1=2W6BmKgZC6kU+gfD-863hufVDbvw-2-zgi-MyME2y5d=zw@mail.gmail.com>
	<CAPTjJmrrzrRmqwaB5X1jGZhru8vqbEHX7MxvOEVZG8jvND5Rrw@mail.gmail.com>
Message-ID: <CAN8CLgmmfVVxmxU8nSme9uphOoYAeVXMjd81mPHMSzoQsV53Ng@mail.gmail.com>

BTW, "What's New": http://www.python.org/download/releases/3.3.0/ still
says 80x for decimal performance.

Tim Delaney
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/a4e36069/attachment.html>

From timothy.c.delaney at gmail.com  Sun Sep 30 00:01:00 2012
From: timothy.c.delaney at gmail.com (Tim Delaney)
Date: Sun, 30 Sep 2012 08:01:00 +1000
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <CAN8CLgmmfVVxmxU8nSme9uphOoYAeVXMjd81mPHMSzoQsV53Ng@mail.gmail.com>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
	<CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
	<20120929091729.GA4859@sleipnir.bytereef.org>
	<CACac1F8HNLeCSMmmk8-AnL1NWjW=cNQrbLwBk5JJ1q=Mf0hdow@mail.gmail.com>
	<CAP1=2W6BmKgZC6kU+gfD-863hufVDbvw-2-zgi-MyME2y5d=zw@mail.gmail.com>
	<CAPTjJmrrzrRmqwaB5X1jGZhru8vqbEHX7MxvOEVZG8jvND5Rrw@mail.gmail.com>
	<CAN8CLgmmfVVxmxU8nSme9uphOoYAeVXMjd81mPHMSzoQsV53Ng@mail.gmail.com>
Message-ID: <CAN8CLgn803rgskFpLd95U=qdwMihAKTYZBi63MKWkKwE7L04Zw@mail.gmail.com>

Also the example at
http://docs.python.org/py3k/whatsnew/3.3.html#pep-409-suppressing-exception-contextreads:

...             raise AttributeError(attr) from None...

Looks like there's an elipsis there that shouldn't be.

Tim Delaney
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/4758c42f/attachment.html>

From nad at acm.org  Sun Sep 30 00:48:11 2012
From: nad at acm.org (Ned Deily)
Date: Sat, 29 Sep 2012 15:48:11 -0700
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
	<CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
	<20120929091729.GA4859@sleipnir.bytereef.org>
	<CACac1F8HNLeCSMmmk8-AnL1NWjW=cNQrbLwBk5JJ1q=Mf0hdow@mail.gmail.com>
	<CAP1=2W6BmKgZC6kU+gfD-863hufVDbvw-2-zgi-MyME2y5d=zw@mail.gmail.com>
	<CAPTjJmrrzrRmqwaB5X1jGZhru8vqbEHX7MxvOEVZG8jvND5Rrw@mail.gmail.com>
	<CAN8CLgmmfVVxmxU8nSme9uphOoYAeVXMjd81mPHMSzoQsV53Ng@mail.gmail.com>
Message-ID: <nad-C375FE.15481129092012@news.gmane.org>

In article 
<CAN8CLgmmfVVxmxU8nSme9uphOoYAeVXMjd81mPHMSzoQsV53Ng at mail.gmail.com>,
 Tim Delaney <timothy.c.delaney at gmail.com> wrote:
> BTW, "What's New": http://www.python.org/download/releases/3.3.0/ still
> says 80x for decimal performance.

Thanks for the report.  The page has now been updated to match the final 
3.3.0 release announcement post.

-- 
 Ned Deily,
 nad at acm.org


From brett at python.org  Sun Sep 30 02:14:20 2012
From: brett at python.org (Brett Cannon)
Date: Sat, 29 Sep 2012 20:14:20 -0400
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <CAP7+vJ+zpbvnHsGGEspeh5znXcMyT_9dbMEjcGtYwN3jC+cKcw@mail.gmail.com>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
	<CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
	<20120929091729.GA4859@sleipnir.bytereef.org>
	<CACac1F8HNLeCSMmmk8-AnL1NWjW=cNQrbLwBk5JJ1q=Mf0hdow@mail.gmail.com>
	<CAP1=2W6BmKgZC6kU+gfD-863hufVDbvw-2-zgi-MyME2y5d=zw@mail.gmail.com>
	<CAP7+vJ+zpbvnHsGGEspeh5znXcMyT_9dbMEjcGtYwN3jC+cKcw@mail.gmail.com>
Message-ID: <CAP1=2W4cBjpW6uHtJPAn7HqGXrW74mhigutnfEnEcWnTZm0XKw@mail.gmail.com>

On Sep 29, 2012 2:38 PM, "Guido van Rossum" <guido at python.org> wrote:
>
> On Sat, Sep 29, 2012 at 11:26 AM, Brett Cannon <brett at python.org> wrote:
> >
> >
> > On Sat, Sep 29, 2012 at 9:07 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> >>
> >> On 29 September 2012 10:17, Stefan Krah <stefan at bytereef.org> wrote:
> >> > Tim Delaney <timothy.c.delaney at gmail.com> wrote:
> >> >> If those numbers are similar in other benchmarks, would it be
accurate
> >> >> and/or
> >> >> reasonable to include a statement along the lines of:
> >> >>
> >> >> "comparable to float performance - usually no more than 3x for
> >> >> calculations
> >> >> within the range of numbers covered by float"
> >> >
> >> > For numerical programs, 1.4x (9 digits) to 3x (19 digits) slower
would
> >> > be
> >> > accurate. On Windows the difference is even less.
> >> >
> >> > For output formatting, cdecimal is faster than float (at least it was
> >> > when
> >> > I posted a benchmark a couple of months ago).
> >>
> >> To me, this means that the key point is that for the casual user,
> >> float is no longer the "obvious" choice. You'd choose float for the
> >> convenience of a built in type, and Decimal for the more natural
> >> rounding and precision semantics. If you are sufficiently interested
> >> in performance for it to matter, you're no longer a "casual" user. (Up
> >> until now, I'd have said use float unless your need for the better
> >> behaviour justifies the performance loss - that's no longer the case)
> >
> >
> > Does this mean we want to re-open the discussion about decimal
constants?
> > Last time this came up I think we decided that we wanted to wait for
> > cdecimal (which is obviously here) and work out how to handle contexts,
the
> > syntax, etc.
>
> I think that ought to be a Python 4 feature if we ever want it to be a
> feature. And I'm not saying this to kill the discussion; I just think
> it will be a huge change that we have to consider very carefully.
> While the existing float definitely has problems for beginners, it is
> incredibly useful for advanced users. Consider e.g. the implications
> for numpy / scipy, one of the fastest-growing specialized Python user
> communities.
>
> Now if you want to introduce a new notation for decimals, e.g. 3.14d
> and 1e42d, that would be a fine thing. (Should we also have complex
> decimals? 1jd anyone?)

That's actually what I meant and not replacing floats (unless a command
line flag was used); sorry if that wasn't clear.

-brett

>
> --
> --Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120929/909965ce/attachment-0001.html>

From zuo at chopin.edu.pl  Sun Sep 30 02:43:39 2012
From: zuo at chopin.edu.pl (Jan Kaliszewski)
Date: Sun, 30 Sep 2012 02:43:39 +0200
Subject: [Python-Dev] =?utf-8?b?IkRlY2ltYWwoMikgIT0gZmxvYXQoMikiPz8/?=
Message-ID: <ea4f3f44203528476ea7f9672fc9cf1b@chopin.edu.pl>

Hello,

In http://docs.python.org/release/3.2.3/reference/expressions.html#in 
we read: "[...] This can create the illusion of non-transitivity between 
supported cross-type comparisons and unsupported comparisons. For 
example, Decimal(2) == 2 and 2 == float(2) but Decimal(2) != float(2)."

(The same is in the 3.3 docs).

But:

     Python 3.2.3 (default, Sep 10 2012, 18:14:40)
     [GCC 4.6.3] on linux2
     Type "help", "copyright", "credits" or "license" for more 
information.
     >>> import decimal
     >>> decimal.Decimal(2) == float(2)
     True

Is it a bug in the docs or in Python itself? (I checked that in 3.2, 
but it may be true for 3.3 as well)

Regards.
*j


From python at mrabarnett.plus.com  Sun Sep 30 03:03:11 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 30 Sep 2012 02:03:11 +0100
Subject: [Python-Dev] "Decimal(2) != float(2)"???
In-Reply-To: <ea4f3f44203528476ea7f9672fc9cf1b@chopin.edu.pl>
References: <ea4f3f44203528476ea7f9672fc9cf1b@chopin.edu.pl>
Message-ID: <50679A4F.2010509@mrabarnett.plus.com>

On 2012-09-30 01:43, Jan Kaliszewski wrote:
> Hello,
>
> In http://docs.python.org/release/3.2.3/reference/expressions.html#in
> we read: "[...] This can create the illusion of non-transitivity between
> supported cross-type comparisons and unsupported comparisons. For
> example, Decimal(2) == 2 and 2 == float(2) but Decimal(2) != float(2)."
>
> (The same is in the 3.3 docs).
>
> But:
>
>       Python 3.2.3 (default, Sep 10 2012, 18:14:40)
>       [GCC 4.6.3] on linux2
>       Type "help", "copyright", "credits" or "license" for more
> information.
>       >>> import decimal
>       >>> decimal.Decimal(2) == float(2)
>       True
>
> Is it a bug in the docs or in Python itself? (I checked that in 3.2,
> but it may be true for 3.3 as well)
>
It's the same in Python 3.3:

 >>> decimal.Decimal(2) == float(2)
True

Also:

 >>> decimal.Decimal(0.1) == 0.1
True
 >>> decimal.Decimal("0.1") == 0.1
False

This is because floats work in binary:

 >>> decimal.Decimal(0.1) # from a float
Decimal('0.1000000000000000055511151231257827021181583404541015625')
 >>> decimal.Decimal("0.1")
Decimal('0.1')


From steve at pearwood.info  Sun Sep 30 05:48:07 2012
From: steve at pearwood.info (Steven D'Aprano)
Date: Sun, 30 Sep 2012 13:48:07 +1000
Subject: [Python-Dev] "Decimal(2) != float(2)"???
In-Reply-To: <ea4f3f44203528476ea7f9672fc9cf1b@chopin.edu.pl>
References: <ea4f3f44203528476ea7f9672fc9cf1b@chopin.edu.pl>
Message-ID: <5067C0F7.3050804@pearwood.info>

On 30/09/12 10:43, Jan Kaliszewski wrote:
> Hello,
>
> In http://docs.python.org/release/3.2.3/reference/expressions.html#in we
>read: "[...] This can create the illusion of non-transitivity between
>supported cross-type comparisons and unsupported comparisons. For example,
>Decimal(2) == 2 and 2 == float(2) but Decimal(2) != float(2)."
[...]
> Is it a bug in the docs or in Python itself? (I checked that in 3.2, but it may be true for 3.3 as well)

Documentation bug. It used to be the case that Decimal and float did not compare equal:

steve at runes:~$ python3.1
Python 3.1.3 (r313:86834, Nov 28 2010, 11:28:10)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
py> from decimal import Decimal
py> Decimal(2) == 2.0
False

but starting in 3.2 they do. But of course there are traps for the unwary,
due to binary floats, e.g. Decimal("0.1") != 0.1



-- 
Steven

From rdmurray at bitdance.com  Sun Sep 30 07:09:14 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 30 Sep 2012 01:09:14 -0400
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <CAN8CLgn803rgskFpLd95U=qdwMihAKTYZBi63MKWkKwE7L04Zw@mail.gmail.com>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
	<CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
	<20120929091729.GA4859@sleipnir.bytereef.org>
	<CACac1F8HNLeCSMmmk8-AnL1NWjW=cNQrbLwBk5JJ1q=Mf0hdow@mail.gmail.com>
	<CAP1=2W6BmKgZC6kU+gfD-863hufVDbvw-2-zgi-MyME2y5d=zw@mail.gmail.com>
	<CAPTjJmrrzrRmqwaB5X1jGZhru8vqbEHX7MxvOEVZG8jvND5Rrw@mail.gmail.com>
	<CAN8CLgmmfVVxmxU8nSme9uphOoYAeVXMjd81mPHMSzoQsV53Ng@mail.gmail.com>
	<CAN8CLgn803rgskFpLd95U=qdwMihAKTYZBi63MKWkKwE7L04Zw@mail.gmail.com>
Message-ID: <20120930050916.214FE250054@webabinitio.net>

On Sun, 30 Sep 2012 08:01:00 +1000, Tim Delaney <timothy.c.delaney at gmail.com> wrote:
> Also the example at
> http://docs.python.org/py3k/whatsnew/3.3.html#pep-409-suppressing-exception-contextreads:
> 
> ...             raise AttributeError(attr) from None...
> 
> Looks like there's an elipsis there that shouldn't be.

This appears to be a Sphinx bug of some sort.  The ReST source
is correct.

--David

From chris.jerdonek at gmail.com  Sun Sep 30 07:17:59 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sat, 29 Sep 2012 22:17:59 -0700
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <20120930050916.214FE250054@webabinitio.net>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
	<CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
	<20120929091729.GA4859@sleipnir.bytereef.org>
	<CACac1F8HNLeCSMmmk8-AnL1NWjW=cNQrbLwBk5JJ1q=Mf0hdow@mail.gmail.com>
	<CAP1=2W6BmKgZC6kU+gfD-863hufVDbvw-2-zgi-MyME2y5d=zw@mail.gmail.com>
	<CAPTjJmrrzrRmqwaB5X1jGZhru8vqbEHX7MxvOEVZG8jvND5Rrw@mail.gmail.com>
	<CAN8CLgmmfVVxmxU8nSme9uphOoYAeVXMjd81mPHMSzoQsV53Ng@mail.gmail.com>
	<CAN8CLgn803rgskFpLd95U=qdwMihAKTYZBi63MKWkKwE7L04Zw@mail.gmail.com>
	<20120930050916.214FE250054@webabinitio.net>
Message-ID: <CAOTb1weXkGzFSNSaDSVLHefK1x=QZYCv6xHJTTp7RX9MbH4Zfg@mail.gmail.com>

On Sat, Sep 29, 2012 at 10:09 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> On Sun, 30 Sep 2012 08:01:00 +1000, Tim Delaney <timothy.c.delaney at gmail.com> wrote:
>> Also the example at
>> http://docs.python.org/py3k/whatsnew/3.3.html#pep-409-suppressing-exception-contextreads:
>>
>> ...             raise AttributeError(attr) from None...
>>
>> Looks like there's an elipsis there that shouldn't be.
>
> This appears to be a Sphinx bug of some sort.  The ReST source
> is correct.

Yes, this was previously discussed here:

http://mail.python.org/pipermail/python-dev/2012-August/121467.html

where Georg wrote:

> this is fixed in the latest Pygments, and will be fine in the doc once I
> update its version used for building.  Until then, you could disable
> syntax highlighting on that particular code block.

--Chris

From tjreedy at udel.edu  Sun Sep 30 07:44:42 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 30 Sep 2012 01:44:42 -0400
Subject: [Python-Dev] "Decimal(2) != float(2)"???
In-Reply-To: <5067C0F7.3050804@pearwood.info>
References: <ea4f3f44203528476ea7f9672fc9cf1b@chopin.edu.pl>
	<5067C0F7.3050804@pearwood.info>
Message-ID: <k48m8b$f7s$2@ger.gmane.org>

On 9/29/2012 11:48 PM, Steven D'Aprano wrote:
> On 30/09/12 10:43, Jan Kaliszewski wrote:
>> Hello,
>>
>> In http://docs.python.org/release/3.2.3/reference/expressions.html#in we
>> read: "[...] This can create the illusion of non-transitivity between
>> supported cross-type comparisons and unsupported comparisons. For
>> example,
>> Decimal(2) == 2 and 2 == float(2) but Decimal(2) != float(2)."
> [...]
>> Is it a bug in the docs or in Python itself? (I checked that in 3.2,
>> but it may be true for 3.3 as well)
>
> Documentation bug. It used to be the case that Decimal and float did not
> compare equal:

Questions about past releases are better directed to python-list (where 
Steven would have given same answer ;-). But anyway, please open a doc 
issue on the tracker to update that item.

> steve at runes:~$ python3.1
> Python 3.1.3 (r313:86834, Nov 28 2010, 11:28:10)
> [GCC 4.4.5] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> py> from decimal import Decimal
> py> Decimal(2) == 2.0
> False
>
> but starting in 3.2 they do. But of course there are traps for the unwary,
> due to binary floats, e.g. Decimal("0.1") != 0.1
>
>
>


-- 
Terry Jan Reedy


From tjreedy at udel.edu  Sun Sep 30 08:18:50 2012
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 30 Sep 2012 02:18:50 -0400
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <CAP7+vJ+zpbvnHsGGEspeh5znXcMyT_9dbMEjcGtYwN3jC+cKcw@mail.gmail.com>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
	<CACac1F-ZZkXVfq+pvP11kp+GraB9f8s4R+5H45zt4X5MsTXfsQ@mail.gmail.com>
	<CAN8CLgmXz=khKpkzmO35+9RxVY4PmqSJ7KxEkNwaf1BnjnqoxA@mail.gmail.com>
	<20120929091729.GA4859@sleipnir.bytereef.org>
	<CACac1F8HNLeCSMmmk8-AnL1NWjW=cNQrbLwBk5JJ1q=Mf0hdow@mail.gmail.com>
	<CAP1=2W6BmKgZC6kU+gfD-863hufVDbvw-2-zgi-MyME2y5d=zw@mail.gmail.com>
	<CAP7+vJ+zpbvnHsGGEspeh5znXcMyT_9dbMEjcGtYwN3jC+cKcw@mail.gmail.com>
Message-ID: <k48o8b$s2v$1@ger.gmane.org>

On 9/29/2012 2:38 PM, Guido van Rossum wrote:

>> Does this mean we want to re-open the discussion about decimal constants?
>> Last time this came up I think we decided that we wanted to wait for
>> cdecimal (which is obviously here) and work out how to handle contexts, the
>> syntax, etc.
>
> I think that ought to be a Python 4 feature if we ever want it to be a
> feature. And I'm not saying this to kill the discussion; I just think
> it will be a huge change that we have to consider very carefully.
> While the existing float definitely has problems for beginners, it is
> incredibly useful for advanced users. Consider e.g. the implications
> for numpy / scipy, one of the fastest-growing specialized Python user
> communities.

It is also one of, if not *the* oldest application communities.

> Now if you want to introduce a new notation for decimals, e.g. 3.14d
> and 1e42d, that would be a fine thing. (Should we also have complex
> decimals? 1jd anyone?)

I think not. The money community does not use complexes that I know of, 
and complex decimals would not be very useful without a complex decimal 
module (and 3rd party modules). Even the complex float operations and 
cmath library could use work to touch up and test corner cases.

I can imagine giving IDLE a calulator mode. If Python itself had a 
startup switch, Idle could just restart the remote process. If a new 
suffix is used, Idle could add the suffix as literal numbers are entered.

-- 
Terry Jan Reedy


From stefan at bytereef.org  Sun Sep 30 09:25:48 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Sun, 30 Sep 2012 09:25:48 +0200
Subject: [Python-Dev] [RELEASED] Python 3.3.0 release candidate 3
In-Reply-To: <20120928181925.GA21350@sleipnir.bytereef.org>
References: <505FFB47.30901@python.org>
	<20120928154042.GA19170@sleipnir.bytereef.org>
	<CAP1=2W4b-rac=Dx_BpMOJ5ubkyCu+AUoQS_h9hzDwsp9c-b_-A@mail.gmail.com>
	<20120928181925.GA21350@sleipnir.bytereef.org>
Message-ID: <20120930072548.GA17275@sleipnir.bytereef.org>

Stefan Krah <stefan at bytereef.org> wrote:
> Precision: 19 decimal digits
> 
> float:
> result: 3.1415926535897927
> time: 0.112874s
> 
> cdecimal:
> result: 3.141592653589793236
> time: 0.348100s
> 
> decimal:
> result: 3.141592653589793236
> time: 43.241220s

Apparently there were concerns about the correctness of the result on some
message board. To be clear: This is a *benchmark* designed to execute the
same code for float, cdecimal and decimal.

The *original* algorithm from the decimal docs uses a higher intermediate
context precision. Since floats do not have such a context, this step was
deliberately dropped from the benchmark.


Stefan Krah



From lukasz at langa.pl  Sun Sep 30 11:32:39 2012
From: lukasz at langa.pl (=?iso-8859-2?Q?=A3ukasz_Langa?=)
Date: Sun, 30 Sep 2012 11:32:39 +0200
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <50660E2B.5000209@gmail.com>
References: <5064D399.5050208@netwok.org> <50660E2B.5000209@gmail.com>
Message-ID: <5496F2E4-C36D-409E-9C11-8DA6F9ADD1CE@langa.pl>

Wiadomo?? napisana przez Maciej Szulik <soltysh at gmail.com> w dniu 28 wrz 2012, o godz. 22:52:

> On 09/28/2012 12:30 AM, ?ric Araujo wrote:
>> Hi all,
>> 
>> The Montreal-Python user group would like to host a bug day on October
>> 27 (to be confirmed) at a partner university in Montreal.  It would be
>> cool to do a bug day on IRC like we used to (and in other physical
>> locations if people want to!) to get new contributors and close bugs.
>> What do you think?
>> 
> 
> Great idea, I'll talk to ?ukasz Langa about possibilities to gather
> a group of Poles interested in this topic.

Why not, why not :)

-- 
Best regards,
?ukasz Langa
Senior Systems Architecture Engineer

IT Infrastructure Department
Grupa Allegro Sp. z o.o.

http://lukasz.langa.pl/
+48 791 080 144

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/75042cf8/attachment.html>

From kristjan at ccpgames.com  Sun Sep 30 12:55:54 2012
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Sun, 30 Sep 2012 10:55:54 +0000
Subject: [Python-Dev] =?iso-8859-1?q?what=B4s_new_3=2E3?=
Message-ID: <EFE3877620384242A686D52278B7CCD361F66E@RKV-IT-EXCH104.ccp.ad.local>

Hi there.

Not having kept up, I realized I failed to contribute to the What's new thingie.

Here's stuff I remember working on and putting in:

  1.
pickling support for built in iterators (#14288)
  2.
inter process socket duplication for windows (#14310)
  3.
Progress callback for gc module (#10576)
  4.
Faster locking on windows (#15038)



DiSome of this should probably be mentioned in the What's new document, even if only in its online version.

K


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/4de8d3db/attachment.html>

From g.brandl at gmx.net  Sun Sep 30 13:26:28 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 30 Sep 2012 13:26:28 +0200
Subject: [Python-Dev] [RELEASED] Python 3.3.0
In-Reply-To: <CAP7+vJJmQPfSi+0++RO6=AG400+OMCyKSe12paLf64Xi41Z8EA@mail.gmail.com>
References: <5066E72E.2010100@python.org>
	<CAP7+vJJmQPfSi+0++RO6=AG400+OMCyKSe12paLf64Xi41Z8EA@mail.gmail.com>
Message-ID: <k49a92$ltj$1@ger.gmane.org>

Thanks. It's really a team effort: a little digging in the hg history says
that:

* 86 people have committed during the 3.3 development
* 70 during 3.2 development and
* 55 during 3.1 development

No surprise the feature list is so long...

cheers,
Georg

On 09/29/2012 05:52 PM, Guido van Rossum wrote:
> Congrats Georg and team! I am incredibly proud of you all for
> producing such a great release. As the marketeers would say, "Python
> 3.3 is the best Python ever!" The feature list is amazing.
>
> --Guido
>
> On Sat, Sep 29, 2012 at 5:18 AM, Georg Brandl <georg at python.org> wrote:
>> On behalf of the Python development team, I'm delighted to announce the
>> Python 3.3.0 final release.
>>
>> Python 3.3 includes a range of improvements of the 3.x series, as well
>> as easier porting between 2.x and 3.x.  Major new features and changes
>> in the 3.3 release series are:
>>
>> * PEP 380, syntax for delegating to a subgenerator ("yield from")
>> * PEP 393, flexible string representation (doing away with the
>>    distinction between "wide" and "narrow" Unicode builds)
>> * A C implementation of the "decimal" module, with up to 120x speedup
>>    for decimal-heavy applications
>> * The import system (__import__) now based on importlib by default
>> * The new "lzma" module with LZMA/XZ support
>> * PEP 397, a Python launcher for Windows
>> * PEP 405, virtual environment support in core
>> * PEP 420, namespace package support
>> * PEP 3151, reworking the OS and IO exception hierarchy
>> * PEP 3155, qualified name for classes and functions
>> * PEP 409, suppressing exception context
>> * PEP 414, explicit Unicode literals to help with porting
>> * PEP 418, extended platform-independent clocks in the "time" module
>> * PEP 412, a new key-sharing dictionary implementation that
>>    significantly saves memory for object-oriented code
>> * PEP 362, the function-signature object
>> * The new "faulthandler" module that helps diagnosing crashes
>> * The new "unittest.mock" module
>> * The new "ipaddress" module
>> * The "sys.implementation" attribute
>> * A policy framework for the email package, with a provisional (see
>>    PEP 411) policy that adds much improved unicode support for email
>>    header parsing
>> * A "collections.ChainMap" class for linking mappings to a single unit
>> * Wrappers for many more POSIX functions in the "os" and "signal"
>>    modules, as well as other useful functions such as "sendfile()"
>> * Hash randomization, introduced in earlier bugfix releases, is now
>>    switched on by default
>>
>> In total, almost 500 API items are new or improved in Python 3.3.
>> For a more extensive list of changes in 3.3.0, see
>>
>>      http://docs.python.org/3.3/whatsnew/3.3.html
>>
>> To download Python 3.3.0 visit:
>>
>>      http://www.python.org/download/releases/3.3.0/
>>
>> This is a production release, please report any bugs to
>>
>>       http://bugs.python.org/
>>
>>
>> Enjoy!
>>
>> --
>> Georg Brandl, Release Manager
>> georg at python.org
>> (on behalf of the entire python-dev team and 3.3's contributors)
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>



From phd at phdru.name  Sun Sep 30 14:01:04 2012
From: phd at phdru.name (Oleg Broytman)
Date: Sun, 30 Sep 2012 16:01:04 +0400
Subject: [Python-Dev] [RELEASED] Python 3.3.0
In-Reply-To: <k49a92$ltj$1@ger.gmane.org>
References: <5066E72E.2010100@python.org>
	<CAP7+vJJmQPfSi+0++RO6=AG400+OMCyKSe12paLf64Xi41Z8EA@mail.gmail.com>
	<k49a92$ltj$1@ger.gmane.org>
Message-ID: <20120930120104.GA6615@iskra.aviel.ru>

On Sun, Sep 30, 2012 at 01:26:28PM +0200, Georg Brandl <g.brandl at gmx.net> wrote:
> Thanks. It's really a team effort: a little digging in the hg history says
> that:
> 
> * 86 people have committed during the 3.3 development
> * 70 during 3.2 development and
> * 55 during 3.1 development
> 
> No surprise the feature list is so long...

   Many kudos to the team and to all contributors!

   Linux Weekly News regularly publishes tables "Who done what in Linux
Kernel": http://lwn.net/Articles/507986/
http://lwn.net/SubscriberLink/517564/bec11e6ace6ad699/

   It would be interesting to see tables like these for Python.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From regebro at gmail.com  Sun Sep 30 14:47:28 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 30 Sep 2012 14:47:28 +0200
Subject: [Python-Dev] Stdlib and timezones, again
Message-ID: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>

With 3.3 out, it's time to bring up something for 3.4. And it's about pytz
and stdlib, basically. And we have been over that again, but I have a
proposal anyway.

The problem with including pytz in the stdlib is that it contains the
tz/zoneinfo/Olson database, and it updates much more often than Python
does. I propose to solve this problem in the following way:

1. Merge the functionality of pytz into the stdlib datetime module. That
means extending datetime.tzinfo with the localize() function and and adding
a top-level datetime.timezone() function. any other extra functionality
pytz has.

2. The datetime.timezone() function will work as the pytz.timezone()
function, ie return a tzinfo object from the tz database.

3. However, Python will not include the database. Instead it will use the
one that the OS provides, if it provides one (ie Unices, including OS X if
I remember correctly).

4. On any other OS, datetime.timezone() will return an error explaining
that no tz database can be found, but that it is available in the module
'pytzdata' which then will need to be created. If this module is installed,
the datetime.timezone() function will prefer that database before the OS
database.

5. We can also consider implementing something like the get_localzone()
function of my module tzlocal.

This solves two problems.

i. We have real, useful timezone support in Python. For Unix, it's
included, for all other systems, it's just an extra module. The extra
module can also be installed to give Python up to date tz information even
when the operating system is outdated and unsupported and no
longer receiving updates.

ii. pytz is_dst flag becomes included in stdlib, solving one of the few
imperfections in the current library.

What do you say? Is this a path worth pursuing?

//Lennart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/1216ad01/attachment.html>

From dirkjan at ochtman.nl  Sun Sep 30 14:54:38 2012
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Sun, 30 Sep 2012 14:54:38 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
Message-ID: <CAKmKYaC-USfmnsAqs8Yee436+L_2AwzbJiTCvvk=i-a4-D0NKQ@mail.gmail.com>

On Sun, Sep 30, 2012 at 2:47 PM, Lennart Regebro <regebro at gmail.com> wrote:
> What do you say? Is this a path worth pursuing?

+1. It's the kind of low-level thing that should be solved in the
stdlib as far as possible, and the pytz interface is as stable as the
stdlib's.

Cheers,

Dirkjan

From christian at python.org  Sun Sep 30 14:56:54 2012
From: christian at python.org (Christian Heimes)
Date: Sun, 30 Sep 2012 14:56:54 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
Message-ID: <50684196.6050908@python.org>

Am 30.09.2012 14:47, schrieb Lennart Regebro:
> What do you say? Is this a path worth pursuing?

+1

I'm eager to read your PEP!


From solipsis at pitrou.net  Sun Sep 30 15:03:45 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 30 Sep 2012 15:03:45 +0200
Subject: [Python-Dev] Stdlib and timezones, again
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
Message-ID: <20120930150345.4ad47c57@pitrou.net>

On Sun, 30 Sep 2012 14:47:28 +0200
Lennart Regebro <regebro at gmail.com> wrote:
> With 3.3 out, it's time to bring up something for 3.4. And it's about pytz
> and stdlib, basically. And we have been over that again, but I have a
> proposal anyway.
> 
> The problem with including pytz in the stdlib is that it contains the
> tz/zoneinfo/Olson database, and it updates much more often than Python
> does. I propose to solve this problem in the following way:
> 
> 1. Merge the functionality of pytz into the stdlib datetime module. That
> means extending datetime.tzinfo with the localize() function and and adding
> a top-level datetime.timezone() function. any other extra functionality
> pytz has.
> 
> 2. The datetime.timezone() function will work as the pytz.timezone()
> function, ie return a tzinfo object from the tz database.
> 
> 3. However, Python will not include the database. Instead it will use the
> one that the OS provides, if it provides one (ie Unices, including OS X if
> I remember correctly).
> 
> 4. On any other OS, datetime.timezone() will return an error explaining
> that no tz database can be found, but that it is available in the module
> 'pytzdata' which then will need to be created. If this module is installed,
> the datetime.timezone() function will prefer that database before the OS
> database.

Can't we simply include the Olson database in Windows installers?

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From solipsis at pitrou.net  Sun Sep 30 15:06:02 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 30 Sep 2012 15:06:02 +0200
Subject: [Python-Dev] =?iso-8859-1?q?what=B4s_new_3=2E3?=
References: <EFE3877620384242A686D52278B7CCD361F66E@RKV-IT-EXCH104.ccp.ad.local>
Message-ID: <20120930150602.4785b03e@pitrou.net>

On Sun, 30 Sep 2012 10:55:54 +0000
Kristj?n Valur J?nsson <kristjan at ccpgames.com> wrote:
> Hi there.
> 
> Not having kept up, I realized I failed to contribute to the What's new thingie.
[...]
> DiSome of this should probably be mentioned in the What's new document, even if only in its online version.

Well, it's not too late for you to fix that oversight.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From dirkjan at ochtman.nl  Sun Sep 30 15:10:06 2012
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Sun, 30 Sep 2012 15:10:06 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20120930150345.4ad47c57@pitrou.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
Message-ID: <CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>

On Sun, Sep 30, 2012 at 3:03 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Can't we simply include the Olson database in Windows installers?

We probably can, but the problem is that it's updated quite often (for
example, in 2011, there were about 14 releases; in 2009, there were
21). So you'd want to have a mechanism to override the data that is
included in the stdlib.

Cheers,

Dirkjan

From solipsis at pitrou.net  Sun Sep 30 15:15:33 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 30 Sep 2012 15:15:33 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
Message-ID: <20120930151533.38fa85fd@pitrou.net>

On Sun, 30 Sep 2012 15:10:06 +0200
Dirkjan Ochtman <dirkjan at ochtman.nl> wrote:
> On Sun, Sep 30, 2012 at 3:03 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > Can't we simply include the Olson database in Windows installers?
> 
> We probably can, but the problem is that it's updated quite often (for
> example, in 2011, there were about 14 releases; in 2009, there were
> 21). So you'd want to have a mechanism to override the data that is
> included in the stdlib.

Probably, but for most purposes I would guess a 2-year old database is
still good enough? After all, you don't see many people complaining
about the outdated Unicode database that is hard-wired in past Pythons.

Regards

Antoine.

From g.brandl at gmx.net  Sun Sep 30 15:17:36 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 30 Sep 2012 15:17:36 +0200
Subject: [Python-Dev] experimental: Misc/NEWS included in docs
Message-ID: <k49gpe$2dt$1@ger.gmane.org>

Hi,

at http://docs.python.org/3.3/whatsnew/news.html, there is now
a rendering of Misc/NEWS with tracker links and a crude filtering
capability.  I thought that this will complement the "whatsnew"
documents nicely for people looking for more detail.

Please let me know if it's useful, or what could be done to make
it *more* useful.

cheers,
Georg


From ned at nedbatchelder.com  Sun Sep 30 15:22:17 2012
From: ned at nedbatchelder.com (Ned Batchelder)
Date: Sun, 30 Sep 2012 09:22:17 -0400
Subject: [Python-Dev] Python Bug Day in October
In-Reply-To: <5064D399.5050208@netwok.org>
References: <5064D399.5050208@netwok.org>
Message-ID: <50684789.5060109@nedbatchelder.com>

On 9/27/2012 6:30 PM, ?ric Araujo wrote:
> Hi all,
>
> The Montreal-Python user group would like to host a bug day on October
> 27 (to be confirmed) at a partner university in Montreal.  It would be
> cool to do a bug day on IRC like we used to (and in other physical
> locations if people want to!) to get new contributors and close bugs.
> What do you think?
Boston Python might get a physical location and join in.  Keep us posted!

--Ned.

> Cheers
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com
>


From g.brandl at gmx.net  Sun Sep 30 15:19:50 2012
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 30 Sep 2012 15:19:50 +0200
Subject: [Python-Dev] [RELEASED] Python 3.3.0
In-Reply-To: <20120929185341.641ab269@pitrou.net>
References: <5066E72E.2010100@python.org> <20120929185341.641ab269@pitrou.net>
Message-ID: <k49gtk$2dt$2@ger.gmane.org>

On 09/29/2012 06:53 PM, Antoine Pitrou wrote:
>
> Hello,
>
> I've created a 3.3 category on the buildbots:
> http://buildbot.python.org/3.3/
> http://buildbot.python.org/3.3.stable/
>
> Someone will have to update the following HTML page:
> http://python.org/dev/buildbot/

Should be done now.

Georg



From python-dev at masklinn.net  Sun Sep 30 15:30:04 2012
From: python-dev at masklinn.net (Xavier Morel)
Date: Sun, 30 Sep 2012 15:30:04 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20120930151533.38fa85fd@pitrou.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
Message-ID: <7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>

On 2012-09-30, at 15:15 , Antoine Pitrou wrote:

> On Sun, 30 Sep 2012 15:10:06 +0200
> Dirkjan Ochtman <dirkjan at ochtman.nl> wrote:
>> On Sun, Sep 30, 2012 at 3:03 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>> Can't we simply include the Olson database in Windows installers?
>> 
>> We probably can, but the problem is that it's updated quite often (for
>> example, in 2011, there were about 14 releases; in 2009, there were
>> 21). So you'd want to have a mechanism to override the data that is
>> included in the stdlib.
> 
> Probably, but for most purposes I would guess a 2-year old database is
> still good enough? After all, you don't see many people complaining
> about the outdated Unicode database that is hard-wired in past Pythons.

But at worst, an outdated unicode database will be missing data right?

Doesn't an outdated timezone db have the risk of returning *incorrect* data?

From rosuav at gmail.com  Sun Sep 30 15:32:01 2012
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 30 Sep 2012 23:32:01 +1000
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20120930151533.38fa85fd@pitrou.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
Message-ID: <CAPTjJmrYj5+3KyW-qxDR3PL_iDcJBrW0hCpJ8taEVdBQKS05qQ@mail.gmail.com>

On Sun, Sep 30, 2012 at 11:15 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sun, 30 Sep 2012 15:10:06 +0200
> Dirkjan Ochtman <dirkjan at ochtman.nl> wrote:
>> On Sun, Sep 30, 2012 at 3:03 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> > Can't we simply include the Olson database in Windows installers?
>>
>> We probably can, but the problem is that it's updated quite often (for
>> example, in 2011, there were about 14 releases; in 2009, there were
>> 21). So you'd want to have a mechanism to override the data that is
>> included in the stdlib.
>
> Probably, but for most purposes I would guess a 2-year old database is
> still good enough? After all, you don't see many people complaining
> about the outdated Unicode database that is hard-wired in past Pythons.

An out-of-date timezone database will bite someone at some point.
They'll be running code that expects some locality to be on/off DST
when it isn't, and some customer will be confused or annoyed. I
suspect most people won't much care, though. It's only a few who'd
explicitly upgrade their tz database.

ChrisA

From trent at snakebite.org  Sun Sep 30 16:20:55 2012
From: trent at snakebite.org (Trent Nelson)
Date: Sun, 30 Sep 2012 10:20:55 -0400
Subject: [Python-Dev] buildbot with -O
In-Reply-To: <CAPZV6o-NAcrUm9z4cpP=jn+FcKcfoKGeCB9q1kaH0dhi9rMnEw@mail.gmail.com>
References: <CAPZV6o-NAcrUm9z4cpP=jn+FcKcfoKGeCB9q1kaH0dhi9rMnEw@mail.gmail.com>
Message-ID: <20120930141932.GA26599@snakebite.org>

    Just as an FYI, we set up a new "optimized" Snakebite slave
    yesterday on the Mac OS X Mountain Lion box.  I also set up
    remote access to that box in the process -- you can connect
    to it via the ~/.snakebite/sb 'x8' alias.

    (That box happens to be the fastest (and most contemporary)
     Snakebite machine; Mac Pro circa late 2009, dual quad core
     3.2GHz Xenons (12MB L2 cache on each CPU!), 32GB RAM. See,
     it's not just a network of old esoteric junk, honest ;-).)

    Regards,

        Trent.

On Wed, Sep 26, 2012 at 04:00:53PM -0700, Benjamin Peterson wrote:
> In light of issue #16046, I think it would be helpful to have a
> buildbot running the testsuite with -O enabled.
>
> -- 
> Regards,
> Benjamin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/trent%40snakebite.org

From rdmurray at bitdance.com  Sun Sep 30 16:30:29 2012
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 30 Sep 2012 10:30:29 -0400
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
Message-ID: <20120930143029.811922500FA@webabinitio.net>

On Sun, 30 Sep 2012 15:30:04 +0200, Xavier Morel <python-dev at masklinn.net> wrote:
> On 2012-09-30, at 15:15 , Antoine Pitrou wrote:
> 
> > On Sun, 30 Sep 2012 15:10:06 +0200
> > Dirkjan Ochtman <dirkjan at ochtman.nl> wrote:
> >> On Sun, Sep 30, 2012 at 3:03 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> >>> Can't we simply include the Olson database in Windows installers?
> >> 
> >> We probably can, but the problem is that it's updated quite often (for
> >> example, in 2011, there were about 14 releases; in 2009, there were
> >> 21). So you'd want to have a mechanism to override the data that is
> >> included in the stdlib.
> > 
> > Probably, but for most purposes I would guess a 2-year old database is
> > still good enough? After all, you don't see many people complaining
> > about the outdated Unicode database that is hard-wired in past Pythons.
> 
> But at worst, an outdated unicode database will be missing data right?
> 
> Doesn't an outdated timezone db have the risk of returning *incorrect* data?

Yes.  And the timezeone database is much more volatile than the unicode
database.  The changes are, after all, driven almost entirely by
politics.

--David

From stefan at bytereef.org  Sun Sep 30 16:39:34 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Sun, 30 Sep 2012 16:39:34 +0200
Subject: [Python-Dev] [Python-checkins] cpython (3.3): Change libmpdec
 to use ANSI code in strict ansi mode as inline asm isn't
In-Reply-To: <3XV7Mn2CknzPH2@mail.python.org>
References: <3XV7Mn2CknzPH2@mail.python.org>
Message-ID: <20120930143934.GA21303@sleipnir.bytereef.org>

christian.heimes <python-checkins at python.org> wrote:
> +/* ASM isn't available in strict ansi C mode */
> +#if defined(ASM) && defined(__STRICT_ANSI__)
> +  #undef ASM
> +  #define ANSI
> +#endif

I'm not too fond of this solution: the libmpdec build output should always
be accurate. Now you have ...

   -ansi -DCONFIG_64=1 -DASM=1

... but the build is silently non-asm.


Like for pymath.c, __asm__ is tested for and selected in ./configure. So I'll
replace asm with __asm__, same as in pymath.c and ceval.c.


Stefan Krah




From benjamin at python.org  Sun Sep 30 17:33:55 2012
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 30 Sep 2012 11:33:55 -0400
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
Message-ID: <CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>

2012/9/30 Xavier Morel <python-dev at masklinn.net>:
> But at worst, an outdated unicode database will be missing data right?
>
> Doesn't an outdated timezone db have the risk of returning *incorrect* data?

Unicode updates also include corrections; however, it seems there are
not significant enough or about obscure enough scripts that not many
notice. :)



-- 
Regards,
Benjamin

From stefan at bytereef.org  Sun Sep 30 18:04:24 2012
From: stefan at bytereef.org (Stefan Krah)
Date: Sun, 30 Sep 2012 18:04:24 +0200
Subject: [Python-Dev] [Python-checkins] cpython (3.3): Change libmpdec
 to use ANSI code in strict ansi mode as inline asm isn't
In-Reply-To: <20120930143934.GA21303@sleipnir.bytereef.org>
References: <3XV7Mn2CknzPH2@mail.python.org>
	<20120930143934.GA21303@sleipnir.bytereef.org>
Message-ID: <20120930160424.GA22046@sleipnir.bytereef.org>

If anyone wants to force either an ANSI build or a specific libmpdec
configuration, setup.py recognizes this environment variable (example):

   export PYTHON_DECIMAL_WITH_MACHINE=ansi64


The supported configurations listed here:

   http://hg.python.org/cpython/file/07d9d2901adb/Modules/_decimal/README.txt



Stefan Krah



From __peter__ at web.de  Sun Sep 30 18:36:20 2012
From: __peter__ at web.de (Peter Otten)
Date: Sun, 30 Sep 2012 18:36:20 +0200
Subject: [Python-Dev] experimental: Misc/NEWS included in docs
References: <k49gpe$2dt$1@ger.gmane.org>
Message-ID: <k49se2$q2k$1@ger.gmane.org>

Georg Brandl wrote:

> at http://docs.python.org/3.3/whatsnew/news.html, there is now
> a rendering of Misc/NEWS with tracker links and a crude filtering
> capability.  I thought that this will complement the "whatsnew"
> documents nicely for people looking for more detail.
> 
> Please let me know if it's useful, or what could be done to make
> it *more* useful.

Maybe you could supress those sections that become empty when a filter is 
applied.


From greg at krypto.org  Sun Sep 30 20:18:26 2012
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 30 Sep 2012 11:18:26 -0700
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
	<7D496514-D6B7-46E8-90EC-27BE7CC4191E@masklinn.net>
	<CAPZV6o-jmfHb6sKChjUAGNVG9rqD+6NnWYX6eoxqByMUvspCvA@mail.gmail.com>
Message-ID: <CAGE7PNK0o0oXnp52NB62MXsO9jFVX6O0TPfnG5pYgUEwxLue+w@mail.gmail.com>

On Sun, Sep 30, 2012 at 8:33 AM, Benjamin Peterson <benjamin at python.org>wrote:

> 2012/9/30 Xavier Morel <python-dev at masklinn.net>:
> > But at worst, an outdated unicode database will be missing data right?
> >
> > Doesn't an outdated timezone db have the risk of returning *incorrect*
> data?
>
> Unicode updates also include corrections; however, it seems there are
> not significant enough or about obscure enough scripts that not many
> notice. :)
>

We never hear anyone complain because the corrections are not for English
or other "western" languages that the majority of us speak.  ;)

Regardless, I think including a version of the database on windows releases
makes sense.  Update it on a best effort basis before each .x minor
release.  The documentation can make it clear that it is a changing
database how to use an updated version with your application.

One additional thing I'd like to see: Don't let the successful importing of
a 'pytzdata' module be the only way to get an updated tz database.  Create
an API for someone to supply one themselves at runtime that is cleaner than
shoving something into sys.modules['pytzdata'].  And define in which order
they'll be used:

priority:
  1) api call supplying tz data to the process.
  2) pytzdata module if it exists
  3) tz data from the underlying operating system
  4) error.

-gps

PS Unicode data is political as well, just less politically active than
stupid time zones and "savings" times.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/ab12d30d/attachment.html>

From storchaka at gmail.com  Sun Sep 30 20:55:34 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 30 Sep 2012 21:55:34 +0300
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20120930151533.38fa85fd@pitrou.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net>
Message-ID: <k4a4jd$p8p$1@ger.gmane.org>

On 30.09.12 16:15, Antoine Pitrou wrote:
> Probably, but for most purposes I would guess a 2-year old database is
> still good enough? After all, you don't see many people complaining
> about the outdated Unicode database that is hard-wired in past Pythons.

In 2011 Ukrainian timezone data was changed twice for year. And perhaps 
even change in 2013. Russian timezones were changed over the last few 
years and most likely will change in the near future. Correct time is 
critical for many applications.


From chris.jerdonek at gmail.com  Sun Sep 30 21:14:37 2012
From: chris.jerdonek at gmail.com (Chris Jerdonek)
Date: Sun, 30 Sep 2012 12:14:37 -0700
Subject: [Python-Dev] experimental: Misc/NEWS included in docs
In-Reply-To: <k49gpe$2dt$1@ger.gmane.org>
References: <k49gpe$2dt$1@ger.gmane.org>
Message-ID: <CAOTb1wd364Yc00JHVSkB9ahK1LOa4J8N+9Ve76_i6HMc7yFLOA@mail.gmail.com>

On Sun, Sep 30, 2012 at 6:17 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> at http://docs.python.org/3.3/whatsnew/news.html, there is now
> a rendering of Misc/NEWS with tracker links and a crude filtering
> capability.  I thought that this will complement the "whatsnew"
> documents nicely for people looking for more detail.
>
> Please let me know if it's useful, or what could be done to make
> it *more* useful.

Good idea.  Some suggestions:

* A note at the top explaining what this page is and how it differs
from the other What's New page (including perhaps a link to the other
What's New page).  It might also be good if the two had more
distinctive titles.  They are currently "What?s New in Python" and
"Python News."

* It would be good if the section links could somehow be
"perma-links."  They will currently vary over time, e.g.
  http://docs.python.org/py3k/whatsnew/news.html#id3

* Lastly, I think skimming this list would be more informative to me
at least if each bullet began with the name of the affected module or
area (at least in the case where just one module is affected).  The
issue number is less meaningful as a descriptive header.  Changing
this would likely require changes in how we prepare Misc/NEWS though.

On a separate but related note, I never know where to put the various
subsections in relation to one another in Misc/NEWS (Core and
Builtins, Library, Extension Modules, etc).  It would be good if there
was guidance on that -- even if to say it completely doesn't matter.

--Chris

From solipsis at pitrou.net  Sun Sep 30 21:24:30 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 30 Sep 2012 21:24:30 +0200
Subject: [Python-Dev] Stdlib and timezones, again
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net> <k4a4jd$p8p$1@ger.gmane.org>
Message-ID: <20120930212430.0caecbce@pitrou.net>

On Sun, 30 Sep 2012 21:55:34 +0300
Serhiy Storchaka <storchaka at gmail.com> wrote:
> On 30.09.12 16:15, Antoine Pitrou wrote:
> > Probably, but for most purposes I would guess a 2-year old database is
> > still good enough? After all, you don't see many people complaining
> > about the outdated Unicode database that is hard-wired in past Pythons.
> 
> In 2011 Ukrainian timezone data was changed twice for year. And perhaps 
> even change in 2013. Russian timezones were changed over the last few 
> years and most likely will change in the near future. Correct time is 
> critical for many applications.

Perhaps, but that's the responsibility of governements.
Just because some governments have erratic policies shouldn't be a
reason for residents of other countries not to enjoy the benefits of
their stable timezones.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From storchaka at gmail.com  Sun Sep 30 21:35:54 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sun, 30 Sep 2012 22:35:54 +0300
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20120930212430.0caecbce@pitrou.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net> <k4a4jd$p8p$1@ger.gmane.org>
	<20120930212430.0caecbce@pitrou.net>
Message-ID: <k4a6v2$beu$1@ger.gmane.org>

On 30.09.12 22:24, Antoine Pitrou wrote:
> Just because some governments have erratic policies shouldn't be a
> reason for residents of other countries not to enjoy the benefits of
> their stable timezones.

This is similar to ascii-chauvinism. ;-)



From solipsis at pitrou.net  Sun Sep 30 21:51:52 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 30 Sep 2012 21:51:52 +0200
Subject: [Python-Dev] Stdlib and timezones, again
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net> <k4a4jd$p8p$1@ger.gmane.org>
	<20120930212430.0caecbce@pitrou.net> <k4a6v2$beu$1@ger.gmane.org>
Message-ID: <20120930215152.48ae89c4@pitrou.net>

On Sun, 30 Sep 2012 22:35:54 +0300
Serhiy Storchaka <storchaka at gmail.com> wrote:
> On 30.09.12 22:24, Antoine Pitrou wrote:
> > Just because some governments have erratic policies shouldn't be a
> > reason for residents of other countries not to enjoy the benefits of
> > their stable timezones.
> 
> This is similar to ascii-chauvinism. ;-)

Well, no, this isn't similar. Choosing one's timezone policies is a
contemporary political decision, while choosing a language and its
alphabet is not really a decision people ever make (it's just an aspect
of a society's long-term evolution) - except Atat?rk, perhaps :-)

Furthermore, the proposal I'm making does *not* disadvantage residents
of Russia and Ukraine: whether our Windows installer provides a database
or not, they have to download a new database if they want up-to-date
information. And they have to download it afresh every few months, if
I'm following you.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From python at mrabarnett.plus.com  Sun Sep 30 21:53:42 2012
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 30 Sep 2012 20:53:42 +0100
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <k4a4jd$p8p$1@ger.gmane.org>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net> <k4a4jd$p8p$1@ger.gmane.org>
Message-ID: <5068A346.50509@mrabarnett.plus.com>

On 2012-09-30 19:55, Serhiy Storchaka wrote:
> On 30.09.12 16:15, Antoine Pitrou wrote:
>> Probably, but for most purposes I would guess a 2-year old database is
>> still good enough? After all, you don't see many people complaining
>> about the outdated Unicode database that is hard-wired in past Pythons.
>
> In 2011 Ukrainian timezone data was changed twice for year. And perhaps
> even change in 2013. Russian timezones were changed over the last few
> years and most likely will change in the near future. Correct time is
> critical for many applications.
>
That's what UTC is for! :-)

I think that it would be a good idea to provide a database with the
release plus a tool for updating it, the updates being announced by
email or RSS, and the ability to use the system's database if there's
one.

From brett at python.org  Sun Sep 30 22:12:34 2012
From: brett at python.org (Brett Cannon)
Date: Sun, 30 Sep 2012 16:12:34 -0400
Subject: [Python-Dev] experimental: Misc/NEWS included in docs
In-Reply-To: <CAOTb1wd364Yc00JHVSkB9ahK1LOa4J8N+9Ve76_i6HMc7yFLOA@mail.gmail.com>
References: <k49gpe$2dt$1@ger.gmane.org>
	<CAOTb1wd364Yc00JHVSkB9ahK1LOa4J8N+9Ve76_i6HMc7yFLOA@mail.gmail.com>
Message-ID: <CAP1=2W67t4o+_QcJi9Xui6VGrz+FtzT8FjKALTrk4XqWGB+49Q@mail.gmail.com>

On Sun, Sep 30, 2012 at 3:14 PM, Chris Jerdonek <chris.jerdonek at gmail.com>wrote:

> On Sun, Sep 30, 2012 at 6:17 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> > at http://docs.python.org/3.3/whatsnew/news.html, there is now
> > a rendering of Misc/NEWS with tracker links and a crude filtering
> > capability.  I thought that this will complement the "whatsnew"
> > documents nicely for people looking for more detail.
> >
> > Please let me know if it's useful, or what could be done to make
> > it *more* useful.
>
> Good idea.  Some suggestions:
>
> * A note at the top explaining what this page is and how it differs
> from the other What's New page (including perhaps a link to the other
> What's New page).  It might also be good if the two had more
> distinctive titles.  They are currently "What?s New in Python" and
> "Python News."
>
> * It would be good if the section links could somehow be
> "perma-links."  They will currently vary over time, e.g.
>   http://docs.python.org/py3k/whatsnew/news.html#id3
>
> * Lastly, I think skimming this list would be more informative to me
> at least if each bullet began with the name of the affected module or
> area (at least in the case where just one module is affected).  The
> issue number is less meaningful as a descriptive header.  Changing
> this would likely require changes in how we prepare Misc/NEWS though.
>
> On a separate but related note, I never know where to put the various
> subsections in relation to one another in Misc/NEWS (Core and
> Builtins, Library, Extension Modules, etc).  It would be good if there
> was guidance on that -- even if to say it completely doesn't matter.
>

It doesn't matter. When I've done it I have either copied the empty
sections previously or not even bothered with the empty sections and just
let others add as necessary.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/7a95d811/attachment.html>

From breamoreboy at yahoo.co.uk  Sun Sep 30 22:26:58 2012
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 30 Sep 2012 21:26:58 +0100
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <5068A346.50509@mrabarnett.plus.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net> <k4a4jd$p8p$1@ger.gmane.org>
	<5068A346.50509@mrabarnett.plus.com>
Message-ID: <k4a9q6$hi$1@ger.gmane.org>

On 30/09/2012 20:53, MRAB wrote:
> That's what UTC is for! :-)
>
> I think that it would be a good idea to provide a database with the
> release plus a tool for updating it, the updates being announced by
> email or RSS, and the ability to use the system's database if there's
> one.

Very naive extremely simple approach reference it on pypi?!?!

-- 
Cheers.

Mark Lawrence.


From solipsis at pitrou.net  Sun Sep 30 22:31:20 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 30 Sep 2012 22:31:20 +0200
Subject: [Python-Dev] experimental: Misc/NEWS included in docs
References: <k49gpe$2dt$1@ger.gmane.org>
	<CAOTb1wd364Yc00JHVSkB9ahK1LOa4J8N+9Ve76_i6HMc7yFLOA@mail.gmail.com>
Message-ID: <20120930223120.2c2e22c2@pitrou.net>

On Sun, 30 Sep 2012 12:14:37 -0700
Chris Jerdonek <chris.jerdonek at gmail.com> wrote:

> On Sun, Sep 30, 2012 at 6:17 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> > at http://docs.python.org/3.3/whatsnew/news.html, there is now
> > a rendering of Misc/NEWS with tracker links and a crude filtering
> > capability.  I thought that this will complement the "whatsnew"
> > documents nicely for people looking for more detail.
> >
> > Please let me know if it's useful, or what could be done to make
> > it *more* useful.
> 
> Good idea.  Some suggestions:
> 
> * A note at the top explaining what this page is and how it differs
> from the other What's New page (including perhaps a link to the other
> What's New page).  It might also be good if the two had more
> distinctive titles.  They are currently "What?s New in Python" and
> "Python News."

Agreed. I think the titles are currently very confusing. The page
you've just added should be called something like "Changelog", to
stress its more bare'n'technical natural compared to our official
"What's New" pages.

Also it would be extra nice if the "What's new in Python X.Y" section
headers could be renamed to "Python X.Y changelog", or similar.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net



From guido at python.org  Sun Sep 30 22:33:29 2012
From: guido at python.org (Guido van Rossum)
Date: Sun, 30 Sep 2012 13:33:29 -0700
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <5068A346.50509@mrabarnett.plus.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net> <k4a4jd$p8p$1@ger.gmane.org>
	<5068A346.50509@mrabarnett.plus.com>
Message-ID: <CAP7+vJL=hHw8syzL6LwcAq75_g=x2=wzqXAa76bxUSdPaovQKA@mail.gmail.com>

>
> When people use pytz they have to reinstall pytz too if thet want to
> benefit from the updates. (Or depend on automated updates via some vendor
> package management system.) If we can ensure that with pytz in the stdlib,
> updates to the Olson database can be installed just as easily as before
> (probably through some new custom mechanism, as a new Python bugfix release
> is too heavy-handed), I see no problem with adopting pytz into the stdlib.


--Guido


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/ca4ef4e8/attachment.html>

From regebro at gmail.com  Sun Sep 30 22:56:34 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 30 Sep 2012 22:56:34 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <CAP7+vJL=hHw8syzL6LwcAq75_g=x2=wzqXAa76bxUSdPaovQKA@mail.gmail.com>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net> <k4a4jd$p8p$1@ger.gmane.org>
	<5068A346.50509@mrabarnett.plus.com>
	<CAP7+vJL=hHw8syzL6LwcAq75_g=x2=wzqXAa76bxUSdPaovQKA@mail.gmail.com>
Message-ID: <CAL0kPAX1buZKmk+zBgZup3Dhc1hVAw2--SPnQxkH2pm9nMmFBA@mail.gmail.com>

On Sep 30, 2012 10:34 PM, "Guido van Rossum" <guido at python.org> wrote:
>>
>> When people use pytz they have to reinstall pytz too if thet want to
benefit from the updates. (Or depend on automated updates via some vendor
package management system.) If we can ensure that with pytz in the stdlib,
updates to the Olson database can be installed just as easily as before
(probably through some new custom mechanism, as a new Python bugfix release
is too heavy-handed), I see no problem with adopting pytz into the stdlib.

The mechanism would be exactly the same, but a different modulename.

I?ll go forward and make a PEP on this soonish.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/9b3096de/attachment.html>

From storchaka at gmail.com  Sun Sep 30 23:28:41 2012
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 01 Oct 2012 00:28:41 +0300
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <20120930215152.48ae89c4@pitrou.net>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net> <k4a4jd$p8p$1@ger.gmane.org>
	<20120930212430.0caecbce@pitrou.net> <k4a6v2$beu$1@ger.gmane.org>
	<20120930215152.48ae89c4@pitrou.net>
Message-ID: <k4adic$r0b$1@ger.gmane.org>

On 30.09.12 22:51, Antoine Pitrou wrote:
> Well, no, this isn't similar. Choosing one's timezone policies is a
> contemporary political decision, while choosing a language and its
> alphabet is not really a decision people ever make (it's just an aspect
> of a society's long-term evolution) - except Atat?rk, perhaps :-)

Oh, no. Choosing of alphabet (and sometimes language) is also a 
contemporary political decision. For the last 25 years new letter ? has 
been added to the Ukrainian alphabet, and the letter ? changed its place 
in the alphabet. There were at least 4 family of absolutely different 
character sets for Ukrainian (not counting the Unicode), some of them 
contains several incompatible variants. In several neighboring countries 
the alphabet was changed completely (from Cyrillic-based to Latin). Why 
ASCII is not enough for all?

> Furthermore, the proposal I'm making does *not* disadvantage residents
> of Russia and Ukraine: whether our Windows installer provides a database
> or not, they have to download a new database if they want up-to-date
> information. And they have to download it afresh every few months, if
> I'm following you.

Who will update the database? The developer which distributes the 
application with embedded Python can forget about the tz updates, as 
well as about non-ascii encodings. Native Unicode support in Python 
makes the second error less likely.

Why not use the system data which are updated by the OS? I know that 
Windows also changes the clock for local DST.


From regebro at gmail.com  Sun Sep 30 23:39:59 2012
From: regebro at gmail.com (Lennart Regebro)
Date: Sun, 30 Sep 2012 23:39:59 +0200
Subject: [Python-Dev] Stdlib and timezones, again
In-Reply-To: <k4adic$r0b$1@ger.gmane.org>
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net> <k4a4jd$p8p$1@ger.gmane.org>
	<20120930212430.0caecbce@pitrou.net> <k4a6v2$beu$1@ger.gmane.org>
	<20120930215152.48ae89c4@pitrou.net> <k4adic$r0b$1@ger.gmane.org>
Message-ID: <CAL0kPAVKUDGUa5CdiTwaRtwQEwfqZrv6hYPq8akwE6SLXBdN7A@mail.gmail.com>

On Sun, Sep 30, 2012 at 11:28 PM, Serhiy Storchaka <storchaka at gmail.com>wrote:

> Why not use the system data which are updated by the OS? I know that
> Windows also changes the clock for local DST.
>

The Windows timezone information does not include any historical
information, as it's designed primarily to keep your computers clock
correctly, not to convert date times between different timezones. As a
result using to to convert data in the past can give incorrect information.
We therefore want to consistently use the tz database.

//Lennart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120930/9c19fe2b/attachment.html>

From solipsis at pitrou.net  Sun Sep 30 23:43:25 2012
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 30 Sep 2012 23:43:25 +0200
Subject: [Python-Dev] Stdlib and timezones, again
References: <CAL0kPAXmWVqZCWUaCT3OktGSvX5q_t4OHGETFEFPqvSfRhUKeQ@mail.gmail.com>
	<20120930150345.4ad47c57@pitrou.net>
	<CAKmKYaBCcn_YE0d++ZAz+6x3+2Qxxbmqd=AcLS=wOHv_EOOFyA@mail.gmail.com>
	<20120930151533.38fa85fd@pitrou.net> <k4a4jd$p8p$1@ger.gmane.org>
	<20120930212430.0caecbce@pitrou.net> <k4a6v2$beu$1@ger.gmane.org>
	<20120930215152.48ae89c4@pitrou.net> <k4adic$r0b$1@ger.gmane.org>
Message-ID: <20120930234325.53796bf9@pitrou.net>

On Mon, 01 Oct 2012 00:28:41 +0300
Serhiy Storchaka <storchaka at gmail.com> wrote:
> On 30.09.12 22:51, Antoine Pitrou wrote:
> > Well, no, this isn't similar. Choosing one's timezone policies is a
> > contemporary political decision, while choosing a language and its
> > alphabet is not really a decision people ever make (it's just an aspect
> > of a society's long-term evolution) - except Atat?rk, perhaps :-)
> 
> Oh, no. Choosing of alphabet (and sometimes language) is also a 
> contemporary political decision. For the last 25 years new letter ? has 
> been added to the Ukrainian alphabet, and the letter ? changed its place 
> in the alphabet.

Well, yeah, but it's not like you can do it on a whim either, and you
can't change the large body of existing text.

> Who will update the database? The developer which distributes the 
> application with embedded Python can forget about the tz updates, as 
> well as about non-ascii encodings. Native Unicode support in Python 
> makes the second error less likely.
> 
> Why not use the system data which are updated by the OS? I know that 
> Windows also changes the clock for local DST.

If that's possible, then it sounds the ideal solution indeed.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net